# eMotion-GAN: A Motion-based GAN for Photorealistic and Facial Expression Preserving Frontal View Synthesis

Omar Ikne, Benjamin Allaert, Ioan Marius Bilasco, Hazem Wannous

► To cite this version:

Omar Ikne, Benjamin Allaert, Ioan Marius Bilasco, Hazem Wannous. eMotion-GAN: A Motion-based GAN for Photorealistic and Facial Expression Preserving Frontal View Synthesis. Computer Vision and Image Understanding, 2025, 262, pp.104555. 10.1016/j.cviu.2025.104555. hal-05477740

HAL Id: hal-05477740

https://hal.science/hal-05477740v1

Submitted on 26 Jan 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# eMotion-GAN: A Motion-based GAN for Photorealistic and Facial Expression Preserving Frontal View Synthesis

Omar Ikne, Benjamin Allaert, Ioan Marius Bilasco, Hazem Wannous,

**Abstract**—Facial expression recognition (FER) systems frequently suffer significant performance degradation when confronted with head pose variations, a pervasive challenge in real-world applications ranging from healthcare monitoring to human-computer interaction. While existing frontal view synthesis (FVS) methods attempt to address this issue, they predominantly operate in the appearance domain, often introducing artifacts that distort the subtle motion patterns crucial for accurate expression analysis. We present eMotion-GAN, a two-stage generative motion-domain framework that fundamentally rethinks frontalization by decomposing facial dynamics into two distinct components: (1) expression-related motion stemming from muscle activity, and (2) pose-related motion acting as noise. We conducted extensive evaluations using several widely recognized dynamic FER datasets, which encompass sequences exhibiting various degrees of head pose variations in both intensity and orientation. Our results demonstrate the effectiveness of our approach in significantly reducing the FER performance gap between frontal and non-frontal faces. Specifically, we achieved a FER improvement of up to +5% for small pose variations and up to +20% improvement for larger pose variations. Code and pre-trained models are available at: https://github.com/o-ikne/eMotion-GAN.git.

**Index Terms**—Facial expressions, frontal view synthesis, optical flow, motion warping, generative adversarial networks.

## I. INTRODUCTION

Facial Expression Recognition (FER) plays a vital role in human-centered computing, enabling applications from assistive healthcare to emotion-aware robotics [1]–[5]. While current methods achieve strong performance in controlled settings with frontal poses [6], [7], they struggle significantly in unconstrained environments where natural head movements occur [8]. This pose sensitivity remains a fundamental challenge for real-world deployment, particularly in critical scenarios like video surveillance and human-robot interaction.

FER can be performed in both image (texture/geometry) and motion domains. However, the image domain is more commonly used, as deep learning models can be trained on large-scale datasets. Nonetheless, FER models trained on images have shown a significant decrease in accuracy in the presence of head pose variations [1], [10]. Although various Frontal View Synthesis (FVS) methods have been proposed

<figure><img src="image_2.png" /><figcaption>Fig. 1. eMotion-GAN estimates the motion induced by the variation of the head pose (column 5), to keep only the motion induced by the facial muscles. The latter is then transposed in the frontal plane (column 6) to match the ground truth (column 4) and warped into a neutral face (column 7) to assist in FER. The network training is based on a synchronous capture system (SNaP-2DFe [9]), where facial motion is computed simultaneously in the absence (cam1) and presence (cam2) of head pose variation.</figcaption></figure>

to correct head pose variations and bring the face into a frontal view, they often introduce artifacts that can hamper the accuracy of facial expression recognition models [9], given that these methods are mainly used in the field of face recognition [11]–[13], where the appearance of deformations causes minor performance issues. However, they are not well suited to facial expression recognition, where preservation of facial geometry is very important to correctly characterize facial movements.

Recent research leverages optical flow for frontalization quality [14]. Building on biomechanical studies of facial motion [15], [16], we formalize this through two physiological constraints: 1) **Expression Motion Prior**: Facial muscle contractions produce characteristic motion patterns (e.g., zygomaticus major activation in smiles [17]) that are preserved across poses. 2) **Pose Motion Bounds**: Head rotations induce rigid motion fields bounded by neck kinematics [18], distinguishable from non-rigid expression motions.

We suggest considering head pose variation as structured noise in optical flow fields, separable from expression-related facial motions through their distinct kinematic properties: rigid head movements follow global affine patterns, while expression deformations exhibit local non-rigid dynamics constrained by facial biomechanics [22]. This decomposition leverages the demographic invariance of expression motions [18]—where expressions produce consistent muscle activation patterns across individuals (Fig. 1)—to enable robust frontalization.

O. Ikne, B. Allaert and H. Wannous are with IMT Nord Europe, Institut Mines-Télécom, Univ. Lille, Centre for Digital Systems, F-59000 Lille, France. e-mail: {omar.ikne,bejamin.allaert,hazem.wannous}@imt-nord-europe.fr

I.-M. Bilasco is with Centre de Recherche en Informatique Signal et Automatique de Lille, Univ. Lille, CNRS, Centrale Lille, UMR 9189 - CRISTAL -, F-59000 Lille, France. e-mail: marius.bilasco@univ-lille.fr
The extracted expression flows are then warped onto canonical frontal faces using a learned physically-plausible deformation model (see Fig. 1), preserving both anatomical constraints (e.g., tissue elasticity) and dynamic expression characteristics for improved FER accuracy.

Despite advances, two gaps persist: (1) Existing FVS methods prioritize identity preservation over expression fidelity [2], and (2) motion-based approaches lack explicit modeling of bio-mechanical noise [19], [20]. We bridge these gaps by reformulating pose variation as motion-domain noise and decomposing facial motion via a two-stage process: (a) adversarial filtering of pose-induced noise in optical flows, and (b) expression-preserving warping onto neutral faces (Fig. 1). This preserves cross-individual motion patterns while eliminating landmark dependencies.

Given the rise of generative models and their ability in tackling denoising and style transfer challenges, we explore a solution based on conditioned Generative Adversarial Networks (cGANs) [21] to address this issue. Our novel approach brings forth a range of distinctive features, such as:

**Unconstrained inter-individual variation** - This strategy takes advantage of the similarity of facial motion to reduce inter-individual variation for the same facial expression.

**Facial landmarks independence** - In contrast to recent approaches based on face texture and geometry, the proposed approach do not rely on landmarks or other facial attributes and is therefore not strongly impacted by the accurate location of facial landmarks.

**Expression transfer ability** - By generating facial motion flows instead of face images, this approach gains in flexibility. The ability to warp facial motion to any face, real or synthetic, extends its utility to a variety of face categories, including people, animals and drawings, and can serve as a useful data augmentation technique.

The structure of this paper is as follows: in Section II, we review relevant recent methods for FER and the FVS methods for dealing with the variation in head pose. Section III present our generative model to deal with the head pose variation while preserving facial expressions. In Section IV, the experimental setup including datasets, protocols and classification models are detailed. The experimental results are provided in Section V. We conclude and discuss future work in Section VI.

## II. RELATED WORK

### A. Frontal View Synthesis

Existing methods for FVS can be roughly categorized into two main groups: features-based and learning-based methods.

**Features-based methods** - belong to a class of techniques primarily founded on geometric transformation principles that leverage texture and/or geometry. These methods have proven effective in dealing with pose variations in the frontal plane, offering solutions such as rigid alignment achieved through landmark-based techniques [22], non-rigid alignment utilizing optical flow [23], and even a combination of rigid transformation and non-rigid deformation for more refined transformations [24], [25]. These techniques demonstrate considerable success when it comes to inducing transformations for in-plane variations. However, the challenge arises when handling

out-of-plane (3D) pose variations, where the existing features-based methods fall short. To address this limitation, recent advancements have been proposed, such as the incorporation of facial landmarks and thin-plate splines warping [26], or the utilization of the 3D Morphable Model (3DMM) approach [27] to estimate the pose of a non-frontal face relative to a 3D frontal face model, then use the pose coordinates to warp the pixels of the input face onto a frontal face. Despite these efforts, many of these methods require model adjustments and manual annotation of out-of-plane facial landmarks, making the accuracy of these landmarks a crucial factor for achieving effective face frontalization [28]. Moreover, the process of filling in invisible regions due to pose assumptions is based on the assumption of face symmetry, which, in reality, is not always valid due to variations in illumination and occlusions, raising further challenges to the accuracy of frontalization.

**Learning-based methods** - represent another major group of FVS techniques that rely on generative models, most notably Variational Auto-Encoders (VAEs) [29] and Generative Adversarial Networks (GANs) [30]. While VAEs offer relatively easier learning processes, GANs stand out as more suitable for addressing the highly challenging task of frontal view synthesis, thanks to their enhanced flexibility in incorporating constraints during the reconstruction process. The 3DMM field has also witnessed numerous extensions mainly motivated by the capabilities of GANs, leading to a multitude of widely proposed methods [30]–[33]. Recent breakthroughs in texture-based approaches have made it possible to correct pose variations while preserving facial expressions [34]–[37]. Nevertheless, it's essential to acknowledge that texture-based methods require extensive datasets comprising a large number of subjects and samples to yield satisfactory results in face-frontalization tasks, as they rely essentially on image texture and geometry, both of which exhibit significant variation between individuals. Conversely, some researchers have proposed geometric transformation methods that rely on generative models (GANs) and optical flows to address the challenges of frontal view synthesis [20], [38]. Chang et al. [38] use optical flow to train a network to de-express towards a neutral face by suppressing facial motion and re-express to reconstruct the original face. This work highlights the benefits of optical flow for correcting head pose and transferring expression to the face. However, the authors highlight that unsupervised learning tends to perform less well than existing supervised learning. Other approaches [19], [20] exploit optical flow to enhance the quality of facial reconstruction in the frontal plane. Although these methods improve on the performance of current systems, they rely on precise landmarks localization and 3D face shape estimation to correctly frontalize the face. However, the combination of these different features tends to drastically reduce the performance of these methods in the presence of large head pose variations or facial expressions.

It's worth mentioning that while the majority of learning-based approaches focus primarily on improving face recognition performance [19], [20], they may not explicitly prioritize the preservation of facial expressions during the face frontalization process. This aspect requires particular attention in order to guarantee a holistic and natural synthesis of the
face view that accurately represents the facial expressions. Incorporating an FER-related loss as an integral component of the overall reconstruction loss can be an effective way of ensuring the preservation of facial expression dynamics during the frontalization process.

Recent works on FVS being features-based or learning-based often distort expressions due to their reliance on facial symmetry assumptions and landmarks accuracy, limiting their robustness under extreme poses and their suitability for facial expressions related applications. Notably, none explicitly separate expression-induced motion from pose which is critical for FER applications. Our work bridges this by: (1) Motion disentanglement: Directly eliminating pose-related motion. (2) Landmark independence: Avoiding error-prone landmark detection. (3) Bio-mechanical consistency: Preserving natural motion patterns via expression-discriminative losses.

## B. Training strategies and datasets

Training learning-based models, particularly generative models, can be realized through three main strategies. The first, supervised learning, necessitates an extensive dataset containing paired face images depicting non-frontal view faces in conjunction with their corresponding frontal view counterparts, commonly denoted as "ground truths" [30], [31].

The second approach involves semi-supervised learning, in which the dataset includes both annotated data with ground truth and unannotated data [39]. The key aspect of these methods is the use of adversarial identity-preserving loss for efficient learning. The third method is unsupervised learning, which overcomes the need for paired data and relies primarily on cycle consistency [40]. Recent contrastive learning variants [41] attempt to preserve expression semantics through positive/negative pair selection, but may lose subtle pose-expression correlations when disentangling features. Zhou et al. [42] proposed to synthesize profile views from a collection of frontal views in order to generate input versus target pairs for training image-to-image translation GANs.

While semi-supervised and unsupervised learning aim to reduce the need for paired data, supervised learning remains a reliable method, particularly when paired data is available, and offers distinct advantages over semi-supervised and unsupervised learning approaches in image-to-image translation problems [38]. With access to paired data, supervised learning has well-defined performance metrics, making it easier to measure and optimize model performance. Which results in a greater predictive accuracy and a better generalization to unseen data when the training dataset is diverse and representative.

The training of supervised approaches to face frontalization has been based mainly on the MultiPIE [43] and BP4D [44] databases. However, the strategies employed to provide varied poses are not always well adapted, as illustrated in Figure 2. Multi-Pie relies on a multi-view system, which involves taking an image of a face from several angles. BP4D involves recording frontal faces using a 3D capture system. Variations in pose are generated automatically by rotating the camera around the 3D model. Although these datasets tend to provide a wide range of head pose variations, exploiting the frontal

face as ground truth, the data acquired do not reflect real pose variations, where the head is constrained by bio-mechanical rules. These constraints guarantee consistency between head movement and facial expressions, which are not present in most of these datasets. Among the many datasets, SNaP-2DFe [9] stands out for the synchronized capture system, offering simultaneous video acquisition of the expressive face with and without variations in head pose. These datasets offer sequences of basic facial expressions played out, with variations in head pose that closely resemble real-world scenarios. This crucial feature ensures that the dataset accurately captures the complexities of facial expressions and bio-mechanical constraints in natural environments, making it particularly well-suited to meet the challenges of head pose variation in FER tasks. Consequently, the constraint of no paired data is minimal and can be overcome efficiently.

## III. EMOTION-GAN

We propose an end-to-end deep learning approach for face FVS based in the motion domain. It consists of two main phases: a) motion frontalization and b) motion warping as illustrated in Figure 3.

Let $X \subset \mathbb{R}^{W \times H \times C}$ be a source domain, e.g., non-frontal faces with facial expressions, and $X^* \subset \mathbb{R}^{W \times H \times C}$ the corresponding target domain, e.g., frontal faces with same facial expression, where $H$, $W$, and $C$ denote the image height, width, and channel number respectively. We aim at learning a non-linear 2D-to-2D mapping $G: X \mapsto X^*$ that brings any arbitrary non-frontal face into a frontal view with the additional constraint of preserving facial expressions, i.e. $\forall (x, x^*) \in X \times X^*, e(G(x)) = e(x^*)$ where $e: X_f \mapsto \mathcal{E}$ is a mapping of frontal faces $X^*$ into the set of facial expressions $\mathcal{E}$. The mapping $G$ can be roughly written as $G = G_W \circ G_F$, where $G_F(\cdot)$ is a motion frontalization mapping and $G_W(\cdot)$ is a motion warping mapping.

<figure><img src="image_3.png" /><figcaption>Fig. 2. Setup used in datasets to generate head pose variations and provide ground truth of facial movement and expression. GT: Ground Truth; P: Pose.</figcaption></figure>
<figure><img src="image_4.png" /><figcaption>Fig. 3. **eMotion-GAN approach.** 1st phase: motion frontalization. Given an input optical flow (y), the generator ($G_F$) estimates and filters the motion induced by the variation of the head pose and transposes the motion induced by the facial expression in the frontal plane ($\tilde{y}^*$) to approximate the real motion ($y^*$). Besides the reconstruction losses, the expression discriminator $D_E$ is introduced to force $G_F$ to preserve the facial expression through the loss $L_e$. The discriminator $D_P$ ensures that the properties of the facial movement (e.g. intensity and direction) are preserved through the $L_{adv}$ loss. 2nd phase: motion warping. Given a neutral face ($x_t^*$) and the frontalized motion field ($\tilde{y}^*$), the generator $G_W$ generates the corresponding expressive face. A pre-trained classifier $e_{FER}$ predicts the corresponding expression.</figcaption></figure>

The motion frontalization model aims to learn the mapping $G_F: Y \rightarrow Y^*$ that filters pose noise from input flow $y$ to output frontalized flow $\tilde{y}^*$. While the motion warping model $G_F$ operates in flow space, $G_W$ bridges motion and image domains: The two stages are trained jointly but address distinct challenges: $G_F$ focuses on motion disentanglement (Section 3.2), while $G_W$ ensures expression fidelity in the image domain (Section 3.3). This separation allows end-to-end training while preventing feature leakage between tasks.

Considering the recent promising results for image denoising [45] and style transfer [46] problems using GAN-based models, we propose to take advantage of this architecture to solve the problem at hand.

## A. Motion Calculation

Given a pair of consecutive face frames $x_t, x_{t+dt} \in X$ with a facial expression, the optical flow which specifies the *magnitude* and *direction* of pixel motion between the two frames is calculated $y = OF(x_t, x_{t+dt}) \in Y$, where $Y \subset \mathbb{R}^{W \times H \times 2}$ is the set of the optical flows and motion patterns of the expressive faces.

In order to facilitate the generalization of the model to handle various motion intensities, the magnitude of motion is normalized by a common *clipping* technique. The clipping threshold can be chosen efficiently by plotting the distribution of the magnitude over the dataset used and then choosing a value that prevents large information loss, e.g., the third quartile $Q_3$. The calculation process is as follows: first the face is detected, cropped, then the optical flow is calculated, on high

resolution images to avoid large information loss, and resized to match the model input. A larger cropping less focused on the inside of the face allows a better observation of the head movements.

Building on the benchmark study of Allaert et al. [47], we adopt the Farneback optical flow algorithm [48] for estimating facial motion due to its strong performance in FER tasks. Allaert et al. compared both handcrafted methods (Farneback [48], TV-L1 [49], LDOF [50], EpicFlow [51], FlowFields [52]) and deep learning-based approaches (PWC-Net [53], RAFT [54], NCUP [55], GMA [56]) across several FER datasets—including those featuring low-light conditions, background clutter, and partial occlusions—and found Farneback to consistently achieve top performance, ranking first overall. Its robustness under occlusion has also been highlighted in prior work [57], where it successfully reconstructed facial motion for FER despite disruptions. Beyond accuracy, Farneback is computationally efficient, making it well-suited for real-time applications where deep learning methods like RAFT, though more accurate in general-purpose motion estimation, are prohibitively expensive. Nonetheless, Farneback has inherent limitations in extreme conditions: (1) low-light environments with textureless regions that hinder polynomial expansion, (2) partial occlusions that interrupt motion continuity, and (3) rapid head movements exceeding its displacement estimation bounds. In our framework, these challenges are partially addressed through multi-scale processing and focusing on the facial region, though we acknowledge that more sophisticated but computationally demanding flow
estimators could offer advantages in such edge cases. Given our objective of leveraging motion cues to improve head frontalization under pose variation, rather than benchmarking flow algorithms, Farneback provides a well-validated, efficient, and task-appropriate solution.

## B. Motion Frontalization

Given an optical flow $y \in Y$ of non-frontal expressive face, $G_F$ estimates and filters the motion induced by the variation of the head pose and transposes the motion induced by the facial expression in the frontal plane $\tilde{y}^* = G_F(y)$ to match the ground truth flow $y^* \in Y^*$.

In the following, we present our proposed cGAN architecture for the frontalization model, which is mainly based on a flow generator $G_F$ and two discriminators, a motion discriminator $D_P$ and an expression discriminator $D_E$.

a) Flow Generator ($G_F$): The underlying architecture compromise an encoder network and a decoder network with residual blocks as bottleneck. The input and the output of the network are set to $128 \times 128 \times 2$.

The encoder consists of 4 downsampling blocks, each with a 2D convolution, an instance normalization [58], and a ReLU. Reflection padding is added in the first layer to prevent checkerboard artifacts [59]. Instance normalization is applied to all features of a channel, allowing the training process to achieve lower loss levels than batch normalization [58] over a wide range of small batch sizes. Two residual blocks [60] are added to further process the encoded flow. The residual blocks involve a 2D convolution, an instance normalization and a ReLU. The decoder processes the encoded motion with 5 residual blocks and reconstructs the frontalized motion with 4 upsampling, each with a 2D convolution, a layer normalization with affine transformation parameters and a ReLU.

b) PatchGAN Discriminator ($D_P$): Inspired by the Pix2Pix GAN discriminator [61], we design our discriminator based on the PatchGAN approach. By operating at the patch level, the PatchGAN discriminator achieves higher computational efficiency, as it requires fewer parameters compared to those operating at the image level. In addition, image-level discriminators may suffer from overconfidence, i.e. if they rely on a small set of features to distinguish true from fake images, the generator may exploit these features to fool the discriminator. Although control techniques such as dropout or label smoothing can help alleviate this problem, the PatchGAN discriminator is better suited since it discriminates at the patch level, thus avoiding this problem.

Each block of the proposed architecture consists of a 2D convolution layer with a kernel size of 4 and a stride size of 2, an instance normalization and a Leaky ReLU which makes the gradients less sparse.

c) Expression Discriminator ($D_E$): We introduce a second discriminator to force the flow generator $G_F$ to comply with the additional constraint of preserving facial expressions while reconstructing the frontal flows. It serves as an optical flow CNN-based classifier (Flow-CNN) for FER. It takes an optical flow which encodes the facial motion of a subject, as input and predicts the corresponding facial expression. Its

architecture consists of 3 blocks for features extraction, each composed of a 2D convolution, a batch normalization, a max pooling and finally an MLP layer for classification.

## C. Motion Warping

The motion warping phase consists in warping the frontalized facial motion synthesized by $G_F$ onto a neutral frontal face to generate the corresponding expressive frontal face. In this context, given a neutral face image $x_t$ and a frontalized optical flow $\tilde{y}^* = G_F(y)$, the motion warping model aims at reconstructing the corresponding expressive face by applying the given motion field to the neutral face, i.e. $\tilde{x}_{t+dt}^* = G_W(\tilde{y}^*, x_t^*)$. Where $G_W$ denotes the image generator. The expressive frontal faces produced are examined using an image-based facial expression discriminator, called eFER, which guarantees the fidelity of facial expression preservation in the image domain.

a) Image Generator ($G_W$): The image generator includes a motion encoder, an identity encoder for encoding the identity of the subject's face, and a joint decoder for applying the encoded motion to the encoded face identity.

The architectures of encoders and decoder are roughly similar to those of the flow generator with a difference of number of sampling and residual blocks. Both encoders compromise 3 downsampling and 3 residual blocks. While the decoder consists of 3 residual and 3 upsampling blocks.

b) Expression Discriminator ($e_{FER}$): For the FER task on images synthesized by the image generator, we use a pre-trained model, namely DMUE [62], with the design of latent distribution mining and pairwise uncertainty estimation. DMUE demonstrates excellent performance in FER due to its specific design to meet the challenge of annotation ambiguity [62]. Moreover, it provides valuable feedback to the image generator, indicating whether the facial expression has been accurately warped into the face or not.

## D. Loss Functions

For a standard conditional GAN, the objective function which refers to the conditional adversarial loss function is formulated as [21]:

$$ \mathcal{L}_{cGAN}(G, D) = \mathbb{E}_{X,Y}[\log D(Y)] + \mathbb{E}_{X,Y}[\log(1 - D(X, G(X, Z)))] \quad (1) $$

where the model is trying to learn the mapping $G: \{X, Z\} \to Y$, $X$ is the source domain, $Y$ is the target domain and $Z$ is the noise sampled from a normal distribution. The generator tries to minimize this loss while the discriminator tries to maximize it. In our case, the loss of the discriminator compromises the losses of both motion and expression discriminators.

Since GANs generally suffer from poor quality generation and artifacts, we propose to add additional penalties to the loss function to impose constraints on the quality of the generated flows and images.

For the image generator, the loss function consists of three main parts: an MAE loss, a VGG-based perceptual loss [63]
and an image-based facial expression loss. The corresponding total loss is given by:

$$ \mathcal{L}_{G_W} = \alpha_1 \mathcal{L}_1 + \alpha_2 \mathcal{L}_{\text{perc}} + \alpha_3 \mathcal{L}_{\text{FER}} \quad (2) $$

$\mathcal{L}_{\text{FER}}$ is the categorical cross-entropy loss between the true and predicted facial expressions. While $\alpha_1$, $\alpha_2$ and $\alpha_3$ are loss scaling factors. For the flow generator, the generated flows can be improved by using different loss functions to preserve the properties of the facial motion (movement pattern, magnitude and intensity), and to recognize the facial expressions. Two main losses, the End Point Error (EPE) and the Charbonnier loss

a) **EndPoint Loss:** Is a common optical flow metric that prioritizes accuracy and indirectly favors smoothness by penalizing large errors. It's calculated by measuring the Euclidean distance between an estimated flow, denoted as $G_F(y)$, and the ground-truth flow $\tilde{y}^*$.

$$ \mathcal{L}_{\text{epe}} = \mathbb{E}_{y, \tilde{y}^*} [\|\tilde{y}^* - G_F(y)\|] \quad (3) $$

b) **Charbonnier Loss:** Is a differentiable variant of $\mathcal{L}_1$ that is useful for flow and depth estimation tasks that require robustness. Its particularity lies in the fact that it behaves as a quadratic loss near the origin and as an absolute loss far from the origin.

$$ \mathcal{L}_{\text{char}} = \mathbb{E}_{y, \tilde{y}^*} \left[ \| \sqrt{\mathcal{L}_1(\tilde{y}^*, G_F(y))^2 + \epsilon^2} \right] \quad (4) $$

$\epsilon$ is the region for which the loss function changes from being approximately quadratic to being approximately linear.

The total loss of the flow generator is composed of three main parts: the standard GAN loss (for both discriminators), the optical flow reconstruction loss which includes the two weighted losses (EPE, Charbonnier) and the motion warping loss. The resulting loss can be formulated as follows:

$$ \mathcal{L}_{G_F} = \lambda_1 \mathcal{L}_{GAN} + \lambda_2 \mathcal{L}_{\text{epe}} + \lambda_3 \mathcal{L}_{\text{char}} + \mathbf{1}_c \lambda_4 \mathcal{L}_{G_W} \quad (5) $$

where $\lambda_1$, $\lambda_2$, $\lambda_3$ and $\lambda_4$ are scaling factors, $\mathcal{L}_{GAN} = 1/2(\mathcal{L}_{GAN}(G_F, D_P) + \mathcal{L}_{GAN}(G_F, D_E))$ and $\mathbf{1}_c$ is the indicator function to indicate whether to connect the two losses, i.e. $\mathbf{1}_c = 0$ if $\text{epoch} < e_c$ and 1 otherwise. Indeed, the two generators are trained separately in parallel until a given epoch $e_c$ where the image generator comes into play to judge whether the flows frontalized by the flow generator are war payable into the texture. $e_c$ indicates a level at which the image generator manages to generate realistic expressive faces.

Regarding the discriminators, we use standard loss functions, namely sigmoid cross-entropy for the first discriminator and categorical cross-entropy for the emotion discriminator.

# IV. EXPERIMENTAL PROTOCOL

## A. Datasets

We conducted our experiments on five datasets covering a large variability of intra-individual facial expressions. Training our end-to-end model requires a dataset with paired images. We selected SNaP-2DFe [9] as it's the only publicly available dataset with paired images sequences with expressive faces acquired simultaneously with and without head pose variation.

The training phase is assisted with commonly used datasets for FER including CK+ [64], ADFES [65], MMI [66], Oulu-CASIA [67] datasets. These datasets are being included mostly to strengthen the facial motion patterns. Regarding those without frontal versus non-frontal views, we created non-frontal images by applying basic rotations and adding random noise to the original images. We used the original images as the expected frontalized ground-truth. The datasets selected ensure a consistent validation protocol. Other commonly used datasets, such as BP4D [44], are excluded from our training due to the manner in which head pose variations are artificially generated, i.e. rotating the camera around the face in 3D space, bypasses the bio-mechanical constraints of the face, which runs counter to model-based learning.

Simultaneous Natural and Posed 2D Facial expressions dataset (SNaP-2DFe) [9] is an image dataset that meets the requirements requested for the motion frontalization phase. It contains paired images of frontal and non-frontal faces for different subjects and facial expressions. More specifically, it provides images of subjects with expressive faces acquired simultaneously with head pose variation (unconstrained recording) and without head pose variation (constrained recording) from 15 different subjects with 6 categories of head poses (yaw, pitch, roll, diagonal, nothing and Tx) and 7 different facial expressions most commonly used in the literature (disgust, happiness, anger, surprise, neutral, fear and sadness).

## B. Evaluation Protocols

We conduct comprehensive comparisons between two scenarios: (1) baseline performance using original non-frontal images and unprocessed optical flows, versus (2) enhanced results using our frontalized outputs (both warped frontal images and processed motion fields). Crucially, our experiments demonstrate that the proposed model boosts FER accuracy under head pose variations across both image and motion domains, validating its dual-domain effectiveness. This improvement occurs because our frontalization process specifically targets and corrects pose-induced distortions while preserving expression-related features in both representations.

The evaluation of our model is performed following a 10-Fold cross-validation (9-fold for training, 1-fold for testing). The 9 training folds from different datasets are merged to reconstruct the training samples, and the evaluation is done on the remaining fold of each dataset separately. Performance is evaluated on the basis of the classification accuracy using different FER models.

The evaluation is performed in three main steps as illustrated in Figure 4. First, the frontalization and warping models are trained on the training folds. Then, the trained frontalization model is used to normalize the non-frontal flows in the test folds, and these frontalized flows are used with the corresponding frontal neutral faces to reconstruct the expressive faces with the trained warping model. Finally, the reconstructed flows and images are processed by FER models in both motion and image domains to obtain recognition accuracies.

In order to provide a large variety of optical flows and to cover a wide range of facial motion intensities, we propose to
<figure><img src="image_5.png" /><figcaption>Fig. 4. Cross-validation evaluation protocol. A: training the end-to-end model on training folds. B: frontalizing flows and warping images of the test fold using the trained models. C: evaluating performances using FER models on the reconstructed test fold in both motion and image domains.</figcaption></figure>

combine two different ways of calculating optical flows in an image sequence; (1) The *neutral-apex strategy*, it consists in calculating the optical flow between the first frame (neutral) and the last frame (apex) in each image sequence *s* of length *n*, i.e. the set {`(f_{s_0}, f_{s_n}); \forall s`}. The larger *n* is, the larger the head pose and thus the higher the intensity of the motion, and conversely. (2) The *frame-to-frame strategy*, which consists of using all the frames in each sequence *s* by calculating the optical flow between each pair of consecutive frames, i.e. the set {`(f_{s_i}, f_{s_{i+1}}); \forall i, s`}. The motion intensity depends on the time interval between the frames. Yet, for this strategy, the motion is less intense than for the neutral-apex strategy.

### C. Implementation Details

The facial motion is calculated using Farneback method [48] which extracts optical flow from grayscale images. Allaert et al. have shown it to be best suited to FER [47]. We use a clipping value of 10.0 for flow normalization for magnitude calculated according to the average distribution of the facial movement and motion intensities on SNAP-2DFe dataset, and we normalize both magnitudes and directions (in radian).

For the motion frontalization model, important learning parameters include the learning rates. We selected Adam with $1 \times 10^{-5}$ for the flow generator, Adam with $1 \times 10^{-4}$ for the patches discriminator and SGD with $1 \times 10^{-4}$ for the expression discriminator. The learning rates are gradually decreased during training. We follow DCGAN [68] and use a slope (alpha parameter) of 0.2 for the patches discriminator. The loss function scaling factors, are set as follows: $\lambda_1 = 1$, $\lambda_2 = 7$, $\lambda_3 = 3$ and $\lambda_4 = 1$.

For the motion warping model, we used Adam and set the learning rate to $10^{-5}$ for the image generator, the learning is reduced accordingly as well. The loss function scaling factors are set as follows: $\alpha_1 = 2$, $\alpha_2 = 7$ and $\alpha_3 = 1$.

Our model is trained for 15 epochs with a batch size of 4. The connection epoch between the two losses of the flow

generator and the image generator is set to $e_c = 5$. We use the PyTorch libraries [69] to implement our end-to-end model. All experiments are conducted using an NVIDIA GeForce RTX 2080 GPU.

## V. EVALUATIONS

We assess the effectiveness of our method through two sets of experiments. First, we compare our approach with recent FVS methods. Secondly, we illustrate its potential for cross-subject facial motion transfer. Finally, we conduct an ablation study to investigate the contribution of each loss term to the overall performance of the model.

### A. Frontal View Synthesis

We propose a qualitative and a quantitative evaluation in both motion and image domains. First, we demonstrate the efficiency of our end-to-end model to generate frontal faces while preserving facial expressions. Then we compare our approach against different FVS approaches and show its efficiency in reducing the FER gap between frontal and non-frontal faces when using different FER methods.

We compare our approach against the recent best FVS approaches. For hand-crafted approaches, we include SIFT flow [23] (non-rigid registration based on optical flow), while for deep learning approaches, we consider CFR-GAN [36] and pixel2style2pixel (pSp) [70] for image-based methods, and GLU-Net [71] for optical-flow-based methods.

In order to make a fair comparison between the different FVS approaches, all experiments were performed under identical settings, thus ensuring that any biases that might result from model selection or classifier optimization are omitted. The baseline involves only face detection, followed by cropping around the face, that is, no frontalization is applied, only the most basic pre-processing step. All approaches are applied to the baseline images. We compare the results in both motion and image domains.

1) *Qualitative Comparison:* We first conduct a qualitative experiment on evaluation datasets. We compare the FVS results of our proposed method for both motion and texture.

For our method, the flow frontalization is done directly in the motion domain, but for those for which the frontalization is done on the image, we first frontalize the image and then compute the motion on the frontalized faces. The comparison is made on the basis of three common metrics, structural similarity (SSIM) and RMSE for images and EPE for optical flows. We also provide Inference Time (IT) critical for real-time deployment. Table I shows the superiority of our method in both motion and image over the other FVS methods. Indeed, a lower EPE indicates that our frontalization model succeeds in frontalizing motion while preserving facial expression patterns, which is mainly due to the fact that our method frontalizes motion rather than image, allowing it to take advantage of the similarity of expression patterns between different subjects, and a higher SSIM indicates that the warping model successfully warps the frontalized motion in the image, which is mainly driven by the perceptual loss included in the warping model that allows it to generate
<figure><img src="image_6.png" /><figcaption>Fig. 5. Qualitative comparison of different FVS methods in image domain. Our approach effectively handles large variations for frontal view synthesis while preserving facial expressions.</figcaption></figure>

<table><caption>TABLE I
MEAN AND STANDARD DEVIATION OF EPE, SSIM, RMSE AND IT ACROSS VALIDATION DATASETS. ↓ INDICATES THAT THE LOWER THE RESULT, THE BETTER IT IS AND VICE VERSA, THE BEST RESULTS ARE SHOWN IN BOLD.</caption><thead><tr><th>Method</th><th>EPE ↓</th><th>SSIM ↑</th><th>RMSE ↓</th><th>IT(ms) ↓</th></tr></thead><tbody><tr><td>SIFT [23]</td><td>36.07±13.50</td><td>0.58±0.13</td><td>9.10±0.70</td><td>-</td></tr><tr><td>CFR [36]</td><td>44.96±6.24</td><td>0.31±0.07</td><td>9.99±0.11</td><td>13.11</td></tr><tr><td>GLUNet [71]</td><td>33.93±6.58</td><td>0.66±0.13</td><td><b>7.62±1.04</b></td><td><b>8.17</b></td></tr><tr><td>pSp [70]</td><td>40.32±6.01</td><td>0.36±0.07</td><td>9.67±0.10</td><td>15.73</td></tr><tr><td>Ours</td><td><b>14.34±3.99</b></td><td><b>0.78±0.18</b></td><td>7.75±1.00</td><td>11.55</td></tr></tbody></table>

consistent expressive faces while preserving the facial identity of the subject.

Table I also compares end-to-end inference times (IT) for 128×128 RGB inputs under identical hardware. Our pipeline achieves 11.55ms total (5.62ms for motion frontalization $G_F$ + 5.93ms for warping $G_W$), outperforming CFR and pSp. This efficiency, coupled with superior EPE and SSIM, validates our method's real-time capability without compromising pose robustness.

For qualitative warping analysis, Figure 5 shows the results given by the different FVS methods on expressive faces with head pose variations. In terms of image reconstruction, our method manages to warp the frontalized flow into the neutral face to recover the original facial expression. Unlike other methods, our method does not suffer from artifacts and distortion of expression patterns. Thus, allowing the facial expression to be recovered without loss of identity of the subject.

2) FER Comparison: We perform FER comparison between the FVS methods cited in Section V in both motion and image. We denote by $\Delta(\%)$ the percent improvement in recognition accuracy between non-frontalized (original non-frontal faces) and frontalized faces (synthesized faces).

For motion-based FER, we compare the recognition accuracy given by the motion-based CNN (Flow-CNN) detailed in Section III on different evaluation datasets. To our knowledge, no pre-trained model has been proposed for FER in optical flow, therefore we use our proposed motion-based CNN (expression discriminator) to classify the frontalized flows.

For FER on warped images, we cite the latest research advances in the field of FER to evaluate the expressive faces given by different FVS approaches. We include DMUE [62], a FER method that relies on latent distribution mining to estimate the pairwise uncertainty between well-labeled and ambiguous samples. HSEmotion [74] extracts facial features with the single EfficientNet [75] model pre-trained on Affect-Net [76]. DAN [73] is an attention-based approach for FER. We also cite Ad-Corre [72], an approach that employs adaptive correlation loss to maximize correlation for intra-class samples and minimize it for inter-class samples. We use the pre-trained models provided by the authors and we emphasize that all these methods are designed to deal with FER in the wild. DMUE, DAN and Ad-Corre are trained on AffectNet dataset [76] and HSEmotion is trained on AFEW dataset [77].

Table II presents the results obtained by different FVS methods when using different FER methods in both motion and image. In image domain, the baseline seems to achieve the best results given that recent FER approaches are trained to analyze frontal and less constrained faces, thanks to augmentation methods. Image-based frontalization tends to introduce excessive deformation, altering both facial structure and expression. Our motion-based approach applies a loss that preserves face structure and ensures consistent textural rendering, albeit with a slight attenuation of expression (see Figure 5). Nevertheless, our method is still competitive on most datasets as it nearly achieves the results given by recent FER methods in the literature. However, in motion field, the results clearly demonstrate that our proposed method outperforms other FVS methods across datasets. The percent improvement $\Delta$ in optical flow attains +5% on small pose variations and up to +20% on large pose variations. While our method is invariant in frontal settings, e.g. CK+ and ADFES.

These results highlight the strength of motion-based approaches over image-based approaches, where inter-individual variations tend to impact the quality of frontalization and thus reduce FER performance. Furthermore, unlike image-based methods that hardly manage to handle both frontal and non-frontal settings, motion-based methods achieve high performance in non-frontal settings while being invariant in frontal settings.

3) Motion Separability Validation: To empirically validate motion separability, we analyze the latent space of our flow encoder $G_F$: (1) Frontal faces provide expression-only embeddings $E_{exp}$, while (2) non-frontal faces yield combined embeddings $E_{comb}$. The pose component $E_{pose} = E_{comb} - E_{exp}$ reveals (see Fig 6):

* 95.7% cluster purity (expression vs. pose) in t-SNE space (perplexity=30).

* Minimal overlap (6.7% of samples) at decision boundaries.

This confirms our key assumption that expression and pose motions occupy distinct subspaces, with pose variations acting as approximately additive noise. t-SNE plot (Fig. 6) confirms orthogonal subspaces, with pose variations introducing directional offsets to expression clusters.
<table><caption>TABLE II
QUANTITATIVE COMPARISON OF DIFFERENT FVS METHODS USING DIFFERENT FER METHODS IN BOTH IMAGE (TOP PART) AND OPTICAL FLOW (BOTTOM PART) ON VALIDATION DATASETS. THE BEST IMAGE SCORES ARE UNDERLINED AND THE BEST OPTICAL FLOW SCORES ARE IN BOLD FOR EACH DATASET. FOR EACH APPROACH, THE 5 ROWS CORRESPOND TO THE SCORES OBTAINED WITH THE USE OF THE DIFFERENT FVS METHODS IN THE FOLLOWING ORDER: BASELINE (NO FRONTALIZATION), SIFT FLOW [23], CFR-GAN [36], GLUNET [71], PSP [70] AND OUR EMOTION-GAN.</caption><thead><tr><th rowspan="2"></th><th rowspan="2">FER<br>Method</th><th colspan="2">CK+</th><th colspan="2">ADFES</th><th colspan="2">MMI</th><th colspan="2">CASIA</th><th colspan="2">SNAP-F</th><th colspan="2">SNAP-NF</th></tr><tr><th>acc.[%]</th><th>std[%]</th><th>acc.[%]</th><th>std[%]</th><th>acc.[%]</th><th>std[%]</th><th>acc.[%]</th><th>std[%]</th><th>acc.[%]</th><th>std[%]</th><th>acc.[%]</th><th>std[%]</th></tr></thead><tbody><tr><td rowspan="12">Image-based</td><td rowspan="6">Ad-Corre [72]</td><td>80.38</td><td>±5.38</td><td>90.93</td><td>±8.02</td><td>70.07</td><td>±11.79</td><td>62.77</td><td>±7.42</td><td>59.19</td><td>±2.24</td><td>58.13</td><td>±5.08</td></tr><tr><td>78.66</td><td>±7.00</td><td>89.29</td><td>±9.87</td><td>68.05</td><td>±10.47</td><td>61.31</td><td>±7.30</td><td>55.77</td><td>±4.48</td><td>47.03</td><td>±5.24</td></tr><tr><td>68.34</td><td>±4.86</td><td>76.32</td><td>±7.29</td><td>61.71</td><td>±11.84</td><td>54.80</td><td>±3.52</td><td>37.83</td><td>±3.35</td><td>45.09</td><td>±3.98</td></tr><tr><td>79.80</td><td>±6.53</td><td>86.32</td><td>±8.81</td><td>69.05</td><td>±12.24</td><td>60.88</td><td>±6.25</td><td>53.63</td><td>±4.52</td><td>47.65</td><td>±5.34</td></tr><tr><td>65.83</td><td>±4.4</td><td>69.4</td><td>±7.08</td><td>46.36</td><td>±8.17</td><td>34.11</td><td>±4.34</td><td>47.64</td><td>±7.35</td><td>41.66</td><td>±3.85</td></tr><tr><td>77.02</td><td>±8.53</td><td>83.96</td><td>±10.59</td><td>64.71</td><td>±9.20</td><td>60.27</td><td>±4.79</td><td>60.25</td><td>±4.92</td><td>46.35</td><td>±6.16</td></tr><tr><td rowspan="6">DAN [73]</td><td>81.50</td><td>±5.08</td><td>86.32</td><td>±5.51</td><td>70.12</td><td>±11.48</td><td>63.20</td><td>±8.88</td><td>71.17</td><td>±4.41</td><td>67.30</td><td>±6.75</td></tr><tr><td>74.48</td><td>±7.75</td><td>87.03</td><td>±4.88</td><td>70.55</td><td>±8.83</td><td>62.17</td><td>±9.51</td><td>69.46</td><td>±4.87</td><td>56.42</td><td>±4.86</td></tr><tr><td>71.43</td><td>±5.88</td><td>83.13</td><td>±6.78</td><td>62.26</td><td>±10.79</td><td>60.27</td><td>±6.31</td><td>40.62</td><td>±5.75</td><td>49.59</td><td>±5.17</td></tr><tr><td>77.55</td><td>±6.53</td><td>83.24</td><td>±4.42</td><td>66.19</td><td>±7.92</td><td>61.53</td><td>±9.12</td><td>68.82</td><td>±4.80</td><td>60.06</td><td>±7.84</td></tr><tr><td>69.44</td><td>±5.1</td><td>71.81</td><td>±6.57</td><td>52.14</td><td>±7.43</td><td>44.97</td><td>±4.48</td><td>54.06</td><td>±3.81</td><td>51.9</td><td>±4.32</td></tr><tr><td>72.55</td><td>±4.82</td><td>84.01</td><td>±7.90</td><td>65.21</td><td>±5.14</td><td>61.33</td><td>±6.41</td><td>60.04</td><td>±5.80</td><td>45.07</td><td>±5.47</td></tr><tr><td rowspan="6">HSE [74]</td><td>79.25</td><td>±4.94</td><td>93.90</td><td>±5.74</td><td>71.98</td><td>±8.36</td><td>63.00</td><td>±7.58</td><td>67.53</td><td>±3.22</td><td>63.90</td><td>±6.84</td></tr><tr><td>74.77</td><td>±4.84</td><td>90.88</td><td>±8.90</td><td>69.57</td><td>±6.01</td><td>59.85</td><td>±7.48</td><td>62.84</td><td>±3.39</td><td>51.72</td><td>±4.45</td></tr><tr><td>69.76</td><td>±3.77</td><td>82.47</td><td>±6.81</td><td>59.31</td><td>±9.93</td><td>56.92</td><td>±3.58</td><td>39.53</td><td>±3.93</td><td>43.16</td><td>±4.64</td></tr><tr><td>76.74</td><td>±4.72</td><td>90.16</td><td>±8.91</td><td>65.64</td><td>±6.49</td><td>60.27</td><td>±8.41</td><td>65.17</td><td>±3.34</td><td>58.13</td><td>±5.87</td></tr><tr><td>68.63</td><td>±4.72</td><td>83.24</td><td>±9.51</td><td>59.4</td><td>±6.68</td><td>49.38</td><td>±7.58</td><td>52.96</td><td>±5.38</td><td>41.23</td><td>±5.8</td></tr><tr><td>75.63</td><td>±5.91</td><td>87.14</td><td>±11.11</td><td>69.02</td><td>±7.18</td><td>61.52</td><td>±6.25</td><td>59.20</td><td>±6.29</td><td>42.50</td><td>±4.35</td></tr><tr><td rowspan="6">DMUE [62]</td><td>85.71</td><td>±4.10</td><td>92.31</td><td>±4.87</td><td>74.36</td><td>±6.96</td><td>70.70</td><td>±6.42</td><td>69.66</td><td>±1.77</td><td>66.90</td><td>±5.23</td></tr><tr><td>81.50</td><td>±4.84</td><td>88.52</td><td>±5.20</td><td>73.40</td><td>±7.86</td><td>64.66</td><td>±7.96</td><td>68.16</td><td>±4.12</td><td>57.07</td><td>±4.05</td></tr><tr><td>77.57</td><td>±4.78</td><td>82.42</td><td>±4.99</td><td>67.07</td><td>±7.73</td><td>66.31</td><td>±5.38</td><td>42.10</td><td>±4.00</td><td>50.86</td><td>±5.36</td></tr><tr><td>84.59</td><td>±4.58</td><td>91.59</td><td>±4.16</td><td>72.40</td><td>±7.94</td><td>67.36</td><td>±7.20</td><td>68.59</td><td>±2.06</td><td>61.58</td><td>±7.94</td></tr><tr><td>76.71</td><td>±6.61</td><td>86.26</td><td>±8.27</td><td>62.79</td><td>±5.87</td><td>56.06</td><td>±5.65</td><td>60.46</td><td>±5.15</td><td>43.59</td><td>±7.0</td></tr><tr><td>83.45</td><td>±3.06</td><td>88.63</td><td>±8.39</td><td>68.07</td><td>±8.10</td><td>70.51</td><td>±4.16</td><td>67.53</td><td>±3.32</td><td>50.85</td><td>±4.50</td></tr><tr><td rowspan="6">Flow-based</td><td rowspan="6">Flow-CNN</td><td>86.27</td><td>±6.08</td><td>90.83</td><td>±7.96</td><td>69.48</td><td>±10.83</td><td>69.89</td><td>±5.95</td><td><b>86.55</b></td><td>±3.59</td><td>42.31</td><td>±5.76</td></tr><tr><td>75.88</td><td>±4.77</td><td>76.21</td><td>±11.81</td><td>57.40</td><td>±10.87</td><td>59.80</td><td>±6.74</td><td>67.28</td><td>±7.81</td><td>33.96</td><td>±5.44</td></tr><tr><td>69.73</td><td>±6.58</td><td>76.26</td><td>±8.59</td><td>53.02</td><td>±12.74</td><td>56.03</td><td>±6.38</td><td>43.12</td><td>±7.01</td><td>35.66</td><td>±5.68</td></tr><tr><td>80.63</td><td>±4.47</td><td>86.32</td><td>±7.75</td><td>67.02</td><td>±11.89</td><td>66.52</td><td>±7.18</td><td>81.19</td><td>±4.78</td><td>54.91</td><td>±3.35</td></tr><tr><td>70.15</td><td>±6.13</td><td>74.17</td><td>±8.41</td><td>52.06</td><td>±12.96</td><td>56.03</td><td>±6.21</td><td>60.01</td><td>±6.39</td><td>45.84</td><td>±5.43</td></tr><tr><td><b>90.74</b></td><td>±3.01</td><td><b>94.62</b></td><td>±5.19</td><td><b>75.79</b></td><td>±9.81</td><td><b>73.22</b></td><td>±5.64</td><td>80.55</td><td>±6.21</td><td><b>64.10</b></td><td>±5.74</td></tr></tbody></table>

<figure><img src="image_7.png" /><figcaption>Fig. 6. t-SNE visualization of expression (yellow circles) and pose (purple circles) motion embeddings.</figcaption></figure>

## B. Cross-subject Facial Motion Transfer

In this section, we carry out an experiment to demonstrate the effectiveness of our motion warping model for transferring facial motion between two different subjects. In other words, our model can transfer facial expressions from subject A to subject B by warping a motion field, corresponding to the

facial expression, from subject A to subject B's face. We emphasize that the preliminary face registration step guarantees consistency in facial morphology. Figure 7 shows the results of cross-subject motion transfer between different individuals.

Our motion warping model can be used for different interesting applications, including: 1) data augmentation by generating a variety of expressive faces given a neutral face. Indeed, along with our warping model, we can build a conditioned GAN to generate facial expressions motion conditioning on the facial expression and warp the generated facial motion to neutral faces to generate additional data. 2) class balancing: to balance number of samples per expression in a non-balanced FER dataset. 3) intensification or attenuation of facial expressions: to transform micro-expressions into macro-expressions and vice versa. 4) category balancing: as in many datasets, categories such as gender, age and ethnicity are not balanced, our warping model can be used to overcome such constraint.

We emphasize that our warping model can be used to warp a facial motion field into any frontal face, whether real or not. For instance, in real-world applications, e.g. surveillance cameras, it is challenging to obtain the neutral frontal face of a subject. However, the frontalized motion can be warped into any neutral frontal face to get the corresponding expression.
<figure><img src="image_8.png" /><figcaption>Fig. 7. Cross-subject motion transfer: Our motion warping model transfers motion patterns between faces while preserving identity.</figcaption></figure>

For a quantitative evaluation, we propose to warp facial expressions motion from different datasets into a neutral average face (raw 4, column 4 in Figure 7) and then calculate the FER accuracies on the warped faces. The FER results are provided in Table III.

These results highlight the ability of our motion warping model to cross-subject facial expression transfer. As a matter of fact, it manages to obtain scores close to those obtained on the original faces (e.g. ~94% on ADFES).

### C. Ablation Study

In this section, we conduct an ablation study on eMotion-GAN, analyzing variants with one main loss discarded at a time (normalization model loss, emotion discriminator loss, and warping model loss). This allows us to understand the distinct contributions of each loss during training. We use the same end-to-end model, parameters, and training protocol, presenting accuracy results for various scenarios (motion and image) on different datasets in Table IV.

a) Without motion reconstruction loss ($\mathcal{L}_{epe}$ and $\mathcal{L}_{char}$): The absence of the two losses, $\mathcal{L}_{epe}$ and $\mathcal{L}_{char}$, during training

<table><caption>TABLE III
FER ACCURACIES ON WARPED IMAGES ACROSS EVALUATION DATASETS USING AVERAGE FACE.</caption><thead><tr><th>Method</th><th>CK+</th><th>ADFES</th><th>MMI</th><th>CASIA</th><th>SNAP-F</th></tr></thead><tbody><tr><td>Ad-Corre</td><td>75.33%</td><td>85.49%</td><td>59.90%</td><td>52.92%</td><td>55.33%</td></tr><tr><td>DAN</td><td>71.17%</td><td><strong>93.90%</strong></td><td>60.31%</td><td>56.68%</td><td>59.85%</td></tr><tr><td>HSE</td><td>68.33%</td><td>89.40%</td><td>57.50%</td><td>54.61%</td><td>54.09%</td></tr><tr><td>DMUE</td><td><strong>77.31%</strong></td><td>88.52%</td><td><strong>63.74%</strong></td><td>60.03%</td><td><strong>61.12%</strong></td></tr></tbody></table>

<table><caption>TABLE IV
ABLATION STUDY ON DIFFERENT LOSSES WITH FER ACCURACIES IN BOTH IMAGE (TOP LINE) AND OPTICAL FLOW (BOTTOM LINE). BEST IMAGE SCORES ARE UNDERLINED, BEST FLOW SCORES ARE IN BOLD.</caption><thead><tr><th>Method</th><th>ADFES</th><th>MMI</th><th>CASIA</th><th>SNAP-NF</th></tr></thead><tbody><tr><td>Ours w/o L<sub>epe</sub>, L<sub>char</sub></td><td>30.14%<br>44.12%</td><td>38.10%<br>40.05%</td><td>38.75%<br>42.08%</td><td>28.15%<br>30.29%</td></tr><tr><td>Ours w/o L<sub>D<sub>E</sub></sub></td><td>71.43%<br>92.86%</td><td>47.62%<br>66.67%</td><td>45.58%<br>68.75%</td><td>38.30%<br>51.06%</td></tr><tr><td>Ours w/o L<sub>G<sub>W</sub></sub></td><td>73.14%<br>92.86%</td><td>45.95%<br>70.95%</td><td>55.83%<br>70.83%</td><td>40.43%<br>61.70%</td></tr><tr><td>Ours</td><td>88.63%<br><strong>94.62%</strong></td><td>68.07%<br><strong>75.79%</strong></td><td>70.51%<br><strong>73.22%</strong></td><td>50.85%<br><strong>64.10%</strong></td></tr></tbody></table>

leads to a deficiency in the key details of facial expression in both motion and texture. Consequently, classifiers struggle to effectively distinguish between different expressions, resulting in less satisfactory outcomes in both normalized flows and synthesized faces. It is worth pointing out that reconstruction loss is of significant importance as a central component of our model.

b) Without emotion discriminator loss ($\mathcal{L}_{D_E}$): In cases where the loss of the emotion discriminator is not included in the training, the model lacks crucial feedback on the main signal that needs to be preserved, i.e. the motion of the expression. Consequently, this absence has a significant impact on the warping process, leading to sub-optimal results and compromising the overall performance of the model.

c) Without warping model loss ($\mathcal{L}_{G_W}$): When warping model loss is retained as part of the total loss for the motion normalization model, motion flows of the generated frontalized faces are produced; however, they do not exhibit optimal warpability when applied to neutral faces. In other words, the warping process does not effectively align the motion flows to the neutral face, leading to poor results in the overall frontalization process.

## VI. CONCLUSIONS

This paper presented eMotion-GAN, a motion-domain solution to one of FER most persistent challenges: maintaining accuracy under head pose variations. Where existing appearance-based frontalization methods distort expression patterns through artifacts, our two-stage framework achieves breakthrough performance by fundamentally rethinking pose variation as motion-domain noise separable from expression-related muscle movements.

The key contribution of our work lies in establishing motion-domain frontalization as a viable paradigm shift from traditional appearance-based methods, supported by comprehensive theoretical analysis of motion disentanglement for expression preservation. Through extensive experiments, we demonstrate that our approach outperforms SOTA frontalization methods in preserving expression fidelity while achieving consistent accuracy improvements ranging from 5% for small pose variations to 20% for extreme poses.

The implications extend beyond our immediate results. By successfully decomposing facial dynamics into expression and
pose components, we establish motion-domain processing as a viable alternative to appearance-based approaches for all pose-sensitive facial analysis tasks. Future work will focus on three key directions: (1) Developing lightweight versions of eMotion-GAN targeting real-time operation on embedded assistive devices, (2) The current static separation of pose and expression motions could be extended to temporal domain through recurrent architectures, potentially capturing micro-expression dynamics during natural head movement trajectories. (3) The inherent trade-off between flow methods—where optimal motion filtering estimators may differ from FER-optimal ones—warrants further study, particularly for edge-case pose correction.

**Ethical Considerations** Finally, while eMotion-GAN significantly improves facial expression preservation for assistive applications, it's important to recognize its potential misuse to create more convincing deepfakes by precisely transferring expressive motions between faces.

**Acknowledgement** This work is co-funded by the AI@IMT program of the Agence Nationale de la Recherche (ANR) and the region Hauts-de-France in France.

## REFERENCES

[1] Li, "Deep facial expression recognition: A survey," *IEEE transactions on affective computing*, vol. 13, no. 3, pp. 1195-1215, 2020.

[2] M. Sajjad, F. U. M. Ullah, M. Ullah, G. Christodoulou, F. A. Cheikh, M. Hijji, K. Muhammad, and J. J. Rodrigues, "A comprehensive survey on deep facial expression recognition: challenges, applications, and future guidelines," *Alexandria Engineering Journal*, vol. 68, pp. 817-840, 2023.

[3] A. Patwal, M. Diwakar, A. Joshi, and P. Singh, "Facial expression recognition using densenet," in *2022 OITS International Conference on Information Technology (OCIT)*, 2022, pp. 548-552.

[4] T. Sharma, M. Diwakar, P. Singh, S. Lamba, P. Kumar, and K. Joshi, "Emotion analysis for predicting the emotion labels using machine learning approaches," in *2021 IEEE 8th Uttar Pradesh Section International Conference on Electrical, Electronics and Computer Engineering (UPCON)*, 2021, pp. 1-6.

[5] T. Sharma, M. Diwakar, P. Singh, C. Arya, S. Lamba, and P. Kumar, "A review on egg based emotion analysis using machine learning approaches," in *2021 IEEE 8th Uttar Pradesh Section International Conference on Electrical, Electronics and Computer Engineering (UPCON)*, 2021, pp. 1-6.

[6] S. Li and W. Deng, "Deep facial expression recognition: A survey," *IEEE Transactions on Affective Computing*, pp. 1-1, 2020.

[7] H.-Y. An and R.-S. Jia, "Self-supervised facial expression recognition with fine-grained feature selection," *The Visual Computer*, vol. 40, no. 10, pp. 7001-7013, 2024.

[8] Y. Xie, J. Ou, B. Wen, Z. Yu, and W. Tian, "A joint learning method for low-light facial expression recognition," *Complex & Intelligent Systems*, vol. 11, no. 2, pp. 1-17, 2025.

[9] B. Allaert, J. Mennesson, I. M. Bilasco, and C. Djeraba, "Impact of the face registration techniques on facial expressions recognition," *Signal Processing: Image Communication*, vol. 61, pp. 44-53, 2018.

[10] S. Wang, Y. Chang, Q. Li, C. Wang, G. Li, and M. Mao, "Pose-robust personalized facial expression recognition through unsupervised multi-source domain adaptation," *Pattern Recognition*, vol. 150, p. 110311, 2024.

[11] M. H. Alhlffee and Y.-S. Huang, "A novel face frontalization method by seamlessly integrating landmark detection and decision forest into generative adversarial network (gan)," *Mathematics*, vol. 13, no. 3, p. 499, 2025.

[12] K. Zeng, Z. Wang, T. Lu, J. Chen, C. Liang, and Z. Han, "Multi-stage statistical texture-guided gan for tilted face frontalization," *IEEE Transactions on Image Processing*, 2025.

[13] W. Wang, X. Wang, Y. Shi, and X. Lu, "Hifidiff: High-fidelity diffusion model for face hallucination from tiny non-frontal faces," *Neurocomputing*, vol. 616, p. 128882, 2025.

[14] X. Dong, Y. Yang, S.-E. Wei, X. Weng, Y. Sheikh, and S.-I. Yu, "Supervision by registration and triangulation for landmark detection," *IEEE transactions on pattern analysis and machine intelligence*, vol. 43, no. 10, pp. 3681-3694, 2020.

[15] T. Wu, A. Hung, and K. Mithraratne, "Generating facial expressions using an anatomically accurate biomechanical model," *IEEE transactions on visualization and computer graphics*, vol. 20, no. 11, pp. 1519-1529, 2014.

[16] X. Zhang, L. Yin, J. F. Cohn, S. Canavan, M. Reale, A. Horowitz, and P. Liu, "A high-resolution spontaneous 3d dynamic facial expression database," in *2013 10th IEEE international conference and workshops on automatic face and gesture recognition (FG)*. IEEE, 2013, pp. 1-6.

[17] L. O. Kroczek and A. Mühlberger, "Time to smile: How onset asynchronies between reciprocal facial expressions influence the experience of responsiveness of a virtual agent," *Journal of Nonverbal Behavior*, vol. 47, no. 3, pp. 345-360, 2023.

[18] Y. Poleg, C. Arora, and S. Peleg, "Head motion signatures from egocentric videos," in *Asian Conference on Computer Vision*. Springer, 2014, pp. 315-329.

[19] M. R. Koujan, A. Roussos, and S. Zafeiriou, "Deepfaceflow: in-the-wild dense 3d facial motion estimation," in *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, 2020, pp. 6618-6627.

[20] Z. Peng, B. Jiang, H. Xu, W. Feng, and J. Zhang, "Facial optical flow estimation via neural non-rigid registration," *Computational Visual Media*, vol. 9, no. 1, pp. 109-122, 2023.

[21] M. Mirza and S. Osindero, "Conditional generative adversarial nets," *arXiv preprint arXiv:1411.1784*, 2014.

[22] P. Zarbakhsh and H. Demirel, "4d facial expression recognition using multimodal time series analysis of geometric landmark-based deformations," *The Visual Computer*, vol. 36, no. 5, pp. 951-965, 2020.

[23] C. Liu, J. Yuen, A. Torralba, J. Sivic, and W. T. Freeman, "Sift flow: Dense correspondence across different scenes," in *European conference on computer vision*. Springer, 2008, pp. 28-42.

[24] Z. Kang, M. Sadeghi, R. Horaud, and X. Alameda-Pineda, "Expression-preserving face frontalization improves visually assisted speech processing," *International Journal of Computer Vision*, vol. 131, no. 5, pp. 1122-1140, 2023.

[25] Z. Kang, R. Horaud, and M. Sadeghi, "Robust face frontalization for visual speech recognition," in *Proceedings of the IEEE/CVF International Conference on Computer Vision*, 2021, pp. 2485-2495.

[26] W. N. I. Al-Obaydy and S. A. Suandi, "Automatic pose normalization for open-set single-sample face recognition in video surveillance," *Multimedia Tools and Applications*, vol. 79, no. 3, pp. 2897-2915, 2020.

[27] X. Zhu, Z. Lei, J. Yan, D. Yi, and S. Z. Li, "High-fidelity pose and expression normalization for face recognition in the wild," in *Proceedings of the IEEE conference on computer vision and pattern recognition*, 2015, pp. 787-796.

[28] R. Belmonte, B. Allaert, P. Tirilly, I. M. Bilasco, C. Djeraba, and N. Sebe, "Impact of facial landmark localization on facial expression recognition," *IEEE Transactions on Affective Computing*, 2021.

[29] D. P. Kingma and M. Welling, "Auto-encoding variational bayes," *arXiv preprint arXiv:1312.6114*, 2013.

[30] X. Yin, X. Yu, K. Sohn, X. Liu, and M. Chandraker, "Towards large-pose face frontalization in the wild," in *Proceedings of the IEEE international conference on computer vision*, 2017, pp. 3990-3999.

[31] R. Huang, S. Zhang, T. Li, and R. He, "Beyond face rotation: Global and local perception gan for photorealistic and identity preserving frontal view synthesis," in *Proceedings of the IEEE international conference on computer vision*, 2017, pp. 2439-2448.

[32] R. Wu, G. Zhang, S. Lu, and T. Chen, "Cascade ef-gan: Progressive facial expression editing with local focuses," in *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, 2020, pp. 5021-5030.

[33] Y. Lee, T. Choi, H. Go, H. Lee, S. Cho, and J. Kim, "Exp-gan: 3d-aware facial image generation with expression control," in *Proceedings of the Asian Conference on Computer Vision*, 2022, pp. 3812-3827.

[34] Z. Zhang, X. Chen, B. Wang, G. Hu, W. Zuo, and E. R. Hancock, "Face frontalization using an appearance-flow-based convolutional neural network," *IEEE Transactions on Image Processing*, vol. 28, no. 5, pp. 2187-2199, 2018.

[35] S. Jie and Q. Yongsheng, "Multi-view facial expression recognition with multi-view facial expression light weight network," *Pattern Recognition and Image Analysis*, vol. 30, no. 4, pp. 805-814, 2020.

[36] Y.-J. Ju, G.-H. Lee, J.-H. Hong, and S.-W. Lee, "Complete face recovery gan: Unsupervised joint face rotation and de-occlusion from a single-
view image,” in *Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision*, 2022, pp. 3711–3721.

[37] Y. Liu, J. Peng, W. Dai, J. Zeng, and S. Shan, “Joint spatial and scale attention network for multi-view facial expression recognition,” *Pattern Recognition*, vol. 139, p. 109496, 2023.

[38] J.-R. Chang, Y.-S. Chen, and W.-C. Chiu, “Learning facial representations from the cycle-consistency of face,” in *Proceedings of the IEEE/CVF International Conference on Computer Vision*, 2021, pp. 9680–9689.

[39] Z. Zhang, R. Liang, X. Chen, X. Xu, G. Hu, W. Zuo, and E. R. Hancock, “Semi-supervised face frontalization in the wild,” *IEEE Transactions on Information Forensics and Security*, vol. 16, pp. 909–922, 2020.

[40] J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, “Unpaired image-to-image translation using cycle-consistent adversarial networks,” in *Proceedings of the IEEE international conference on computer vision*, 2017, pp. 2223–2232.

[41] Y. Liu, S. Feng, S. Liu, Y. Zhan, D. Tao, Z. Chen, and Z. Chen, “Sample-cohesives pose-aware contrastive facial representation learning,” *International Journal of Computer Vision*, pp. 1–19, 2025.

[42] H. Zhou, J. Liu, Z. Liu, Y. Liu, and X. Wang, “Rotate-and-render: Unsupervised photorealistic face rotation from single-view images,” in *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, 2020, pp. 5911–5920.

[43] R. Gross, I. Matthews, J. Cohn, T. Kanade, and S. Baker, “Multi-pie,” *Image and vision computing*, vol. 28, no. 5, pp. 807–813, 2010.

[44] X. Zhang, L. Yin, J. F. Cohn, S. Canavan, M. Reale, A. Horowitz, P. Liu, and J. M. Girard, “Bp4d-spontaneous: a high-resolution spontaneous 3d dynamic facial expression database,” *Image and Vision Computing*, vol. 32, no. 10, pp. 692–706, 2014.

[45] J. Chen, J. Chen, H. Chao, and M. Yang, “Image blind denoising with generative adversarial network based noise modeling,” in *Proceedings of the IEEE conference on computer vision and pattern recognition*, 2018, pp. 3155–3164.

[46] W. Xu, C. Long, R. Wang, and G. Wang, “Drb-gan: A dynamic resblock generative adversarial network for artistic style transfer,” in *Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)*, October 2021, pp. 6383–6392.

[47] B. Allaert, I. R. Ward, M. Bilasco, C. Djeraba, and M. Bennamoun, “A comparative study on optical flow for facial expression analysis,” *Neurocomputing*, 2022.

[48] G. Farnebäck, “Two-frame motion estimation based on polynomial expansion,” in *Scandinavian conference on Image analysis*. Springer, 2003, pp. 363–370.

[49] A. Wedel, T. Pock, C. Zach, H. Bischof, and D. Cremers, “An improved algorithm for tv-1 1 optical flow,” in *Statistical and Geometrical Approaches to Visual Motion Analysis: International Dagstuhl Seminar, Dagstuhl Castle, Germany, July 13-18, 2008*. Revised Papers. Springer, 2009, pp. 23–45.

[50] T. Brox and J. Malik, “Large displacement optical flow: descriptor matching in variational motion estimation,” *IEEE transactions on pattern analysis and machine intelligence*, vol. 33, no. 3, pp. 500–513, 2010.

[51] J. Revaud, P. Weinzaepfel, Z. Harchaoui, and C. Schmid, “Epicflow: Edge-preserving interpolation of correspondences for optical flow,” in *Proceedings of the IEEE conference on computer vision and pattern recognition*, 2015, pp. 1164–1172.

[52] C. Bailer, B. Taetz, and D. Stricker, “Flow fields: Dense correspondence fields for highly accurate large displacement optical flow estimation,” in *Proceedings of the IEEE international conference on computer vision*, 2015, pp. 4015–4023.

[53] D. Sun, X. Yang, M.-Y. Liu, and J. Kautz, “Pwc-net: Cnn s for optical flow using pyramid, warping, and cost volume,” in *Proceedings of the IEEE conference on computer vision and pattern recognition*, 2018, pp. 8934–8943.

[54] Z. Teed and J. Deng, “Raft: Recurrent all-pairs field transforms for optical flow,” in *Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part II 16*. Springer, 2020, pp. 402–419.

[55] A. Eldesokey and M. Felsberg, “Normalized convolution upsampling for refined optical flow estimation,” *arXiv preprint arXiv:2102.06979*, 2021.

[56] S. Jiang, D. Campbell, Y. Lu, H. Li, and R. Hartley, “Learning to estimate hidden motions with global motion aggregation,” in *Proceedings of the IEEE/CVF international conference on computer vision*, 2021, pp. 9772–9781.

[57] D. Poux, B. Allaert, N. Ihaddadene, I. M. Bilasco, C. Djeraba, and M. Bennamoun, “Dynamic facial expression recognition under partial occlusion with optical flow reconstruction,” *IEEE Transactions on Image Processing*, vol. 31, pp. 446–457, 2021.

[58] D. Ulyanov, A. Vedaldi, and V. Lempitsky, “Improved texture networks: Maximizing quality and diversity in feed-forward stylization and texture synthesis,” in *Proceedings of the IEEE conference on computer vision and pattern recognition*, 2017, pp. 6924–6932.

[59] S. Niklaus, L. Mai, and F. Liu, “Video frame interpolation via adaptive separable convolution,” in *Proceedings of the IEEE international conference on computer vision*, 2017, pp. 261–270.

[60] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in *Proceedings of the IEEE conference on computer vision and pattern recognition*, 2016, pp. 770–778.

[61] P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros, “Image-to-image translation with conditional adversarial networks,” in *Proceedings of the IEEE conference on computer vision and pattern recognition*, 2017, pp. 1125–1134.

[62] J. She, Y. Hu, H. Shi, J. Wang, Q. Shen, and T. Mei, “Dive into ambiguity: Latent distribution mining and pairwise uncertainty estimation for facial expression recognition,” in *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, 2021, pp. 6248–6257.

[63] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” *CoRR*, vol. abs/1409.1556, 2014.

[64] P. Lucey, J. F. Cohn, T. Kanade, J. Saragih, Z. Ambadar, and I. Matthews, “The extended cohn-kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression,” in *2010 ieee computer society conference on computer vision and pattern recognition-workshops*. IEEE, 2010, pp. 94–101.

[65] J. Van Der Schalk, S. T. Hawk, A. H. Fischer, and B. Doosje, “Moving faces, looking places: validation of the amsterdam dynamic facial expression set (adfes).” *Emotion*, vol. 11, no. 4, p. 907, 2011.

[66] M. Pantic, M. Valstar, R. Rademaker, and L. Maat, “Web-based database for facial expression analysis,” in *2005 IEEE international conference on multimedia and Expo*. IEEE, 2005, pp. 5–pp.

[67] G. Zhao, X. Huang, M. Taini, S. Z. Li, and M. Pietikäinen, “Facial expression recognition from near-infrared videos,” *Image and vision computing*, vol. 29, no. 9, pp. 607–619, 2011.

[68] A. Radford, L. Metz, and S. Chintala, “Unsupervised representation learning with deep convolutional generative adversarial networks,” *arXiv preprint arXiv:1511.06434*, 2015.

[69] A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A. Desmaison, L. Antiga, and A. Lerer, “Automatic differentiation in pytorch,” *In NIPS Autodiff Workshop*, 2017.

[70] E. Richardson, Y. Alaluf, O. Patashnik, Y. Nitzan, Y. Azar, S. Shapiro, and D. Cohen-Or, “Encoding in style: a stylegan encoder for image-to-image translation,” in *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, 2021, pp. 2287–2296.

[71] P. Truong, M. Danelljan, and R. Timofte, “Glu-net: Global-local universal network for dense flow and correspondences,” in *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, 2020, pp. 6258–6268.

[72] A. P. Fard and M. H. Mahoor, “Ad-corre: Adaptive correlation-based loss for facial expression recognition in the wild,” *IEEE Access*, vol. 10, pp. 26756–26768, 2022.

[73] Z. Wen, W. Lin, T. Wang, and G. Xu, “Distract your attention: Multi-head cross attention network for facial expression recognition,” *Biomimetics*, vol. 8, no. 2, 2023. [Online]. Available: https://www.mdpi.com/2313-7673/8/2/199

[74] A. V. Savchenko, “Video-based frame-level facial analysis of affective behavior on mobile devices using efficientnets,” in *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, 2022, pp. 2359–2366.

[75] M. Tan and Q. Le, “Efficientnet: Rethinking model scaling for convolutional neural networks,” in *International conference on machine learning*. PMLR, 2019, pp. 6105–6114.

[76] A. Mollahosseini, B. Hasani, and M. H. Mahoor, “Affectnet: A database for facial expression, valence, and arousal computing in the wild,” *IEEE Transactions on Affective Computing*, vol. 10, no. 1, pp. 18–31, 2017.

[77] J. Kossaifi, G. Tzimiropoulos, S. Todorovic, and M. Pantic, “Afew-va database for valence and arousal estimation in-the-wild,” *Image and Vision Computing*, vol. 65, pp. 23–36, 2017.