FacialFlowNet: Advancing Facial Optical Flow Estimation with a Diverse Dataset and a Decomposed Model

Authors: jianzhi  lu
ruian  he
shili  zhou
weimin  tan
bo  yan
Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-03-25 03:50:15

Abstract: Facial movements play a crucial role in conveying altitude and intentions, and facial optical flow provides a dynamic and detailed representation of it. However, the scarcity of datasets and a modern baseline hinders the progress in facial optical flow research. This paper proposes FacialFlowNet (FFN), a novel large-scale facial optical flow dataset, and the Decomposed Facial Flow Model (DecFlow), the first method capable of decomposing facial flow. FFN comprises 9,635 identities and 105,970 image pairs, offering unprecedented diversity for detailed facial and head motion analysis. DecFlow features a facial semantic-aware encoder and a decomposed flow decoder, excelling in accurately estimating and decomposing facial flow into head and expression components. Comprehensive experiments demonstrate that FFN significantly enhances the accuracy of facial flow estimation across various optical flow methods, achieving up to an 11% reduction in Endpoint Error (EPE) (from 3.91 to 3.48). Moreover, DecFlow, when coupled with FFN, outperforms existing methods in both synthetic and real-world scenarios, enhancing facial expression analysis. The decomposed expression flow achieves a substantial accuracy improvement of 18% (from 69.1% to 82.1%) in micro expressions recognition. These contributions represent a significant advancement in facial motion analysis and optical flow estimation. Codes and datasets will be available to the public.

Main paper: 
 introduction: 
 The human face could be the most encountered object in a person's life, which emphasizes the vital role of analyzing it [9, 22, 45, 48, 50] . Facial optical flow, representing facial movement, is crucial in applications like micro and macro expression recognition [26] , facial motion capture [54] , facial video generation [25, 52] , and more. Despite its importance, the absence of a dedicated dataset and baseline has hindered its advancement.The primary challenges in facial optical flow estimation involve: 1) Facial expressions arise from facial muscle movements, presenting a non-rigid motion [24] distinct from the rigid motion observed in general datasets like Sintel [7] and KITTI [34] ; 2) Delicate expression movements are often overshadowed by overall head motion, as illustrated by the mouth region in Fig. 1(c ). While state-of-the-art optical flow methods [18, 21, 44] can estimate the entangled facial flow Fig. 1 (b), they are not able to separately isolate these local movements, which are crucial for facial expression analysis.In this paper, we aim to precisely estimate facial optical flow and decompose it into two components: Head Flow, which signifies the isolated rotation and movement of the human head, and Expression Flow, representing the transformation of local facial expressions. We show the optical flow and error map of GMA [21] (left) and our model (right). (c) Our method can generalize to real-world datasets like MEAD [46] and the expression flow can be utilized for downstream analysis.To achieve this, we propose FacialFlowNet (FFN), a large-scale facial optical flow dataset. Our dataset, depicted in Fig. 1(a) , is divided into two parallel segments: FFN-F, comprising images with overall facial motion and labels for facial flow, and FFN-H, offering corresponding frames and labels for head flow. By subtracting head flow from facial flow, we can isolate the expression flow.Leveraging this dataset, we further propose Decomposed Facial Flow (DecFlow), a novel optical flow estimation network featuring a facial semantic-aware encoder and a decomposed flow decoder. As illustrated in Fig. 1 (c ), this approach can accurately estimate facial optical flow and further decompose it into head flow and expression flow, which can facilitate downstream facial analysis like dynamic facial expression recognition.In summary, our main contributions are: 1) We contribute FacialFlowNet (FFN), a large-scale facial optical flow dataset comprising 105,970 pairs of realistic images from 9,635 identities, along with precise optical flow labels for both facial flow and head flow.2) We present DecFlow, the first network capable of decomposing facial optical flow into head and expression flow. The decomposed (a) FlyingThings [32] (b) Sintel [7] (c) Spring [33] (d) FacialFlowNet expression flow achieves a substantial accuracy improvement of 18% (from 69.1% to 82.1%) in micro expressions recognition.3) Extensive experiments demonstrate that FFN significantly enhances the accuracy of facial flow estimation across various optical flow methods, achieving up to 11% reduction in Endpoint Error (EPE) (from 3.91 to 3.48). Moreover, DecFlow outperforms other state-of-the-art methods, providing better insights for the analysis of facial movements. 
related work: 
 General Optical Flow Estimation. Optical flow estimation has been a fundamental vision task ever since this concept was brought by Berthold et al. [17] . Initially approached as an energy minimization problem, it utilized human-designed data and prior terms as optimization objectives [3] [4] [5] [6] 51] . With the evolution of convolutional neural networks and deep learning, contemporary approaches adopt an end-to-end learning paradigm [13, 19, 42] . Notably, RAFT [44] represents a significant advancement in optical flow estimation, it constructs a 4D multiscale correlation volume and utilizes a GRU block to operate flows. Addressing the occlusion problem, GMA [21] and later works [18, 43] propose to use global feature and motion aggregation that could perceive longrange connections. These approaches primarily address general challenges such as fast-moving objects and occlusions. However, they are not designed to meet specific challenges like non-rigid motion and entangled representation in facial optical flow estimation, as mentioned earlier.General Optical flow datasets. Datasets for optical flow estimation can be broadly categorized into real-world [15, 23, 34, 39, 40] data and synthetic data [7, 13, 14, 32, 33, 38, 41] . Among real-world data, KITTI [15, 34] is renowned in the domain of autonomous driving. It offers sophisticated training data derived from intricate device setups. On the synthetic side, Flyingchairs [13] and Flyingthings [32] generate optical flow labels by orchestrating random movements of foreground object models against a background. MPI Sintel [7] , Monkaa [32] and Spring [33] gain the flow-image pairs from rendered animation movie scenes. AutoFlow [41] proposes an approach to search the hyperparameter for rendering training data, while RealFlow [16] synthesizes images using predicted optical flows to generate training data. However, as illustrated in Fig. 2 , FlyingChairs [13] and FlyingThings [32] do not include any motion related to faces. Sintel [7] and Spring [33] do include frames with faces, but their character models have a cartoon style with facial features proportions differing significantly from real faces. Moreover, as observed in the optical flow visualization in Fig. 2 , they treat the face as a rigid object, lacking the local non-rigid motion corresponding to real facial expressions. Therefore, these datasets are less suitable for predicting facial optical flow. In contrast, our dataset features realistic facial characteristics and simulates expressive movements.Facial Optical Flow Estimation. While the above datasets and methods have achieved considerable success, the field of facial optical flow estimation has been overlooked. DeepFaceFlow [24] proposes a 3D facial flow dataset from 1,600 subjects and a corresponding network. Flow information is derived by calculating differences between face meshes obtained through 3D reconstruction based on images. However, disparities between meshes and actual human faces mean the flow in the dataset does not fully correspond to facial movements. Alkaddour et al. [1] presents a self-supervised approach and a dataset that consists of 41 subjects. They generate flows by partitioning faces into triangular meshes and calculating the local motion for each triangle. This partitioning may propose distinct artifacts in the ground truth of optical flow, making the labels unreliable. Unfortunately, the above datasets are not publicly available. Therefore, we contribute the FacialFlowNet dataset to accelerate the facial optical flow research with more diverse face images (9,635 identities) and more accurate optical flow labels with a rendering engine like Blender1 . 
facialflownet dataset: 
 We generate a realistic facial flow dataset with 9,635 unique faces, each displaying diverse shapes, expressions, and head poses. It comprises 105,970 image pairs at 512x512 pixels resolution, with corresponding flow labels. Similar to previous works [7, 13, 32, 33, 38] ,we utilize Blender and its Cycles rendering engine for generating synthetic frames and flow labels. Our pipeline, illustrated in Fig. 3 , takes a UV texture, a set of FLAME parameters, a background image, and camera/light parameters as input. It outputs video sequences with lengths of 5, 10, 15, or 20 frames, along with corresponding optical flow labels. The following sections provide details on the modules used for dataset generation. EQUATIONwith vertices 𝑉 ∈ R 𝑛 𝑣 ×3 and 𝑛 𝑓 = 9976 faces 𝐹 ∈ R 𝑛 𝑓 ×3 . Face reconstruction: We choose EMOCA [10] as our reconstruction method, this approach takes a single in-the-wild image and reconstructs a 3D face with sufficient facial expression detail to convey the emotional state of the input image. It enables us to generate high-quality face meshes with realistic expressions. Dataset: For our dataset, which necessitates diversity in expressions, distinct identity faces, and high-quality images, we opt for AffectNet [35] . AffectNet stands out as one of the largest databases for facial expression, valence, and arousal. It contains more than 1M images with faces and extracted landmark points. We specifically select approximately a thousand images from each of the nine categories: neutral, happy, sad, surprise, fear, disgust, anger, contempt, and none, as the input for the reconstruction module. FLAME comes with an appearance model, which is converted from Basel Face Model's albedo space [37] to FLAME's UV layout [29] . To enhance the realism of our dataset, high-fidelity, and quality texture maps are essential. However, as depicted in Fig. 4 (c), the existing method [28] yields low-quality, low-resolution textures, potentially resulting in unrecognizable and detail-lacking reconstructed results. AffectNet's in-the-wild variations further challenge texture quality. FFHQ-UV [2] offers a solution to this issue. Dataset: FFHQ-UV [2] proposes a StyleGAN-Based Facial Image Editing module, creating FFHQ-Norm with consistent lighting, neutral expressions, and no occlusions from in-the-wild images. This dataset serves as the ideal input for our UV-texture Extraction module. Method: FFHQ-UV's [2] textures are incompatible with FLAME, for they use the facial parametric model HiFi3D++ [8] , which differs from FLAME in terms of both topology and vertex count. So we adapt their pipeline using a FLAME-based reconstruction method and create a high-quality, FLAME-based UV-texture dataset. Refer to the supplementary materials for details.Image Generation: For image generation, we use the open-source 3D creation suite Blender. Inspired by [20, 32, 38] , we aim to improve the network's robustness by integrating real photos as backgrounds in our dataset. We randomly select 400 background images from the internet, crop them to the same size, and ensure they encompass various indoor and outdoor scenes. During rendering, we position the image as a stationary plane behind the head model, manually adjusting the camera and lighting parameters to align the rendered results with the source images.Ground Truth Generation: Modifying Blender's internal render engine pipeline enables the passage of vectors between different frames. The render pass is typically used for producing motion blur [7, 13, 32, 33, 38] , and it produces the motion in the image space of each pixel; i.e., the ground truth of optical flow. Additionally, we also employ it to generate ground truth depth information, examples are available in the supplementary material.Facial Expression Diversity: We expect our dataset to inherit AffectNet's expression diversity. [35] . To confirm this, we employ DAN [47] , a facial expression recognition network, to classify the 9000 faces selected from AffectNet [35] and our dataset. UV-Texture Diversity: To assess identity diversity in our UV-Textures dataset, we calculate the identity vector using Arcface [11] for each image. The standard deviation and coefficient of variation of these vectors measure the identity variations. Tab. 1 presents evaluation results of the original dataset (FFHQ-Norm), and three rendered datasets with different UV-textures (FFHQ-UV [2] , PO.[28] and ours). It shows that our textures preserve the most identity variations in FFHQ-Norm, whereas PO. hardly preserves identity differences. the identity preservation from FFHQ-Norm to the rendered dataset, we also calculate the identity cosine similarity between each image in FFHQ-Norm and the rendered images. Tab. 1 indicates that the average identity similarity score of our dataset reaches 0.78, suggesting that our UV-textures can preserve the most facial features and details. 
decflow architecture 4.1 overview: 
 Our overall network diagram is shown in Fig. 6 . We base our approach design on the successful recurrent architecture in RAFT [44] and GMA [21] . However, the previous architecture lacks the perception of facial semantic information and only provides an entangled optical flow for face movements. Therefore, we propose a facial semantic-aware encoder and a decomposed decoder to address the problem. Our model takes in a pair of consecutive frames of face movement and extracts the correlation volume and context feature. The decoder takes the context features and motion features as input, producing aggregated motion features that share information across the image. Finally, the two decoders predict the facial flow and head flow separately and obtain the expression flow by subtracting the head flow from the facial flow.Inspired by the extra encoders in SAMFlow [53] and MatchFlow [12] , we posit that incorporating features with facial semantics can enhance the accuracy of facial optical flow prediction, given the relatively fixed structure of the human face (nose, eyes, and mouth).To extract the 2D context features, we start by individually using the encoder of DAD-3DNet [31] and the context network of GMA [21] to encode the first image. The results from these encoders are concatenated and passed through a residual convolutional block to reduce channels and fuse features. Mathematically, this process can be expressed as:EQUATIONwhere 𝐸 𝐺 and 𝐸 𝐷 are the GMA context network and DAD-3DNet encoder. The symbol ⊕ denotes the concatenation operator, and 𝑅𝑒𝑠 corresponds to the residual convolutional block. During training, we freeze the parameters of DAD-3DNet's encoder and utilize only the pre-trained weights. This approach allows us to obtain context features with facial semantic information. In parallel, we employ GMA's cost volume encoder to extract 2D matching features and compute a 4D correlation volume.We further propose a facial decomposed flow decoder that can decompose the facial motion into the head flow and expression flow. We adopted the same decoder structure as GMA, but the key difference is that we employed two parallel decoders to independently predict facial and head flow. The head flow 𝐹 ℎ is subtracted from the facial flow 𝐹 𝑓 to obtain the expression flow 𝐹 𝑒 : 𝐹 𝑒 = 𝐹 𝑓 -𝐹 ℎ . In summary, our network takes two frames as input and outputs three types of flows: facial, head, and expression flow. We first train the facial flow decoder and fix the parameters to train the head flow decoder. It is because the two decoders have separate optimization goals and can not simply co-train, which will cause a performance drop in facial flow (Tab. 5). When training the head flow decoder, we constrain head flow and expression flow as optimization targets. This way, the decoder can perceive the expression difference.To supervise the estimating results, we define separate loss functions for facial, head, and expression flow. When training for the facial flow decoder, we use the optical flow loss in RAFT [44] . Then, we train the head flow decoder with the decomposed optical flow [44] 0.275 0.316 0.433 0.695 0.354 GMA [21] 0.267 0.306 0.422 0.664 0.343 SKFlow [43] 0.242 0.276 0.388 0.635 0.314 FlowFormer [18] 0 [44] for head flow and expression flow. 
experiments 5.1 implementation details: 
 We follow the standard optical flow training procedure [19, 42] of first training GMA [21] on FlyingChairs [13] for 120k iterations (batch size 8) and FlyingThings [32] for 120k iterations (batch size 6). Further training on MPI Sintel [7] for another 120k iterations (batch size 6). We then train our DecFlow on FacialFlowNet for 10k iterations (batch size 6) separately for the facial flow decoder and the head flow decoder, with the pre-trained weights from GMA.The training is performed on two 3090 GPUs with PyTorch [36] , following GMA's hyperparameters and strategy [21] .FacialFlowNet (FFN) is divided into training, testing, and validation sets with a ratio of 97:2:1, comprising 90,193 pairs, 10,395 pairs, and 5,382 pairs of images respectively. MEAD [46] contains high-resolution video frames with rich facial movements. From this dataset, we select 26 videos filmed from a frontal perspective, comprising a total of 3,402 frames. This subset is used to evaluate our method's efficacy in analyzing real facial dynamics. Additionally, we apply the method by Alkaddour et al. [1] to generate flow labels for 20,445 image pairs extracted from MEAD, denoted as FMEAD, for comparison with dedicated facial optical flow datasets. CK+ [30] has 593 acted facial expression sequences from 123 participants. We extracted the first and last frames of these sequences, resulting in 1,186 images for evaluating our method's performance in real facial optical flow estimation.CASME II [49] contains 256 micro-expression videos from 26 subjects, featuring five prototypical expressions: happiness, disgust, repression, surprise, and others.The primary evaluation metric is the average end-point error (EPE).To address the static background in FFN, we calculate the average EPE within the moving regions using masks derived from depth information. Refer to the supplementary materials for more details. Note that, due to the relatively small facial motion magnitude, the differences between evaluation metrics are also small. We divide the test set into four groups based on video length and evaluate various recent methods. The results are shown in Tab. 2. For fairness, we also finetune RAFT [44] , SKFlow [43] , FlowFormer [18] , and GMA [21] on our training set. Our method achieves 7.0% higher accuracy than GMA, which indicates our approach can achieve superior accuracy while also being able to independently estimate facial flow, head flow, and expression flow.We conduct experiments to validate our method's effectiveness in real facial flow estimation, addressing the domain gap between synthetic and real data. Due to the absence of ground-truth flow labels for real images, we compare the estimated flows with 3DMM's vertices and facial landmarks. Evaluation with 3DMM's vertices: MICA [54] provide a metrical monocular tracker that can track facial motion by performing precise 3D face reconstruction on every frame of the video. Each reconstructed face mesh consists of 1787 facial vertices. We use it to perform facial motion tracking on images from MEAD [46] and CK+ [30] , obtaining facial vertex coordinates for each image. As shown in Fig. 7 , vertices were categorized into lips, forehead, cheeks, nose, and eye regions. For precision, we only used the 1435 vertices from the lips, cheeks, and eye regions, as the vertices in the nose and forehead regions remained relatively stationary.To compare optical flow with the selected vertices, we use EPE as our metric. For each pair of images, such as 𝐼 1 and 𝐼 2 , we obtain the facial vertex coordinates 𝐶 1 , 𝐶 2 , and the optical flow 𝐹𝑙𝑜𝑤. The average EPE was then calculated using the following formula: In Tab. 3, FFN-F enhances the accuracy of all methods in estimating facial optical flow, with FlowFormer achieving the greatest improvement, reducing the EPE by 5.8% (from 4.96 to 4.67). However, finetuning on FMEAD leads to performance declines for all models. That's because Alkaddour's [1] method heavily relies on precise facial landmark predictions and triangle transformations, resulting in noticeable artifacts and inaccuracies in the flow labels. These findings underscore the superiority of FFN over both general datasets [7, 13, 32] and dedicated facial dataset [1] .EQUATIONFurthermore, DecFlow, which incorporates the ability to decompose facial flow, achieves comparable or superior accuracy in real facial optical flow estimation. Overall, with FFN and DecFlow, we achieve an accuracy improvement of up to 4.9% (from 4.91 to 4.67) compared to our baseline method (GMA with C+T+S). Evaluation with facial landmarks: Due to the potential discrepancies between the reconstructed vertices and real faces, we also utilize the facial landmarks labels provided by CK+, as shown in Fig. 8 . Since these labels are manually annotated, the evaluation results would be more precise. With these labels, we calculate the EPE using Eq. ( 6). The results in Tab. 3 demonstrate that FFN significantly improves the accuracy of various methods, achieving a maximum reduction of 11.0% (3.91 to 3.48) EPE for FlowFormer. Compared to the baseline (GMA with C+T+S), we achieve a maximum improvement of 10.1% (3.86 to 3.47) in facial flow accuracy. Visual examples in Fig. 8 illustrate that our method, in contrast to GMA, accurately predicts movements in the eyebrow and lip regions, aligning more closely with the ground truth.We choose micro-expressions recognition as the downstream task and employ MMNet [26] to assess the performance of various optical flow networks. Then, we estimate the optical flow of the onset frames and apex frames in the original, untrimmed, and unaligned videos. These flows serve as inputs for the main branch of MMNet to learn motion-pattern features.Tab. 4 shows the performance comparison of different optical flow models in both 5-class and 3-class classification tasks. After finetuning on FFN-F, various methods showed different degrees of [21] with general datasets (C+T+S), our approach and dataset demonstrated an improvement of up to 13.0%/0.191 and 9.6%/0.154 of accuracy/F1-score in both 5-class and 3-class classification. It highlights the superiority of our approach in estimating facial optical flow and the significance of decomposed expression flow in analyzing facial movements.Qualitative evaluation results of various methods are presented in Fig. 9 . The samples in rows 1-4 have small emotional movements, entirely obscured by head motion. However, by using our method, these masked expression flows can be clearly decomposed. The examples in rows 5-8 depict expressions with intense emotional features and noticeable movements. Consequently, in the facial flow, the motion regions of these expressions are more pronounced. Compared to other methods, our approach predicts more accurate facial flow, visualized as clearer facial features, rather than focusing solely on large motion regions. Furthermore, by observing the last Figure 9 : Qualitative results on real-world images from CASME II [49] (1-4), MEAD [46] (5,6), and the Internet (7, 8) . 𝐷𝑒𝑐𝐹𝑙𝑜𝑤 𝑓 , 𝐷𝑒𝑐𝐹𝑙𝑜𝑤 ℎ , and 𝐷𝑒𝑐𝐹𝑙𝑜𝑤 𝑒 denote facial, head, and expression flow obtained by our method, respectively. column of Fig. 9 , we can find that the decomposed expression flows normalize different facial movements to a similar range, visualized by similar colors. This information is crucial for facial expression analysis.Tab. 3 demonstrate that finetuning on FacialFlowNet enhances the accuracy of multiple baselines [18, 21, 43, 44] in both synthetic and real-world datasets, confirming the effectiveness of our dataset. Tab. 4 also highlights the significant advantage of expression flow over facial flow in micro-expressions recognition, confirming the effectiveness of our decomposed flow decoder. Subsequently, an ablation study validates our facial semantic-aware decoder, as presented in Tab. 5. Compared with GMA, adding another decoder for head flow estimation alone may affect the accuracy of facial flow. However, with the inclusion of the facial semantic-aware encoder, the network achieves superior accuracy compared to GMA while demonstrating the ability to decompose the facial flow. 
conclusion: 
 This paper focuses on the challenges in non-rigid motion and entangled representation in facial flow estimation. We contribute  
 
 Appendix: https://www.blender.org 


Decision: The manuscript proposes a face based optical flow method. The dataset and flow information is generated using Blender based rendering. A flow method is also presented and evaluation is performed on multiple relevant databases.

The reviewers indicate the novelty of the work. The rebuttal has clarified the reviewer queries.

Reviews: review: paper_summary: paper_summary: The introduction of FacialFlowNet dataset comprising a large-scale collection of facial optical flow data with over 9,600 identities and 105,970 image pairs is a notable contribution to the field. The dataset addresses the scarcity of comprehensive facial optical flow datasets and provides a valuable resource for training and evaluation purposes.
strength_weakness: strength_weakness: The paper introduces DecFlow, a novel network designed to decompose facial optical flow, which is a significant contribution to the field.
limitations: limitations: The dataset would be beneficial for the paper to discuss in more detail the process of data collection, annotation, and potential biases present in the dataset to ensure its reliability and generalizability.

The evaluation of the proposed methodology could be strengthened by providing a more comprehensive comparison with existing state-of-the-art methods. Specifically, discussing how DecFlow performs compared to other facial flow estimation techniques in terms of accuracy, computational efficiency, and robustness across different datasets and scenarios would enhance the credibility of the proposed approach.
paper_summary: paper_summary: This work aims to decouple the facial optical flow into head motion optical flow and facial expression optical flow. To achieve this goal, it starts by rendering a synthetic dataset, FacialFlowNet (FFN), using UV-texture extracted from FFHQ-Norm and facial expression extracted from AffectNet. Then, it builds a facial optical flow detection model, DecFlow, using RAFT and GMA, composed of two decoders designed to detect facial flow and head flow. Finally, the expression flow can be obtained by subtracting the head flow from the facial flow. Experiments on several benchmarks showcase the efficacy of DecFlow.
strength_weakness: strength_weakness: * Extensive ablation studies verify the effectiveness of different components of the dataset creation design and the model efficacy.
* Experiments on several benchmarks showcase the superiority of the introduced DecFlow
* The paper is well structured, and it is easy to follow.
limitations: limitations: * It is difficult to understand the main purpose of separating head flow from facial flow. I am curious about the main practical applications that require decoupling these two types of flows. Providing some figures to showcase a simple application would be much better. For example, is it possible / easier to perform expression editing with the decoupling?   How about enhancing the intensity of an expression, e.g., making a small smile into a big smile?

* It is unclear why the two decoders cannot be co-trained. L559 mentioned that it is because the two decoders have different objectives, and co-training them will lead to a performance drop. However, I do not understand which experiment in Tab. 5 showcases this conclusion. Besides, I am not convinced why it cannot be co-trained, given that each of them has a specific loss function to supervise. Does it suggest that combined training will lead to worse performance? It sounds counter-intuitive.
paper_summary: paper_summary: The paper introduces FacialFlowNet (FFN), which is a novel large-scale facial optical flow dataset. Based on this dataset, the authors also propose the Decomposed Facial Flow Model (DecFlow), the first method capable of decomposing facial flow. Extensive experiments demonstrate the importance of FFN and the effectiveness of DecFlow.
strength_weakness: strength_weakness: - This work can bring contributions to multiple communities related to face analysis and face generation. Focused on faces, the proposed dataset provides precise optical flow labels for both facial flow and head movement. The proposed model has also been verified to achieve more accurate optical flow estimation compared to existing models. 
- The paper is well-organized and easy to follow.
- The experiments are comprehensive, and the experimental results show that the proposed dataset and method can promote more accurate facial optical flow estimation.
limitations: limitations: - In the paper, the acquisition of GT comes from the render pass. I am curious whether it is possible to calculate optical flow using the vertices obtained from 3D Face Reconstruction. Specifically, since the vertices between two adjacent frames are directly corresponded, we can explicitly calculate the movement of each vertex, and then project it onto 2D to obtain facial optical flow. Compared with the method in this paper, the dataset obtained by this method can have real-world visual data instead of synthetic data.
- In general, the learning difficulty in a two-stage learning process is progressive. However, in the paper, the first stage requires the model to already be able to fully predict facial optical flow. Since expression flow can be derived by subtracting the head flow from the facial flow, the main task of the second stage is to predict the head flow, which is less challenging than the first stage. Moreover, the two decoders seem somewhat redundant; they should have some modules that can be shared. Streamlining the model structure could improve the model's efficiency.