# Late Fusion-Based Video Transformer for Facial Micro-Expression Recognition
## 6 Conclusions  
Recently, various studies have been proposed to solve FME recognition, but no studies have used transformers. In this research, we focused on whether the transformer model can be suitable for FME recognition. Since transformers generally require a large amount of data but there is no sufficient dataset for FME recognition, our main purpose is to train the transformer model successfully. We achieve the purpose using the optical flow, which was mainly used by video processing models, and LF with the transformer. As a result, our model becomes the SOTA in SMIC and achieves comparable performance in CASME II, although we do not use methods specialized for FME recognition as in other studies.  
The contribution of the authors for this publication article are as follows: J.H.: methodology, software, conceptualization, writing--original draft, writing--review and editing. C.L.: investigation, writing--review and editing. H.J.: conceptualization, project administration, supervision, writing--review and editing. All authors have read and agreed to the published version of the manuscript.  
This work was supported by Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea government(MSIT) (2019-0-00330, Development of AI Technology for Early Screening of Infant/Child Autism Spectrum Disorders based on Cognition of the Psychological Behavior and Response). Furthermore, this research was also supported in part  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Backbone**} & **Video Length** & **Preprocess** & **Fusion** & **UF1** & **UAR** &
\begin{tabular}{c} **Acc** \\ (\%) \\ \end{tabular} \\ \hline \multirow{8}{*}{3 DResNet [27]} & \multirow{8}{*}{8} & \multirow{2}{*}{RGB + OF} & Early & 0.6291 & 0.6403 & 64.02 \\ \cline{3-6}  & & & Late & 0.6759 & 0.6624 & 68.90 \\ \cline{3-6}  & & & Early & 0.6087 & 0.6082 & 64.63 \\ \cline{3-6}  & & & Late & 0.6142 & 0.6096 & 62.20 \\ \cline{3-6}  & & & Early & 0.6476 & 0.6540 & 65.24 \\ \cline{3-6}  & & & Late & 0.6598 & 0.6464 & 67.07 \\ \cline{3-6}  & & & Early & **0.6888** & **0.6773** & **71.34** \\ \cline{3-6}  & & & Late & 0.6034 & 0.6151 & 61.59 \\ \hline \multirow{8}{*}{Video Swin [20]} & \multirow{8}{*}{8} & \multirow{2}{*}{RGB + OF} & Early & 0.6481 & 0.6648 & 64.63 \\ \cline{3-6}  & & & Late & 0.7027 & 0.7309 & 70.12 \\ \cline{3-6}  & & & Early & 0.6977 & 0.6958 & 69.51 \\ \cline{3-6}  & & & Late & 0.6638 & 0.7005 & 66.46 \\ \cline{3-6}  & & & Early & 0.6301 & 0.6416 & 62.80 \\ \cline{3-6}  & & & Late & 0.7013 & 0.7208 & 70.12 \\ \cline{3-6}  & & & Early & 0.6226 & 0.6168 & 62.80 \\ \cline{3-6}  & & & Late & **0.7447** & **0.7377** & **73.17** \\ \hline \multirow{8}{*}{TimeSFormer [28]} & \multirow{8}{*}{8} & \multirow{2}{*}{RGB + OF} & Early & 0.6632 & 0.6834 & 65.85 \\ \cline{3-6}  & & & Late & 0.6914 & **0.6983** & 68.90 \\ \cline{1-1} \cline{3-6}  & & & Early & 0.6632 & 0.6834 & 65.85 \\ \cline{1-1} \cline{3-6}  & & & Late & 0.6536 & 0.6499 & 65.24 \\ \cline{1-1} \cline{3-6}  & & & Early & **0.7038** & 0.6953 & **71.34** \\ \cline{1-1} \cline{3-6}  & & & Late & 0.6474 & 0.6452 & 64.02 \\ \cline{1-1} \cline{3-6}  & & & Early & 0.6471 & 0.6703 & 64.63 \\ \cline{1-1} \cline{3-6}  & & & Late & 0.6615 & 0.6594 & 66.46 \\ \hline \end{tabular} Note: **Bold** show the best performance of each model in each metric.  
\end{table}
Table 6: Comparison of early vs. late fusion.  
by a National Research Foundation of Korea (NRF) grant funded by the Korean government (MSIT) (No. 2020R1C1C1007423). In addition, this work was partly supported by the National Research Foundation (NRF), Korea, under project BK21 FOUR.  
**Institutional Review Board Statement:** Not applicable.  
**Informed Consent Statement:** Not applicable.  
**Data Availability Statement:** Data sharing not applicable.  
**Conflicts of Interest:** The authors declare no conflict of interest.