# LGNMNet-RF: Micro-Expression Detection Using Motion History Images
## 6 Conclusions  
This article proposed a deep learning approach called the LGNMNet-RF method based on MHI features to detect ME and MaEs in long videos. When using the temporal features of MHI, these essential features achieved the best results. In addition, the experiment results showed that most of the datasets for the detection of only ME or MaE showed that the Random Forest classifier improved the accuracy more than the conventional Softmax classifier. The detection F1-score for the CAS(ME)\({}^{2}\) dataset was 0.3019 for ME and 0.3400 for MaE. The overall F1-score was 0.3594. On the other hand, the detection F1-score for the SAMM-LV dataset was 0.3604 for ME and 0.4444 for MaE, with the overall F1-score being 0.4118. This F1-score for ME surpasses those of related works and shows that the MHI features with pseudo-labelling significantly improve detection accuracy. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Network Backbone** & **TP** & **FP** & **FN** & **F1-Score (ME)** \\ \hline ShuffleNet V2 & 4 & 20 & 53 & 0.0988 \\ \hline MobileNet V2 & 9 & 54 & 48 & 0.1500 \\ \hline Convnet & 5 & 29 & 52 & 0.1099 \\ \hline MaxVit & 2 & 19 & 55 & 0.0513 \\ \hline MobileNet V3 & 54 & 250 & 105 & 0.2333 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: F1-scores \(\uparrow\) of ME detection with different backbones for CAS(ME)\({}^{2}\) with MHI as input feature. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Input Image** & **Classifier** & **TP** & **FP** & **FN** & **F1 (ME)** \\ \hline \multirow{4}{*}{MHI} & Softmax & 9 & 74 & 48 & 0.1374 \\ \cline{2-5}  & Random Forest & 16 & 33 & 41 & 0.3019 \\ \cline{2-5}  & SVC & 15 & 46 & 42 & 0.2542 \\ \cline{2-5}  & Linear Regression & 9 & 42 & 48 & 0.1667 \\ \cline{2-5}  & kNN & 9 & 35 & 48 & 0.1782 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: F1-scores \(\uparrow\) of ME detection with different input features and classifiers of CAS(ME)\({}^{2}\). Since the manual settings of the \(k\) parameter were used, it is possible that the \(k\) length of fixed time for ME intervals was not the best choice for each dataset. In the future, a more optimal \(k\) parameter could be researched in an in-depth study. It is possible to examine the reasonability and proximity of the FP intervals to the GT intervals by analysing the anticipated TP and false positive (FP) intervals. The reasonability of false positives can be assessed by analysing near-missed periods, even in cases where the IoU is not fulfilled. This study uses three input features: MHI, optical flow, and optical flow with optical strain. The same backbone architecture can be used to analyse each feature's TP intervals and determine whether they differ from or are the same as each input feature. Following this analysis, a late fusion approach with a two- or three-stream architecture could be suggested. Testing the early fusion method to determine if it can increase the detection task's accuracy is also possible. While computation time is an important factor in practical applications, the primary focus of this research was on improving the accuracy of micro-expression detection by advancing the methodologies for a better detection performance. In future studies, the potential optimisation of computation time to make the method more applicable in practical settings will be the focus. Additionally, while our method performs well when faces are directly facing the camera, under conditions similar to those of the training dataset, its robustness in varied scenarios, such as different environments and facial orientations, remains an area for future studies. One of the main challenges in extending this work lies in the difficulty of collecting micro-expression datasets under diverse conditions, given the subtle and brief nature of these expressions, which require precise and controlled environments for accurate labelling. To ensure fairness compared to existing methods, this study focused on widely used benchmark datasets, a common practice in the research community.