# Seeing the Unseen: Real-Time Micro-Expression Recognition with Action Units and GPT-Based Reasoning
## 4 Results
### Confusion Matrix Analysis  
The confusion matrix from Figure 5 provides a clear perspective on the performance of the micro-expression detection system using the CASME II dataset.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Micro-Expression** & **Precision** & **Recall** & **F1-Score** & **Support Image Number** \\ \hline Disgust & 1.00 & 0.86 & 0.93 & 80 \\ Fear & 0.99 & 1.00 & 0.99 & 86 \\ Happiness & 1.00 & 0.94 & 0.97 & 17 \\ Repression & 0.97 & 1.00 & 0.98 & 31 \\ Sadness & 0.30 & 1.00 & 0.46 & 3 \\ Surprise & 0.68 & 1.00 & 0.81 & 17 \\ Others & 1.00 & 0.75 & 0.86 & 20 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Performance for each emotional class.  
This analysis highlights the strengths and challenges of the system in correctly identifying the seven emotion categories. The main diagonal of the matrix, representing correct predictions, showed dominant values (72, 84, 15, 16, 30, 3, 17), indicating an overall good system performance. The confusion matrix indicates that the system performed well overall, with most emotions being correctly classified. Notably, disgust (72/80), fear (84/86), repression (30/31), and surprise (17/19) were recognized with high precision, highlighting the system's strength on well-represented and structurally distinct classes.  
However, happiness and others showed moderate confusion, with two instances of happiness misclassified as surprise and others. This may be due to overlapping AU patterns such as AU12 appearing weakly in other expressions.  
The most significant challenge remains sadness, with only three available samples, all misclassified--reflecting the impact of severe class imbalance and the lack of representative AU patterns in training.  
These observations confirm that the recognition quality depends not only on AU distinctiveness, but also on sample distribution, supporting the importance of both rule-based and data-driven components.  
The real-time performance of the proposed system, as depicted in Figure 6, demonstrates its operational viability for practical micro-expression recognition. At a processing rate of 30 FPS (0.033 s/frame), micro-expressions of 100 ms duration were captured across approximately three frames, while shorter expressions of 40-50 ms were preserved in \(\sim\)1.5 frames, providing fine-grained temporal resolution consistent with the established benchmarks for high-sensitivity facial analysis. Even when operating at 24 FPS (0.042 s/frame), the system maintained an adequate resolution, capturing 100 ms events in \(\sim\)2.4 frames and shorter events in at least one frame. This satisfies the minimum functional threshold of 20 FPS, widely cited as the baseline requirement for real-time facial behavior monitoring systems (as noted in [9,44]). The observed drop-in frame rate during expression processing was expected due to the additional computational load  
Figure 5: Confusion matrix.  
involved in landmark tracking, AU analysis, and language model inference. However, this drop remained within the acceptable performance envelope (24-30 FPS) for real-time applications in human-computer interaction, psychological assessment, and security contexts. This temporal resolution enabled the system to capture the onset and apex of micro-expressions--critical phases for accurate emotion interpretation--thus preserving detection integrity even under moderate computational constraints.  
Figure 6: Cont.  
Classification latency using the OpenAI API was recorded as the round-trip duration between the API request submission and receipt of the model's response. This measurement does not reflect a consistent runtime performance, as it depends on the external server load and network latency. For this reason, only the local processing time (AU extraction and rule-based classification) was used in the timing comparisons.