# Seeing the Unseen: Real-Time Micro-Expression Recognition with Action Units and GPT-Based Reasoning
## 3 Materials and Methods
### Materials  
To develop our proposed system, we integrated multiple cutting-edge technologies, each selected for its specific capabilities and strengths. Python 3.8 served as the primary development language, providing a flexible and powerful foundation for implementation. OpenCV 4.6.0 was employed for image and video processing tasks, while the MediaPipe 0.8.9 framework from Google (Mountain View, CA, USA) enabled accurate facial landmark detection. For efficient data manipulation and handling, we utilized the NumPy 1.21.0 and Pandas 1.3.4 libraries. Scikit-learn 1.0.2 was used to evaluate the system performance through established machine learning metrics. Additionally, the OpenAI (San Francisco, CA, USA) ChatGPT-3.5-Turbo API was incorporated to perform advanced classification tasks. Finally, Matplotlib 1.3.0 and Seaborn 0.13.2 were applied to visualize the results and facilitate comprehensive analysis. The proposed micro-expression detection system adopts a modular architecture to ensure clarity, flexibility, and robustness across all processing stages, as shown in Figure 1. The workflow begins with the **Video Preprocessing Module**, where raw video streams are enhanced and prepared for analysis. This is followed by **MediaPipe Facial Detection**, which identifies and tracks facial landmarks with high precision. The detected features are then processed by the **Facial AU System**, enabling a structured representation of facial movements. To further refine interpretation, the **Expert Rules Module** applies specialized decision rules aimed at resolving typical emotional ambiguities. Subsequently, the **Amplification of Weak Signals** stage enhances subtle facial cues that are characteristic of micro-expressions. The refined data are passed to the **Classification Ensemble**, where multiple classifiers work collectively to improve recognition accuracy. Finally, the system outputs the **Identified Micro-Expression**, offering a comprehensive and accurate analysis of subtle emotional expressions. This modular approach, while promoting scalability and easy updates, also allows for targeted improvements in individual system components. Figure 1: Proposed modular approach in the micro-expression detection system. Our proposed system for real-time micro-expression detection was tested on the CASME II dataset, and its general organization is presented in Figure 2.  
The processing flow begins with extracting relevant frames from videos or web camera video streams, followed by face and facial point detection in each frame. This information is then converted into facial AUs, which are used to infer emotions through multiple complementary methods. The system includes feedback and self-improvement components that allow adaptation to specific subject and context characteristics. **The Video Preprocessing Module** uses OpenCV and is responsible for extracting frames from video sequences or real-time web camera streams, performing normalization, facial alignment, and temporal filtering. **The MediaPipe Facial Detector** uses the MediaPipe Face Landmarker model to extract facial points and expressions. It provides 468 high-precision facial landmarks, facial blendshape estimations representing specific facial configurations, and stable tracking between frames for continuous analysis. This detector offers significant advantages compared with traditional approaches: high precision even for subtle movements, robustness to lighting and position variations, and computational efficiency for real-time processing. **Facial AU System**--An important aspect of the system is converting MediaPipe blendshapes to standardized AUs according to the FACS system. This mapping allows for the use of existing scientific knowledge about the relationship between AUs and emotions. The system implements comprehensive mapping, as shown in Table 2. This detailed mapping covers all AUs relevant to micro-expressions. Figure 2: System architecture. **The Expert Rules Module** implements rules based on specialized knowledge for micro-expression classification.