# Seeing the Unseen: Real-Time Micro-Expression Recognition with Action Units and GPT-Based Reasoning
## 3 Materials and Methods
### Experiment  
The experiments were conducted using a high-performance computing setup to ensure efficient processing and reliable results. The hardware configuration included a system equipped with an Intel Core i9 processor (Intel Corporation, Santa Clara, CA, USA), 64 GB of RAM, and an NVIDIA RTX 3090 GPU (NVIDIA Corporation, Santa Clara, CA, USA). On the software side, the development environment was based on Python 3.8, incorporating libraries such as OpenCV 4.6.0 for image processing, MediaPipe 0.8.9 for facial detection, NumPy 1.21.0 and Pandas 1.3.4 for data manipulation, and Scikit-learn 1.0.2 for performance evaluation. Additionally, external support for advanced classification tasks was provided through the integration of the OpenAI ChatGPT-3.5-Turbo API. This setup ensured that the system operated efficiently across all stages of the micro-expression detection pipeline. The system was developed through a series of methodical steps aimed at optimizing performance and robustness. Its effectiveness was then evaluated through systematic experimental validation using CASME II. Given the system's reliance on fine-grained AU-based reasoning and real-time video processing, CASME II was selected as the primary evaluation dataset due to its comprehensive AU annotations, discrete emotion labels, and frame-level temporal segmentation--criteria not fully met by alternative datasets such as SMIC or SAMM. The CASME II dataset served as the primary dataset, with preprocessing steps carefully applied to ensure data consistency and alignment with our experimental objectives. Initially, metadata were extracted from the CASME2-ObjectiveClasses.xlsx file, after which the file names were validated and normalized. Information regarding the onset and offset frames for each sample was retrieved, and the objective classes were mapped to standard emotional labels to facilitate accurate classification.  
A critical aspect of system tuning involved determining the optimal threshold for AU detection. Threshold values of 0.1, 0.15, 0.2, and 0.25 were systematically evaluated. It was observed that thresholds below 0.1 led to excessive AU detection and considerable noise, while thresholds above 0.25 caused many subtle micro-expressions to be missed. An optimal threshold of 0.15 was identified, offering the best balance between sensitivity and specificity in detecting meaningful facial movements.  
Further experimentation focused on prompt formulation strategies for the OpenAI-based classification component. Three different prompt approaches were tested: general prompts, prompts incorporating explicit decision rules, and prompts optimized for known emotional confusions. General prompts yielded a modest classification accuracy of approximately 13%, while the use of explicit rules significantly improved the performance to around 60%. The highest accuracy, 93.3%, was achieved using prompts specifically optimized to address frequent emotional confusions observed during the preliminary evaluations.  
To rigorously assess the system's performance, several validation strategies were employed. Leave-one-subject-out cross-validation was implemented to test the system's generalization capability to unseen individuals. Additionally, a 5-fold cross-validation protocol was used to evaluate the overall robustness of the model. Performance on specific emotion subsets was also analyzed to gain deeper insights into the system's behavior across different emotional categories. Notably, the CASME II dataset presented a substantial class imbalance, with emotions such as fear (86 samples) and disgust (80 samples) being well-represented, while others like sadness were underrepresented with only 3 samples, posing additional challenges to the classification task.