# Seeing the Unseen: Real-Time Micro-Expression Recognition with Action Units and GPT-Based Reasoning
## 3 Materials and Methods
### Materials
_For Disgust:_  
'AU9': 3.0, # Massive amplification for \"noseWrinkler\" (Disgust)  
'AU10': 2.5, # Amplification for \"upperLipRaiser\" (Disgust)  
_For Happiness:_  
'AU12': 3.0, # Massive amplification for \"lipCornerPuller\" (Happiness)  
'AU6': 2.0, # Amplification for \"cheekRaiser\" (Happiness)  
_For Repression:_  
'AU4': 2.0, # Amplification for \"browLower\" (Repression, Fear)  
'AU7': 3.0, # Massive amplification for \"lidTightener\" (Repression)  
_For Fear_ vs. _Surprise:_  
'AU20': 3.0, # Massive amplification for \"lipStretcher\" (Fear)  
'AU5': 2.0, # Amplification for \"upperLidRaiser\" (Surprise, Fear)  
_For Sadness--resolving confusion:_  
'AU15': 3.0, # Massive amplification for \"lipCornerDepressor\" (Sadness)  
'AU1': 2.0, # Amplification for \"browInnerRaiser\" (Sadness, Surprise, Fear)  
'AU17': 2.0, # Amplification for \"chinRaiser\" (Sadness)  
_For differentiating Surprise_ vs. _Fear:_  
'AU26': 2.5, # Amplification for \"jawOpen\" (Surprise)  
'AU2': 1.8, # Amplification for \"browOuterUpLeft\" and \"browOuterUpRight\" (Surprise, Fear)  
This approach is based on the principle that certain AUs are more diagnostic for specific emotions, and therefore deserve a higher weight in the decision-making process. **Classification Ensemble** combines multiple methods to improve the micro-expression classification accuracy, in the following order:  
1. Rule-based classification--direct application of expert rules for clear cases;
2. Prototype similarity--comparing detected AU configurations with known patterns for each emotion;
3. Contextual analysis--considering subject-specific characteristics;
4. Language model assistance--using OpenAI for difficult cases. This hybrid approach allows the system to combine the advantages of explicitly coded expert knowledge with the flexibility and adaptability of learning-based methods. For similarity cases, a function was used that calculates the Jaccard score between the observed AU configuration and prototypes for each emotion, thus providing an objective measure of similarity. The integration of the LLM, specifically OpenAI's GPT-3.5-turbo, into the proposed micro-expression recognition system is illustrated in Figure 4. This component is activated when the rule-based classifier fails to produce a clear emotion label, typically due to ambiguous or overlapping AU patterns. Upon activation, a structured prompt is dynamically constructed. This prompt includes: (1) explicitly defined classification rules grounded in the FACS; (2) the current AU list detected in the frame; (3) contextual information such as subject-specific emotion trends; and (4) targeted disambiguation guidelines to address known confusions between similar emotions (e.g., fear vs. surprise). The GPT model processes this input using its transformer-based architecture and generates a classification decision. The LLM's output undergoes a post-processing validation stage that verifies the consistency with predefined logical constraints. If inconsistencies are detected, correction rules are applied; otherwise, the system either accepts the result or defaults back to deterministic rule-based classification. Additionally, the system includes a continuous learning mechanism that updates its AU-emotion association knowledge base incrementally, allowing for improved adaptation to individual expression patterns over time. The final classification output is further used by an assessment module that computes the system performance metrics and generates interpretive reports for research analysis. To address ambiguous cases where rule-based classification is inconclusive, we integrated a LLM--specifically, OpenAI's GPT-3.5-turbo--into our micro-expression recognition pipeline. This module serves as a fallback mechanism when partial or overlapping AU activations prevent a confident decision (e.g., AU1, AU2, and AU5 without AU4, which introduces ambiguity between fear and surprise).