# A Hyper-Attentive Multimodal Transformer for Real-Time and Robust Facial Expression Recognition
## 4 Robustness Strategies  
While FER systems have achieved remarkable progress under controlled conditions, real-world deployment scenarios remain fraught with challenges that can severely degrade model performance. These include partial occlusions, variable head poses, illumination fluctuations, facial artifacts, and inter-subject variability. To ensure consistent accuracy and reliability, FERONet is equipped with a suite of robustness mechanisms that explicitly target these real-world complexities at both the architectural and training levels. This section details three primary strategies: occlusion resilience, pose normalization, and illumination and appearance invariance. Occlusions can disrupt critical facial regions responsible for conveying emotion, potentially leading to incorrect predictions. Rather than treating occlusions as noise, FERONet adopts a dual-pronged strategy to detect, adapt to, and compensate for occluded regions during both training and inference. Within the multimodal encoder, we introduce an auxiliary occlusion prediction branch, which estimates a binary occlusion mask \(M_{o}\in R^{H\times W}\). This map is derived from a shallow convolutional subnetwork trained using a binary cross-entropy loss with pseudo-ground-truth masks (generated via synthetic occlusion or semantic segmentation priors). The mask is used to reweight the encoded features:  
\[F_{ocel}=F\odot(1-M_{o}) \tag{28}\]  
This reweighting scheme suppresses activations from occluded regions, preventing the attention modules and transformer from overfitting to noisy or irrelevant inputs. To encourage the model to infer missing information, we introduce a self-reconstruction pathway that learns to regenerate occluded feature regions using an encoder-decoder structure. The reconstruction loss: \(L_{rec}=\left\|\hat{F}_{recon}-F_{clean}\right\|_{2}^{2}\) is minimized alongside the classification loss, where \(F_{clean}\) refers to non-occluded features and \(\hat{F}_{recon}\) is the reconstructed tensor. This auxiliary task enhances the model imputation capability, allowing it to maintain high accuracy even when key facial parts are partially obscured. Head pose variation is another significant source of ambiguity in FER, particularly in in-the-wild datasets where subjects may not face the camera directly. Expressions can appear distorted or subdued under profile views, and spatial misalignment can confuse the attention mechanisms. To mitigate the challenges posed by head pose variability--a common issue in unconstrained facial expression recognition--FERONet integrates a lightweight pose correction module that is trained jointly with the main FER pipeline. This auxiliary module is designed to normalize facial orientation across samples, thereby reducing spatial discrepancies and enhancing the model's ability to generalize across individuals and viewing angles. The correction process begins with 3D landmark estimation, wherein facial keypoints are extracted using a pretrained 3D face alignment network (FAN). These landmarks are represented in a three-dimensional coordinate space \(R^{3\times K}\) where \(K\) denotesthe number of anatomical facial points. Building upon this geometric representation, the model then applies Procrustes alignment to estimate a minimal transformation matrix \(T\), comprising rotation and translation parameters that optimally map the current pose to a canonical frontal view. Once computed, this transformation is applied to both the raw input image and its intermediate feature maps through pose-aligned warping, thereby generating a spatially standardized representation that is subsequently passed into the encoder. This process of pose normalization significantly reduces spatial variance and enhances inter-subject consistency--two factors that are especially critical when distinguishing among expressions that may appear geometrically similar but differ contextually or emotionally.