# Emotional Health Detection in HAR: New Approach Using Ensemble SNN
## 3 Emotion Recognition Analysis
### FER Structure  
Face emotion recognition (FER) is a technique for the recognition of emotions through the analysis of facial expressions in multimodal form. FER has increased in the field of perceptual and cognitive sciences and affective computing with the development of artificial intelligence techniques, virtual reality [27], and augmented reality [28; 29]. Different inputs are available for the FER, such as electromyography (EMG), electrocardiograms (ECG), electroencephalograms (EEG), and the a camera; the latter is preferable because it provides more information and does not require the use of wearable devices. The technology on which the FER is based uses mathematical algorithms to analyze faces acquired from images or videos to recognize emotions or behaviors through facial features. Recognition systems can use 2D images as input data, but newer approaches employ 3D models or combined 2D-3D models called FER multimodal models [30]. Three-dimensional technology performs better, but due to the high resolution and frame rate, it requires more computational power as the amount of data captured in 3D databases increases. In addition to traditional approaches [31], deep learning-based algorithms can be applied for extraction, classification, and recognition activities. FER is divided into the following phases: image acquisition, image processing, face detection, feature extraction, and emotion classification (Figure 1). Image processing is a preliminary phase to eliminate all interfering factors in the input image that can affect classification performance and complicate processing. It consists of locating and extracting the region of the face. It is used to eliminate the background noise through processing filters and to normalize the image's color. For example, one of the most commonly used filters to obtain a sharper image is RIR (Regularized Inverse Auto-Regressive) [32]. Face detection involves distinguishing faces in an image or video and constructing bounding boxes for faces. One algorithm used for this purpose is the Viola-Jones algorithm, developed initially for object detection. This algorithm examines the minor features of a human face in an image, and if all these features are found, the algorithm predicts that there is a face in that image or a secondary image. Its application requires that requirements such as full-view, frontal, vertical, well-lit, and life-size faces in fixed-resolution images are met. [PERSON] and [PERSON] modified [PERSON]'s wavelets to develop so-called Haar-like features. A Haar-like feature considers adjacent rectangular regions in a sensing window, adds pixel intensities in each region and calculates the difference between these sums. This difference is used to categorize subsections of an image. Feature extraction is the process of extracting reference points that facilitate the algorithm to recognize the expression. Extraction methods can be different depending on the type of input image. For static images, the extraction method can be based on \"geometric features\" or \"aspects\". One commonly used geometric feature model is the active shape model (ASM) [33]. It consists of creating a suggested shape by looking at the image around each point for a better location for the point. Based on this aspect, local feature analysis (LFA) methods use the entire face or specific  
Figure 1: Flow chart of facial emotion recognition. measures to extract facial changes. The commonly used methods are local binary pattern (LBP) and Gabor feature extraction. LBP [34] is a texture operator defined as an ordered binary sequence of color depth comparisons between pixel p and pixels belonging to the neighborhood under consideration. To calculate the LPB code, for each generic pixel \"p\", the 8 \"x\" neighbors of the center pixel are compared with the pixel p and assigned a value of one if x \(\geq\) p. Calculating LBP on the entire image means producing a feature vector consisting of a histogram as an output result. Gabor feature extraction applies a series of filters to extract features.