## 3 Materials and Methods
### Feature Extraction  
#### 3.2.1 Handcrafted Feature Extraction Methods  
The features are identified with the feature detection algorithms, and their analysis with vector values is performed with feature descriptor algorithms. Due to their widespread use in facial recognition applications [34; 35; 36], we investigate the LBP and HOG feature descriptor algorithms.  
(A) Local Binary Patterns  
LBP encodes the texture information of a grayscale image by comparing the difference in the intensity of each pixel with its neighboring pixels. Initially, the image is converted into grayscale. Then the image is divided into rectangular cells [k \(\times\) k]. Each pixel \(i\) in the cell is compared (in terms of intensity) to the neighboring pixels in a circle centered on the i-pixel and radius \(r\). In [15], the neighboring pixels are 8, and the radius is 1. By setting the value of the central pixel as the threshold (ranging from 0 to 255), the adjacent pixels get binary values: those with equal or greater values get a value of 1, and those with lower values get a value of 0. These binary values are converted to decimal by multiplying them with powers of 2 (keeping the same direction) and summing them up. The process is repeated with each pixel belonging to 9 different 3 \(\times\) 3 cells so it can take 28 different values. The 256-bin histogram of the frequency of values taken by each pixel constitutes the 256-dimensional feature vector.  
The fact that some of the binary patterns appear more often than others led to an advanced rotation-invariant version of this algorithm named the uniform pattern version [37]. A pattern is uniform when it has at most two transitions 0\(\rightarrow\)1 or 1\(\rightarrow\)0. The histogram, in this case, has one bin for every uniform pattern and one bin for all the non-uniform  
Figure 1: Sample images from the three databases represent the emotion of happiness. Databases KDEF ([[https://www.kdef.se/](https://www.kdef.se/)]([https://www.kdef.se/](https://www.kdef.se/)), accessed on 16 February 2022) and RaFD ([[https://rafd.socsci.ru.nl/RaFD2/RaFD?p=main](https://rafd.socsci.ru.nl/RaFD2/RaFD?p=main)]([https://rafd.socsci.ru.nl/RaFD2/RaFD?p=main](https://rafd.socsci.ru.nl/RaFD2/RaFD?p=main)), accessed on 25 February 2022) have five poses, while JAFFE ([[https://zenodo.org/record/3451524#.YvCcehBxPY](https://zenodo.org/record/3451524#.YvCcehBxPY)]([https://zenodo.org/record/3451524#.YvCcehBxPY](https://zenodo.org/record/3451524#.YvCcehBxPY)), accessed on 16 February 2022) has only one (frontal) pose.  
patterns. The bins are equal to \(P(P-1)+3\), where \(P\) is the number of the neighboring pixels. For 8 neighboring pixels, the \(2^{8}=256\)-dimension histogram is transformed to a 59-dimensional histogram leading to a 77% reduction in feature size. For an \([M\times N]\) image, the feature size is given by (1):  
\[Feature\ Size_{LBP}=\left[floor\left(\frac{M}{k}\right)\times floor\left(\frac{ N}{k}\right)\right]\times\left[P(P-1)+3\right] \tag{1}\]  
HIGogram of Oriented Gradients  
HOG technique describes the edges and corners of an object through the distribution of local intensity gradients. For an \([M\times N]\) image, the gradients of each pixel in the polar form are calculated. These values create the corresponding magnitude and angle matrices with the same dimensions. These matrices are divided into rectangular cells \([k\times k]\). For all \(k^{2}\) values, a 9-point histogram is calculated, with each point having a width of 20 degrees, ranging from 0 to 160 degrees. The positions in the histogram are selected based on the angle of the gradient, and the values in each bin are derived from the percentage of the corresponding magnitude. These 9-point histograms are grouped into blocks of four \((2\times 2)\), creating a feature vector 36. The grouping is performed with overlapping of k pixels. So, the size of the feature is given by (2):  
\[FeatureSize_{HOG}=\left[floor\left(\frac{M}{k}-1\right)\times floor\left( \frac{N}{k}-1\right)\right]\times 36 \tag{2}\]