# Real-Time Facial Emotion Recognition Framework for Employees of Organizations Using Raspberry-Pi
## 3 Proposed Methodology  
This section discerns the proposed framework. The elucidation of each step is elaborated on in the next sections. The entire process is divided into three tightly coupled tasks. The first task is to train the pertained deep network after dividing the dataset into training, validation, and testing. The entire dataset of emotive facial images is divided into an 8:1:1 ratio. A dataset with \(N\) images will be divided into \(\sigma_{\text{TR}}\) for training, \(\sigma_{\text{VD}}\) for cross validation and \(\sigma_{\text{T}}\) for testing purposes. This means a training set of \(N\) number of images \(I\) will consist of \(\sigma_{\text{TR}}=\left\{I_{1}^{TR},I_{2}^{TR},I_{3}^{TR}\dots I_{4N/5}^{TR}\right\}\) as training images, \(\sigma_{VD}=\left\{I_{(4N/5)+1}^{VD}\dots I_{9N/10}^{VD}\right\}\) as validation images and \(\sigma_{\text{T}}=\left\{I_{(9N/10)+1}^{T}\dots I_{N}^{T}\right\}\) as testing images. A deep convolutional neural network known as Mini-Xception is used for training, validation, and testing of emotive facial images. Training, validation, and testing is done on Google-CoLab with 12 GB NVIDIA Tesla K80 GPU using FER 2013 dataset. The entire architecture is divided into two tightly coupled tasks, i.e., face recognition and facial emotion recognition in real time. For face recognition, a pre-trained deep network known as OpenFace is used. To begin with, a real time image from video is captured as \(I_{1}^{r}\). The total number of images captured in real time is \(\delta_{R}=\left\{I_{1}^{r},I_{2}^{r}\ldots I_{N}^{r}\right\}\). To train the deep network, 6 images of each subject have been used and the network is trained with single triplet method of training for 20 different people with \(N\) images and denoted as \(\delta_{TR}\). Once the training is complete, for the real time captured image, \(I_{1}^{r}\), the very first task is to find the face inside the captured image and discard the unwanted information. The description of parameters used in the proposed architecture is given in Table 1.  
A well-known method known as HOG (Histogram of Oriented Gradients) has been used to find the faces. After the detection of the face, the facial image \(I_{1}^{r}\) has been cropped, which is further preprocessed in order to remove the effects of bad lighting, tilted face, and skewness, etc., in the cropped image \(I_{cr}^{r}\). The cropped image is preprocessed with the face landmark estimation algorithm. This algorithm locates 68 landmarks on the cropped image \(I_{cr}^{r}\) and, with the help of simple affine transformations, the image is preprocessed \(I_{pr}^{r}\) using rotation, shear, and scale to center the eyes and mouth of the cropped image \(I_{cr}^{r}\) at best. The preprocessed image \(I_{pr}^{r}\) is fed to the pre-trained network to extract the features from \(I_{pr}^{r}\) image and generate 128 embeddings that are measurements of the face. The feature of the preprocessed image \(I_{pr}^{r}\) is generated with the help of the neural network that will generate a feature vector of 128 embeddings as \(\Phi_{n}^{embd}\). The last step is to classify the image by measuring the closest match of 128 embeddings \(\Phi_{n}^{embd}\) by comparing it with the database images. The feature vector \(\Phi_{n}^{embd}\) is passed through the simple SVM classifier \(\tilde{\mathbb{S}}\), to recognize the face. The entire architecture with its detailed framework is represented in Figure 1.