## 3 Proposed Methodology
### Factors Affecting the Performance of the Proposed Fers
The whole training images were flipped horizontally by applying simple image data augmentation techniques. In this work, we applied these techniques for each resolution of the images. * _Fine Tuning:_- Fine-tuning allows for higher-order feature representations in the base model to make them more relevant for the face recognition tasks. For example, VGG used many layers and generated a higher dimensional feature vector, and thus thw inference was quite costly at run-time due to huge parameters. In this case, fine-tuning techniques were applied when freezing some layers and the number of parameters, and the model was retrained to reduce computational overheads. * _Progressive Resizing:_- Progressive image resizing is an eminent technique that sequentially resizes all images while training the CNN models on smaller, i.e., tinier images to larger image sizes. The progressive resizing technique is used to train a CNN with \(n\times n\) image size, saving the weights, and then the CNN is retrained again for other iterations with the images of increased sizes greater than \(n\). This technique was used for super-resolution [64], where low-resolution images gradually increased to the image with a higher resolution during training processes. The advantages of using progressive resizing are that it improves generalization and reduces overfitting problems. * _Transfer Learning:_- The principle concept behind transfer learning for facial expression recognition and classification problems is that a model trained on large data sets for one problem is effectively used as a generic model in some way on other related problems. The model that has been trained earlier is known as the pre-trained model. Our proposed deep learning convolution neural network model uses a transfer learning technique in which the weights of the pre-trained model and/or a set of layers from the pre-trained model \(CNN_{1}\) are used for the new model \(CNN_{2}\) to solve similar problems. Similarly, the weights of \(CNN_{2}\) have been adopted to solve the \(CNN_{3}\) model. The benefits of using transfer learning are that it reduces the training time and can result in lower generalization errors. * _Scores Fusion:_- In the proposed system, three CNN architectures have been proposed. These architectures take images of different sizes as inputs. Thus, during the recognition of facial expressions on the test sample \(F\), there are three different classification score vectors, namely \(s_{1}=(a_{1}^{1},a_{2}^{1},\cdots,a_{7}^{1})\), \(s_{2}=(a_{1}^{2},a_{2}^{2},\cdots,a_{7}^{2})\), and \(s_{3}=(a_{1}^{3},a_{2}^{3},\cdots,a_{7}^{3})\), where each \(a_{j}^{i}\) is the classification score by the \(CNN_{i}\) architecture and for \(j^{th}\) expression class. These classification scores are fused together using score-level post-classification fusion approaches [14] to increase the performance of the recognition system. In this work, two score-level fusion techniques, namely _Sumrule_ and _Product-rule_, were employed. The _Sum-rule_ and _Product-rule_ techniques are defined as follows: \[\underset{i\
eq j,\;k=\{1,\cdots,7\}}{\max}\{a_{k}^{i}+a_{k}^{j}\}\] (1) \[\underset{i\
eq j,\;k=\{1,\cdots,7\}}{\max}\{a_{k}^{i}*a_{k}^{j}\}\] (2)