## 7 Experimental Setup and Results
### Dataset
The results show that Subspace Discriminant, Linear SVM, and Quadratic SVM are the best methods for classification in FC6, FC7, and FC8, respectively, as shown in Table 4.  
Furthermore, the best accuracies of classifiers on each layer of the AlexNet model are mentioned in the confusion matrix (see Figure 5). \begin{table}
\begin{tabular}{c c c} \hline
**AlexNet** & **Classifier** & **Accuracy** \\ \hline FC6 & Subspace Discriminant & 62.7\% \\ FC7 & Linear SVM & 62.0\% \\ FC8 & Quadratic SVM & 63.4\% \\ \hline \end{tabular}
\end{table} 
Table 4: Improved Accuracy of AlexNet pre-trained model on FC6, FC7, and FC8 with ML classifiers on the MLF-W-FER dataset. \begin{table}
\begin{tabular}{c c c c c} \hline
**Classifier** & **Classifier Type** & **FC6 Accuracy** & **FC7 Accuracy** & **FC8 Accuracy** \\ \hline Decision Tree & Fine Tree & 55.5\% & 52.0\% & 53.8\% \\  & Medium Tree & 55.8\% & 54.4\% & 56.4\% \\  & Coarse Tree & 56.2\% & 53.8\% & 56.8\% \\ \hline Discriminant Analysis & Linear Discriminant & 59.5\% & 48.4\% & 59.5\% \\ \hline Support Vector Machines (SVM) & Linear SVM & 61.9\% & 62.0\% & 62.2\% \\  & Quadratic SVM & 62.5\% & 61.7\% & 63.4\% \\  & Cubic SVM & 59.6\% & 59.8\% & 60.8\% \\  & Fine Gaussian SVM & 55.8\% & 49.8\% & 55.9\% \\  & Medium Gaussian SVM & 62.0\% & 61.6\% & 62.5\% \\  & Coarse Gaussian SVM & 59.5\% & 59.9\% & 59.85 \\ \hline K Nearest Neighbor (KNN) & Fine KNN & 50.3\% & 50.5\% & 50.9\% \\  & Medium KNN & 56.0\% & 54.5\% & 56.2\% \\  & Coarse KNN & 58.45\% & 57.0\% & 57.7\% \\  & Cosine KNN & 57.0\% & 55.5\% & 56.7\% \\  & Cubic KNN & 53.2\% & 54.4\% & 54.4\% \\  & Weighted KNN & 55.7\% & 55.4\% & 56.1\% \\ \hline Ensemble Classifiers & Boosted Trees & 59.0\% & 57.2\% & 58.5\% \\  & Bagged Trees & 56.8\% & 56.2\% & 57.0\% \\  & Subspace Discriminant & 62.7\% & 59.7\% & 62.6\% \\  & Subspace KNN & 50.6\% & 50.8\% & 50.7\% \\  & RUS Boosted Trees & 44.4\% & 46.7\% & 44.6\% \\ \hline \end{tabular}
\end{table} 
Table 2: Performance comparison of ML Algorithm Classification of MLF-W-FER Dataset on AlexNet. \begin{table}
\begin{tabular}{c c} \hline
**Alexnet** & **Accuracy** \\ \hline FC6 & 55.00\% \\ FC6 & 55.35\% \\ FC8 & 55.64\% \\ \hline \end{tabular}
\end{table} 
Table 3: Accuracy AlexNet pre-trained model on FC6, FC7, and FC8 on the masked dataset. In Figure 5, the confusion matrix of FC7 Linear SVM was not able to predict the negative class of the MLF-W-FER dataset. That is why we selected the Quadratic SVM with the second highest accuracy (61.7%) because it is predicting all three classes as positive, negative, and neutral. The Linear SVM has been replaced with Quadratic SVM in Figure 6.  
Second Experiment: Transfer Learning from VGG-16 for Extracting Features Using Different Classification  
In the second experiment, as depicted in Figure 7, the effectiveness of feature selection has been assessed using transfer learning from the VGG-16 network in conjunction with several classifications. According to the results in Table 4 for the FC6 features applied to the various classifiers, Quadratic SVM has a better classification accuracy of 65.5%, followed by Linear SVM with 65.3% and Cubic SVM with 64.4%. The best accuracy score has been achieved by Linear SVM with a score of 62.0%, followed by Quadratic SVM with a score of 61.7%, and Medium Gaussian SVM with a score of 61.6%, according to the results of FC7 features and performance comparison of classifiers. According to the results of the FC8 features and performance comparison of classifiers,  
Figure 5: Best classifiers for AlexNet on FC6, FC7, and FC8 in the Confusion Matrix. Figure 6: Best classifiers for AlexNet on FC6, FC7, and FC8 in the Confusion Matrix. Figure 7: Transfer learning from VGG-16 for extracting features and different classifiers techniques. In this experiment, the features obtained from VGG-16 were fed into the Fitececoc function.