## 5 Results  
Table 3 shows the 10-fold cross validation results of the 16 models (precision, recall, and F1-score). Table 4 shows the results of the phase that tested the two classifier platforms in terms of the 10-fold cross validation accuracy and data of unseen subject for 16 assortments made up of four models and four optimizers trained on the UNBC-McMaster shoulder pain expression archive dataset. All 16 models showed high accuracy and achieved more than 96.00% for 10-fold cross validation, except for ResNeXt50 with SGD. For an unseen subject, InceptionV3 with SGD and ResNeXt50 with SGD achieved 90.56% and 90.19, respectively, whereas ResNet50 with ADAM and InceptionV3 with RAdam achieved 88.21% and 86.10%,  
Figure 8: FEAPAS flow chart. respectively. The rest of the models achieved less than 85.00%. As shown in Figure 9, the SGD optimizer was more stable and less fluctuating in InceptionV3 training comparing with other optimizers (i.e., ADAM, RMSprop, and RAdam). Analyzing the results of the trained models helped us decide which deep-learning framework worked better with our FEAPAS system. Based on the combination of the two accuracy values, InceptionV3 with SGD was selected as the models to be inserted in the FEAPAS system. We therefore conducted more experiments on the FEAPAS system to evaluate the overall performance. Table 5 shows that our proposed approach outperformed previous approaches that were also conducted on the same UNBC-McMaster shoulder pain expression archive dataset and evaluated based on k-fold cross validation. The accuracy of the unseen subject was not provided by previous publications, so we could not compare. Table 5 also shows whether the study used entire images in the dataset, a subset, or a variation of subset plus data collected from other sources. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**The Model**} & \multirow{2}{*}{\begin{tabular}{c} **The** \\ **Optimizer** \\ \end{tabular} } & \multicolumn{2}{c}{\begin{tabular}{c} **10-Fold Cross** \\ **Validation** \\ **Accuracy (\%)** \\ \end{tabular} } & \multicolumn{2}{c}{\
\begin{tabular}{c} **Accuracy of Unseen** \\ **Person (\%)** \\ \end{tabular} } & \multicolumn{2}{c}{
\begin{tabular}{c} **Average** \\ **Accuracy (\%)** \\ \end{tabular} } \\ \hline VGG16 & SGD & 97.41 & 82.06 & 89.74 \\ VGG16 & ADAM & 99.12 & 78.15 & 88.64 \\ VGG16 & RMSprop & 97.90 & 66.05 & 81.98 \\ VGG16 & RAdam & 99.30 & 65.80 & 82.55 \\ InceptionV3 & SGD & 99.10 & 90.56 & **94.83** \\ InceptionV3 & ADAM & 97.45 & 84.48 & 90.97 \\ InceptionV3 & RMSprop & 98.27 & 82.37 & 90.32 \\ InceptionV3 & RAdam & 96.93 & 86.10 & 91.52 \\ ResNet50 & SGD & 96.58 & 76.10 & 86.34 \\ ResNet50 & ADAM & 99.23 & 88.21 & 93.72 \\ ResNet50 & RMSprop & 99.28 & 83.86 & 91.57 \\ ResNet50 & Adam & 99.27 & 83.61 & 91.44 \\ ResNeXt50 & SGD & 89.85 & 90.19 & 90.02 \\ ResNeXt50 & ADAM & 97.98 & 74.98 & 86.48 \\ ResNeXt50 & RMSprop & 98.18 & 71.14 & 84.96 \\ ResNeXt50 & RAdam & 98.40 & 80.76 & 89.58 \\ \hline \hline \end{tabular} Bold number shows the best result.  
\end{table} 
Table 4: Results of VGG16, InceptionV3, ResNet50, and ResNeXt50 with four optimizers. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**The Model** & **The Optimizer** & **Precision** & **Recall** & **F1-Score** \\ \hline VGG16 & SGD & 98.42 & 98.58 & 98.47 \\  & ADAM & 99.84 & 99.81 & 99.83 \\  & RMSprop & 99.12 & 99.17 & 99.14 \\  & RAdam & 99.30 & 99.84 & 99.81 \\ InceptionV3 & SGD & 99.84 & 99.81 & 99.83 \\  & ADAM & 97.33 & 96.94 & 97.05 \\  & RMSprop & 99.94 & 99.93 & 99.96 \\  & RAdam & 99.51 & 99.48 & 99.49 \\ ResNet50 & SGD & 99.84 & 99.81 & 99.83 \\  & RMSprop & 99.84 & 99.81 & 99.83 \\  & RAdam & 99.84 & 99.81 & 99.83 \\ ResNeXt50 & SGD & 96.99 & 96.89 & 96.90 \\  & ADAM & 99.68 & 99.65 & 99.67 \\  & RMSprop & 99.84 & 99.81 & 99.83 \\  & RAdam & 99.69 & 99.63 & 99.66 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: The 10-fold cross validation results of VGG16, InceptionV3, ResNet50, and ResNeXt50 with four optimizers.