## 3 Materials and Methods
### The Evaluation  
CNNs Model's performance is often measured with accuracy, precision, recall, and an F1-Score which are calculated from the confusion matrix (CM) [32]. Accuracy is the ratio of the number of samples correctly classified by the classifier to the total number of samples in the testing data as in Equation (6).  
\[\text{Accuracy}=\frac{\text{TP}+\text{TN}}{(\text{TP}+\text{TN}+\text{FP}+ \text{FN})} \tag{6}\]  
In the equations, TP, TN, FP, and FN refer to true positive (observation is predicted positive and is actually positive), true negative (observation is predicted negative and is  
Figure 8: SAFEPA flow chart.  
Figure 7: The structure of the proposed model SAFEPA.  
actually negative), false positive (observation is predicted positive and is actually negative), and false negative (observation is predicted negative and is actually positive), respectively. Where accuracy shows the gap between the real values and the predicted values, precision deals with the fraction of positive predictions as in Equation (7) and recall deals with the actual positive fraction as in Equation (8).  
\[\text{Precision}=\frac{\text{TP}}{\text{TP}+\text{FP}} \tag{7}\]  
\[\text{Recall}=\frac{\text{TP}}{\text{TP}+\text{FN}} \tag{8}\]  
The F1-Score combines both precision and recall in a single value as in Equation (9). The F1-Score is indicative of a model's balanced ability to describe both positive cases (recall) as well as be accurate with the cases that it captures (precision).  
\[\text{F1 Score}=\frac{\text{2 TP}}{\text{2 TP}+\text{FP}+\text{FN}} \tag{9}\]  
The performance evaluation of the SAFEPA system was conducted using various datasets, as follows:  
1. UNBC-McMaster Dataset  
A 10-fold cross-validation accuracy was measured on a subset of the UNBC-McMaster dataset, consisting of 6000 samples. The obtained results were compared with recently published models designed for pain assessment, as described in references [3; 22; 23]. Subsequently, the confusion matrix and the Receiver Operating Characteristic (ROC) curve were generated to evaluate the performance of SAFEPA's best performing model, achieved through 10-fold cross-validation.  
2. KDEF Dataset--Frontal View Subset  
A 10-fold cross-validation accuracy was measured on a subset of the KDEF dataset, focusing specifically on the frontal view. This subset comprised a total of 980 samples. The obtained results were compared with four recently published models for Facial Expression Recognition (FER) mentioned in references [17; 24; 25; 26].  
3. KDEF Dataset--Full and Half Left/Right Poses  
The evaluation was performed on a subset of the KDEF dataset, which included full and half left/right poses. This subset contained a total of 3920 samples. The results obtained were compared with the outcomes of models mentioned in reference [17].  
4. KDEF Dataset--Entire Dataset  
The evaluation was further performed on the entire KDEF dataset, consisting of 4900 samples. The results obtained were compared with the outcomes of models mentioned in reference [17].  
5. Analysis Metrics on KDEF Subset (Frontal View)  
The evaluation involved analyzing a confusion matrix, precision, recall, F1-Score, and accuracy. This analysis was conducted using a training dataset of 70% and a testing dataset of 30% on the KDEF subset with 980 samples.  
6. Evaluation on BioVid Dataset  
To provide a comprehensive evaluation of the SAFEPA system's performance, accuracy and processing time were measured. The system was tested on the entire unseen BioVid dataset. Processing time was considered a critical factor, especially in real-world pain assessment scenarios where efficient and prompt processing is vital for timely pain management.  
These evaluations and measurements collectively provide a thorough assessment of the SAFEPA system's performance across different datasets and scenarios.