## 5 Experiments and Results
### Fault Detection and Classification
#### 5.2.5 Results Analysis  
This subsection delves into the detailed results obtained from the application of various deep learning models on our dataset, with a focus on their performance across multiple metrics. Table 3 shows the performance metrics of the implemented models, highlighting their accuracy, precision, recall, and F1-score. The F1-score is the harmonic mean of precision and recall, providing a single metric that balances both the false positives and false negatives. It is particularly useful in situations where an uneven class distribution might render metrics like accuracy unuseful. The F1-score ranges from 0 to 1, with a higher value indicating better model performance and a more balanced trade-off between precision and recall. Table 3 orders the models by their performance, demonstrating the comparative effectiveness of each architecture in the context of image classification of _healthy_ and _faulty_ for PFBs. Particularly notable are the performances of the ViT and DenseNet models, which stand out with high accuracy and F1-scores, exemplifying their precision and effectiveness in this task. The F1-score is calculated as follows:  
\[\text{F1-score}=2\times\frac{\text{precision}\times\text{recall}}{\text{ precision}+\text{recall}} \tag{12}\]  
Our extensive analysis, documented through 10 random training runs, culminated in a comparative evaluation of several deep learning models, as illustrated in Table 3. The top performance metrics--average accuracy, precision, recall, and F1-score--highlight the capabilities and effectiveness of these models in detecting anomalies in PFBs. Remarkably, both ViT and DenseNet achieved unparalleled performance across all metrics, securing a perfect score of 100% in accuracy, precision, recall, and F1-score on \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Model Name** & **Avg. Accuracy** & **Avg. Precision** & **Avg. Recall** & **Avg. F1-Score** \\ \hline
**ViT** & **100** & **100** & **100** & **100** \\ \hline
**DenseNet** & **100** & **100** & **100** & **100** \\ \hline Xception & 99.79 & 99.33 & 100 & 99.66 \\ \hline ResNet & 99.58 & 100 & 98.51 & 99.25 \\ \hline VGG & 99.37 & 98.66 & 99.32 & 98.99 \\ \hline AlexNet & 99.37 & 100 & 97.97 & 98.98 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Top performance metrics of implemented models after 10 random training runs (in %). our dataset. This exceptional outcome underscores the advanced capabilities of these models in flawlessly identifying both healthy and faulty PFBs, indicating a potentially groundbreaking advancement in the field of anomaly detection within PFBs. Following closely, Xception demonstrated almost impeccable performance with an average accuracy of 99.79%, precision of 99.33%, and a perfect recall of 100%, culminating in an F1-score of 99.66%. This indicates a strong ability to correctly identify all instances of faulty PFBs, albeit with a marginally lower precision compared to the leading models. ResNet and VGG, while still exhibiting high performance, presented slightly more variation in their metrics. ResNet's perfect precision score of 100% contrasted with a lower recall of 98.51%, indicating a slight tendency to miss a minimal number of faulty PFB instances. VGG, with an accuracy of 99.37%, precision of 98.66%, and recall of 99.32%, demonstrated balanced performance, but with room for improvement in precision. AlexNet, sharing the same accuracy with VGG, showcased a perfect precision score, but a reduced recall of 97.97%, suggesting it occasionally overlooked faulty PFB instances more than other models. The misclassified test images from these architectures are illustrated in Figure 21 for Xception, Figure 22 for ResNet, Figure 23 for VGG, and Figure 24 for AlexNet. According to Figures 21-24, Xception, ResNet, VGG, and AlexNet misclassified 1, 2, 3, and 3 images out of 420 test images, respectively. These findings reveal the significant impact of model selection on the effectiveness of anomaly detection in PFBs.