## 4 Results
### Performance Analysis of Semantic Segmentation  
Comparing the anticipated masks to the given ground truth masks is crucial to determine how accurate picture segmentation is. The validation dataset is essential for evaluating and validating the model's performance during the training process. We use the validation dataset to capture important measures, such as pixel-based accuracy and the mean IoU score, which are often employed in semantic segmentation tasks. Additionally, we carry out a rigorous assessment by contrasting the given ground truth mask with the anticipated binarized aircraft mask produced by the segmentation model. The model's accuracy and segmentation effectiveness are continually checked and improved throughout training thanks to this thorough validation approach.  
Figure 4 illustrates the results of testing the proposed method, presenting the best segmentation results achieved by the model using three test images. In this representation, the first column displays three original images, the second column shows the corresponding ground truth images, and the third column reveals the outcomes of the proposed aircraft semantic segmentation method. As observed in Figure 4, the results of semantic aircraft segmentation closely resemble the provided ground truth images, with only minor variations in some small regions. The overall pixel-wise accuracy, representing the percentage of correctly predicted pixels, impressively reaches 98.24%. Additionally, the mean IoU score achieves a high value of 95.08%. These metrics emphasize the precision and effectiveness of the proposed segmentation model in accurately identifying aircraft in the images.  
By conducting a thorough evaluation of the proposed method, we calculated the confusion matrix specifically for the two classes considered: aircraft and image background. Figure 5 displays the resulting confusion matrix for a single test image. We used the Scikit-learn package to obtain the TP, FP, FN, and TN components of the confusion matrix [44]. These confusion matrix components were assigned various colors for visualization purposes, allowing us to see how they are distributed throughout the picture. Since this makes it possible for us to tell the difference between TP and TN, TP, FP, FN, and TN are all mapped to the cyan, magenta, yellow, and black color spaces in Figure 5. We used the OpenCV library and an algorithm known as alpha compositing to transparently overlay the color mask onto the original grayscale image. Figure 5 illustrates the confusion matrix and its overlay on one of the test photos, providing details on how well the model performs in various areas of the image.  
Figure 4: Results of experiments. (**a**) Original Images; (**b**) Ground Truth Images; (**c**) Predicted Images.  
The mean IoU curves for the U-Net model during training and testing across several epochs are shown in Figure 6. This visual depiction provides information about the model's functionality and development during the training and testing phases. Additionally, it is clear from the charts that the mean IoU first grows significantly as the number of epochs rises. The rate of improvement, however, declines as the training goes on, and the number of epochs exceeds 10, suggesting that the model's learning begins to plateau. Additionally, the graph shows that at epoch 20, which is the final epoch considered, the model achieved its highest mean IoU value. This information highlights the training process's effectiveness and indicates that, beyond a certain point, further training may have diminishing returns in terms of the mean IoU improvement. It provides valuable insights into the optimal training duration and when to conclude the training process for this specific model.  
Figure 5: Confusion matrix overlay mask (TP—cyan = 1014; TN—black = 64,274; FP—magenta = 124; and FN—yellow = 124). (**a**) Confusion matrix elements; (**b**) Predicted Image; (**c**) Overlaid Image; (**d**) Ground Truth.  
Figure 6: Graph of mean IoU versus epoch during validation and training stages.