# A Semantic Segmentation Dataset and Real-Time Localization Model for Anti-UAV Applications
## 5 Experiments
### Model Performance Comparison
#### 5.3.1 Quantitative Results  
Table 3 presents the performance of each model trained on the proposed UAV semantic segmentation dataset. All models were able to produce meaningful mIoU values, confirming that they successfully learned to generate semantic segmentation masks. This result validates the suitability of the dataset for training models on the UAV semantic segmentation task. For comparison, we use four metrics of precision, recall, Dice coefficient, and mIoU. The Dice coefficient and mIoU are metrics to measure the quality of the generated segmentation mask. To calculate those metrics, the metrics of precision and recall are required. Therefore, the final value of the Dice coefficient and mIoU is linear with the value of precision and recall. Among all models, the PAN model achieved the highest validation mIoU, indicating superior accuracy in UAV mask prediction. The UNet model ranked second, with a validation mIoU only 0.001 (or 0.1%) lower than PAN, showing comparable effectiveness. The proposed ThinDyUNet model achieved a validation mIoU of 0.647, placing it third overall. Although it did not surpass the top-performing models, the performance gap is relatively small, just 0.026 (or 2.6%) lower than PAN. The lowest performance was observed in the PSPNet model, which produced a validation mIoU of only 0.64. Overall, the spread of mIoU performance between the models is quite small, indicating that during the training, all models are learning properly. Dice coefficient parameters are also considered as metrics to compare the model performance. The Dice coefficient measures the shape similarities of the generated object mask with the ground truth mask. Based on Table 3, the results are not much different than the mIoU metrics, where all models are able to perform a decent value on Dice coefficient metrics. However, the PSPNet consistently has the lowest performance compared to other models. In this metric, the UNet, PAN, and TransUNet perform better than the ThinDyUNet model, where the proposed model achieves only 0.725. However, the highest Dice coefficient is 0.757, which shows only a 0.032 or 3.2% spread difference compared to the ThinDyUNet, which explains the slight performance difference between the aforementioned models. Despite not leveraging any pretrained weights, the ThinDyUNet model demonstrated competitive performance, closely approaching that of models initialized with pretrained weights and a much larger number of parameters. This effectiveness is largely due to the use of the dynamic convolution mechanism, which allows the model to adaptively extract features and the configuration of the model's layer for more efficient learning. In Table 4, the model performance in the test set is compared. Compared with results in the validation set, several models undergo a drastic reduction in performance for all metrics. Mainly, the PAN model, which is the first-rank performance and validation set, become the worst performance in the test set. Meanwhile, the TransUNet model, which got moderate performance in the validation set, also endured the same situation as the PAN model. The situation where the model achieves decent metrics in the validation set but decreases significantly in the test set is usually caused by overfitting during the model training. Meanwhile, the UNet and PSPNet models achieve a higher metrics performance on the test set compared to the validation set. The ThinDyUNet model achieves a stable performance in both the validation and test sets. It proved that the model can converge well during the model training. The ThinDyUNet achieved the value of 0.744 for Dice coefficient metrics and 0.646 for mIoU metrics in the test set. These metric values showed small differences when compared to the best models that achieved the best metric values, indicating that the proposed model exhibits similar performance to the best models. Moreover, the ThinDyUNet model offers a significant advantage in terms of model size.