# Robust Airport Surface Object Detection Based on Graph Neural Network
## 3 Results
### Result and Analysis
* In our experiments, we have explored several neighbor definition approaches with different setups, including k-nearest-neighbor in pixel space, k-nearest-neighbor in representation space, and k-farthest-neighbor in representation space. The worst definition is k-farthest-neighbor in representation space, whose accuracy always stays at 0 during training. Obviously, gathering information from irrelevant parts does not help to improve model performance. Though the original ViG is amazing, defining neighbors in pixel space surpasses that in representation space verified by our experiments. As for parameter setups, i.e., radius k and dilation number, they need to be adjusted carefully to define a proper receptive field. Due to space limitations, we do not show this part's experiment. Result of robustness under adversarial attack. When applying detection models for airport surveillance, it is necessary to improve their robustness against potential attacks. Considering the importance of ensuring perturbations remain imperceptible within images, we focus on the gradient-based artificial perturbation methods. These methods are well-studied these years and constitute the primary challenge that adversarial training approaches aim to mitigate [28; 29]. For each salient sample, we attack the model's location loss for five iterations to generate a corresponding adversarial sample based on the PGD algorithm [26]. Then, adversarial samples are utilized to test the robustness of models. The result is in Table 3 as follows. According to the experiment above, we come to the conclusion that:  
* On the LEVIR and airport surface dataset, the adversarial attack does cause a significant loss in object detection accuracy for all models above. Especially for detectors with GNN block, they are more vulnerable than purely convolution-based detectors (i.e., YOLOv5s and YOLOv5m) against attack. In detail, detectors with GNN block obtain about 38% and 67% mAP.5 decreases while vanilla YOLOv5s models obtain about 47% and 60% on two datasets, respectively. When using a GNN-based detector in safety supervision, its poor robustness must be taken into consideration. * Adversarial training does help to improve model robustness against attack. Although models through adversarial training still suffer from accuracy loss towards adversarial attacks, this phenomenon has been alleviated obviously compared with those models without adversarial training. \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline \multicolumn{1}{c}{**Models**} & \multicolumn{3}{c}{**LEVIR**} & \multicolumn{3}{c}{**Airports**} \\ \hline \multicolumn{1}{c}{} & \multicolumn{2}{c}{**Without Attack**} & \multicolumn{2}{c}{**Under Attack**} & \multicolumn{2}{c}{**Without Attack**} & \multicolumn{2}{c}{**Under Attack**} \\ \hline \hline  & **mAP.5** & **mAP.5.95** & **mAP.5** & **mAP.5.95** & **mAP.5** & **mAP.5:95** & **mAP.5** & **mAP.5:95** \\ \hline YOLOv5s & 0.937 & 0.684 & 0.492 & 0.211 & 0.786 & 0.377 & 0.315 & 0.076 \\ YOLOv5m & 0.957 & 0.725 & 0.445 & 0.197 & 0.827 & 0.396 & 0.314 & 0.073 \\ YOLOv5-ViG & 0.926 & 0.681 & 0.517 & 0.200 & 0.825 & 0.399 & 0.273 & 0.065 \\ YOLOv5s-ViGE & **0.958** & **0.737** & 0.486 & 0.171 & 0.832 & 0.383 & 0.203 & 0.052 \\ YOLOv5s-A & 0.943 & 0.718 & 0.760 & **0.406** & 0.833 & 0.384 & 0.457 & 0.125 \\ YOLOv5s-ViGE-A & 0.945 & 0.717 & **0.773** & **0.406** & **0.836** & **0.431** & **0.508** & **0.155** \\ \hline \hline \end{tabular} Note: The best performance is marked in bold text.  
\end{table} 
Table 3: Model robustness under adversarial attack. * Adversarial training also helps to improve detection accuracy. This framework takes the use of adversarial samples as auxiliary samples to train the model, which can be seen as a data augmentation approach. It is known that data augmentation is able to improve the model's performance, especially for small-scale datasets.