# Robust Airport Surface Object Detection Based on Graph Neural Network
## 3 Results
### Result and Analysis  
Result on LEVIR. We train and evaluate several models on the LEVIR dataset; the results are in Table 1.  
We illustrate the detection result on an image in Figure 5. According to the experiment above, we can conclude that:  
* Compared with vanilla YOLOv5s, ViGE block brings a significant improvement of about 2% for mAP.5. This proves that introducing a new and proper receptive field for YOLOv5s' backbone is able to extract more effective features. * Compared with the original ViG block, the ViGE block performs better with about 3% mAP.5 increase. The ViG block defines neighbor in representation space with positional embedding, while the ViGE block defines neighbor in pixel space with a well-designed edge feature. The reason why ViGE outperforms the original ViG may come from two aspects. One is that defining neighbor in pixel space does better than that in representation space. This phenomenon confirms the previous success of CNN, which gathers information from neighbors in pixel space. The other is that our edge feature can better capture spatial relations than positional embedding. \begin{table}
\begin{tabular}{l l l l l} \hline \hline
**Models** & **P**\(\uparrow\) & **R**\(\uparrow\) & **mAP.5**\(\uparrow\) & **mAP.5:9**\(\uparrow\) \\ \hline YOLOv5s & 0.926 & 0.883 & 0.937 & 0.684 \\ YOLOv5m & 0.939 & 0.910 & 0.957 & 0.725 \\ YOLOv5s-ViG & 0.933 & **0.913** & 0.926 & 0.681 \\ YOLOX-s & – & – & **0.960** & 0.716 \\ YOLOv5s-ViGE (ours) & **0.945** & 0.912 & 0.958 & **0.737** \\ \hline \hline \end{tabular} Note: The \(\uparrow\) symbol means the larger the better; the best performance is marked in bold text. The \(-\) symbol means that metrics are not provided in the result.  
\end{table} 
Table 1: Model performance on LEVIR. * YOLOv5s-ViGE shows competence against other popular models, i.e., YOLOv5m and YOLOX-s. It is noticed that YOLOX adopts several tricks on the basis of YOLOv5, e.g., decouple head, SimOTA, and so on. We only add a ViGE block on the basis of YOLOv5s and obtain an approximate accuracy with YOLOX-s, which proves the effectiveness of the ViGE block in improving detection accuracy. Result on airport surface dataset. We show different models' test results with their average value and standard deviation on the airport surface dataset in Table 2 as follows. We illustrate the detection result on an image in Figure 6: \begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Models** & **P**\(\uparrow\) & **R**\(\uparrow\) & **mAP.5**\(\uparrow\) & **mAP.5:95**\(\uparrow\) \\ \hline YOLOv5s & **0.851 \(\pm\) 0.020** & 0.667 \(\pm\) 0.018 & 0.768 \(\pm\) 0.024 & 0.378 \(\pm\) 0.003 \\ YOLOv5m & 0.768 \(\pm\) 0.016 & 0.773 \(\pm\) 0.005 & 0.823 \(\pm\) 0.011 & 0.390 \(\pm\) 0.009 \\ YOLOv5s-ViG & 0.805 \(\pm\) 0.012 & **0.797 \(\pm\) 0.004** & 0.822 \(\pm\) 0.007 & **0.386 \(\pm\) 0.009** \\ YOLOX-s & \(-\) & \(-\) & 0.756 \(\pm\) 0.005 & 0.367 \(\pm\) 0.006 \\ YOLOv5s-ViGE (ours) & 0.814 \(\pm\) 0.013 & 0.741 \(\pm\) 0.003 & **0.837 \(\pm\) 0.005** & 0.381 \(\pm\) 0.007 \\ \hline \hline \end{tabular} Note: The \(\uparrow\) symbol means the larger the better; the best performance is marked in bold text. The \(-\) symbol means that metrics are not provided in the result.  
\end{table} 
Table 2: Model performance on airport surface dataset. Figure 5: Detection results for LEVIR dataset (a sample). Figure 6: Detection results for airport surface dataset (a sample). According to the experiment above, we obtain the conclusion that:  
* Our YOLOv5s-ViGE model gets about 7% mAP.5 improvement compared to vanilla YOLOv5s, which verifies the ViGE block's potential on feature extraction. What is more, the original ViG block brings accuracy improvement to YOLOv5s as well. These phenomena demonstrate that GNN can also perform well in image feature extraction. * Though YOLOX-s performs better on LEVIR, its accuracy is lower than YOLOv5s-ViGE on the airport surface dataset. This result indicates that YOLOv5s-ViGE obtains a better convergence on small-scale datasets from another side.