# Improved Dab-Deformable Model for Runway Foreign Object Debris Detection in Airport Optical Images
## 2 Related Theory  
The Transformer was originally introduced to address the challenges of natural language processing (NLP) [39], employing a global attention mechanism within the context of text sequences. This approach involved the parallel processing of the input sequence and the design of a multi-head self-attention mechanism that incorporated positional information to capture global relationships. As a result, the Transformer architecture is widely used for modeling global relationships. In [40], the Transformer is used to learn global information, while the CNN is used to extract local information for efficient bearing fault diagnosis. [PERSON] et al. [41] proposed a mixed network that integrated a CNN and a Transformer to combine global and local information for medical image segmentation. The DETR model was the first to introduce the Transformer architecture into the field of object detection, achieving a performance on the COCO dataset comparable to that of the best detection models available at the time. However, DETR is constrained by its structural design; the global attention mechanism applied to the entire sequence incurs significant computational overhead, limiting the model's performance and inhibiting its ability to perform multi-scale feature aggregation. This makes it unsuitable for detecting small targets. Deformable-DETR replaces the multi-head self-attention mechanism in the encoder with a multi-scale deformable convolution attention mechanism, addressing the high computational demands of DETR and significantly reducing training time. The local attention mechanism, based on reference points, replaces the global attention mechanism, further accelerating the training process. In Deformable-DETR, relative positional features are aggregated by combining reference points with hierarchical information, using a method that involves dimensionality reduction and the concatenation of multi-scale features, which supersedes traditional FPN fusion. This approach effectively leverages the hierarchical and positional information of reference points to enhance detection performance. Notably, even after extending multi-level feature maps into the same sequence, positional information is retained, allowing for better utilization of multi-scale and spatial target information, thereby improving performance in small target detection. Deformable-DETR uses deformable convolution and reference points to address the original DETR model's issues of high computational requirements, long training times, and limitations in small target detection. However, these improvements in computational efficiency do not enable the DETR series to achieve high precision in small target detection. Building on DETR and Deformable-DETR, Dab-Deformable combines their advantages to achieve significant performance and efficiency improvements. The authors of Dab-Deformable identify that DETR's slow training convergence results from the decoder's learnable queries lacking effective positional guidance. By conducting a comparative analysis of the attention mechanisms in the encoder-decoder architecture and the compositional principles of queries, the authors reveal two critical limitations: the semantic queries in the encoder are initialized with zero values, and the positional queries lack explicit spatial priors. These deficiencies collectively necessitate extended training cycles for the model to establish meaningful feature representations. To address these fundamental issues, the authors propose enhancing the positional prior for the decoder's cross-attention mechanism [42]. The authors integrate object scale information into the spatial prior by introducing four-dimensional anchor boxes. This architectural modification establishes explicit constraints on target localization during the encoding phase. This reformulated paradigm demonstrates threefold advantages: First, it enhances the interpretability of DETR's query mechanism by establishing clear spatial-semantic correspondences.