# Residual Transformer YOLO for Detecting Multi-Scale Crowded Pedestrian
## 3 The Proposed Method
### Multi-Scale Fusion Strategy  
RT-YOLO combines BOTrans and employs a multi-scale early fusion strategy during training. This strategy starts with feature fusion, followed by sending the fused feature maps to the prediction head. As the depth of the network progresses, the feature map scale changes with the receptive field, which is the size of the area mapped on the original image by the pixel points on the feature map. High-level feature maps with low resolution, rich semantic information and a global receptive field are advantageous for detecting medium and large objects. Conversely, low-level feature maps with high resolution, detailed information, and a small receptive field are better suited for detecting small-scale objects. Therefore, by fusing feature maps at different scales and combining high-dimensional semantic information, the feature map scale changes with the receptive field. Figure 5: Multi-head Self-Attention in BOTrans. Figure 6: The structure of Detect. mation with low-dimensional detail information, the network characterization ability is improved. This reduces the instances of missing or falsely detecting small-scale objects. The multi-scale fusion training in the RT-YOLO algorithm involves three steps. First, the Backbone extracts image features to create four feature maps at different scales. Next, these four feature maps are fused with the main path of the Neck and up-sampled to produce new feature maps. In the third step, E-ELAN and BOTrans enhance the new feature maps for feature extraction, and then they are down-sampled to achieve feature fusion, resulting in four feature maps of various scales. In YOLOv7, three feature maps are sent to the detection head for prediction, and one of them is dedicated to detecting small objects. In RT-YOLO, there are four feature maps in total, with two of them specifically used for detecting small objects. As shown in Figures 3 and 7, the two low-level feature maps from layers 4 and 6 in the Backbone have high resolution and rich detailed information, making them suitable for small-scale object detection. The two feature maps are first modified in terms of channel numbers using a 1 \(\times\) 1 Convolution with a stride of 1. The output features from layer 4 are then combined with the primary features in Neck layer 19 after being up-sampled. Subsequently, the output features from Layer 4 are fused with the output features from Layer 19, and after enhanced feature extraction by BOTrans, Feature Map 4 is obtained. The output features from Layer 6 are fused with the output features from Layer 18, then combined with Feature Map 4 to yield Feature Map 3. These feature maps combine detailed information from low-level networks and rich semantic information from deep-level networks. They are designed for the specific purpose of detecting small-scale and occluded objects. The acquisition method for Feature Map 2 and Feature Map 1 remains consistent with YOLOv7. In order to observe objects information of interest for each scale of feature map, all objects localization information is presented through gradient heat maps. The GradCam [25] tool with a confidence threshold (conf) = 0.5 is used to visualize the gradient heat map, as shown in Figure 8. Objects in Figure 8a have head and human body, which are small-scale and partially obscured. Feature map 1 in Figure 8c lacks detailed information and primarily extracts object information at larger scales in the near field. Feature map 2 in Figure 8d extracts the global large-scale object information based on Figure 8c, but struggles to distinguish edge features. Feature map 3 in Figure 8e becomes more sensitive to small-scale human features after the initial fusion with low-level features. However, it may not accurately locate smaller-scale head features. Feature map 4 in Figure 8f is rich in detail and semantic information, making it sensitive to small-scale objects and accurately locating head features in the image.