# Residual Transformer YOLO for Detecting Multi-Scale Crowded Pedestrian
## 3 The Proposed Method
### Normalization-Based Attention Module  
The attention mechanism allows the neural network to concentrate on essential image areas, suppressing interference from irrelevant regions. Its lightweight, plug-and-play nature enhances the performance with lower cost. Attention mechanisms are generally categorized into spatial, channel, and hybrid domains, such as CBAM [26], GAM [27], and other attention mechanisms that excel in neural networks are designed in these perspectives.  
In RT-YOLO, Normalization-based Attention (NAM) considers weight factors contribution to attention. It employs the scaling factor of Batch Normalization to calculate these weights, eliminating the need for repetitive stacking of fully connected and convolutional layers. NAM enhances RT-YOLO focus on specific features or channels, automatically  
Figure 8: Gradient heat map of Feature map.  
adjusting weights based on input data characteristics and network requirements, reducing interference, and enabling the model to concentrate on meaningful features. The NAM adopts the modular integration approach from CBAM and introduces weight contribution factors to reconfigure the spatial and channel attention modules. This enables NAM to be seamlessly integrated directly after the network layer and residual structure. The Batch Normalization (BN) weight contribution factor is calculated as follows:  
\[B_{out}=BN(B_{in})=\gamma\frac{B_{in}-\mu_{\beta}}{\sqrt{\sigma_{\beta}^{2}+ \epsilon}}+\beta \tag{3}\]  
where \(\mu_{\beta}\) and \(\sigma_{\beta}\) are mean and standard deviation of output feature map \(B\). \(\gamma\) and \(\beta\) are trainable mapping parameters and also variance in \(BN\). In the model training process, the larger variance represents the richer feature information contained in the channel, which is a more important region to concern.  
In RT-YOLO, NAM is added before the feature fusion is performed to improve the feature extraction efficiency, and the specific operation is to calculate channel attention and spatial attention for input features, whose calculation formula is computed as follows:  
\[y=M_{s}(M_{c}(X)) \tag{4}\]  
Here, \(X\) represents the input, \(y\) represents the output, and the structure of the channel attention sub-module \(M_{c}\) redesigned by weight contribution factor is shown in Figure 9 and Equation (5).  
\[M_{c}=sigmoid(W_{\gamma}(BN(F_{1}))) \tag{5}\]  
Here, \(\gamma\) is the scaling factor of each channel and weight is \(W_{\rm i}=\frac{\gamma_{\rm i}}{\sum_{j=0}\gamma_{j}}\), when the feature map passes the channel attention sub-module in process called pixel normalization, the obtained feature map is input to the spatial attention sub-module \(M_{s}\) that combines weight contribution factors, whose structure is shown in Figure 10 and Equation (6).  
\[M_{s}=sigmoid(W_{\lambda}(BN(F_{2}))) \tag{6}\]  
Here, \(\lambda\) is the scaling factor of the space, and the weight is \(W_{\rm i}=\frac{\lambda_{i}}{\sum_{j=0}\lambda_{j}}\). In order to eliminate effects caused by irrelevant weights, NAM adds a regular term to its loss function, whose expression is Equation (7).  
\[Loss=\sum_{(x,y)}l(f(x,W),y)+p{\sum}g(\gamma)+p{\sum}g