# A Pedestrian Detection Network Based on an Attention Mechanism and Pose Information
## 3 Methodology
### Pedestrian Recognition Network
In each subsequent stage, the current prediction is generated by combining the prediction from the two branches in the previous stage with the original image feature F:  
\[M_{t}=\rho_{t}(F,M_{t-1},A_{t-1}),\forall t\geq 2,\ A_{t}=\varphi_{t}(F,M_{t-1},A_ {t-1}),\forall t\geq 2, \tag{6}\]  
where \(\rho_{t}\) and \(\varphi_{t}\) are composed of two \(7\times 7\), two \(5\times 5\), and two \(1\times 1\) convolution layers, respectively. In the last stage, by connecting the confidence graph \(M_{6}\) with the affinity field \(A_{6}\), the model obtains pedestrian pose information. The pedestrian pose module is initialized using the improved OpenPose [16] model, which was pretrained on a pose dataset. The parameters of the pedestrian pose module remain unchanged during the whole training process as the pedestrian pose module extracts useful pose information. This network maps the pedestrian pose information into a 128-dimension pose feature description \(f_{p}\) with a fully connected layer, because pedestrian pose information can be used to judge whether a candidate region contains a pedestrian target or background. Finally, we use a fully connected classifier and cross-entropy loss \(L_{p}\) to optimize the feature description \(f_{p}\). The obtained visual feature descriptions \(f_{v}\) and pose feature descriptions \(f_{p}\) are first spliced to generate 256-dimension feature descriptions. Second, the fully connected layer is used to reduce the dimensions of the fused features. Finally, the joint loss function \(L_{rpn}\) is minimized to optimize the pedestrian recognition network:  
\[L_{rpn}=L_{c}+\lambda L_{v}+\mu L_{p} \tag{7}\]  
where \(L_{c},\ L_{p}\), and \(L_{p}\) represent the binary cross-entropy losses of the classification, pedestrian pose, and visual feature modules, respectively. \(\lambda\) and \(\mu\) are two weight parameters, referring to previous findings [52], we choose \(\lambda\) = 0.5 and \(\mu\) = 0.5. During the model testing, for any target candidate region, we perform weighted fusion of the confidence scores \(score_{r}\), which is the output of the region proposal network, and the confidence score \(score_{p}\), which is the output of the pedestrian recognition network. The combined score is taken as the final confidence score of the candidate region:  
\[score=\alpha\cdot score_{r}+\beta\cdot score_{p} \tag{8}\]  
Figure 3: The detailed structure of the proposed pedestrian recognition network. where \(\alpha\) and \(\beta\) are two weight parameters. The final prediction is more accurate with the use of fused scores. Based on the results of the parameter analysis in our experiments, we set \(\alpha\) to 0.2 and \(\beta\) to 0.8. When both networks have low predictive scores, the candidate region is considered to be background. To more clearly describe our method, the algorithm is described in Algorithm 1. Algorithm of the proposed pedestrian recognition method. ```
Input: Image F, first threshold parameter \(\lambda\), second threshold parameter \(\mu\) Output: Pedestrian detection results {Pi (i = 0,1,2...)}
1. Initialize pedestrian region proposal network (RPN) using existing detectors (e.g., ALFNet, FCOS) 2. Generate candidate bounding boxes using RPN \(\text{B}_{\text{prn}}\) = RPN(F) 3. Initialize pedestrian recognition network 4. For each candidate bounding box b in \(\text{B}_{\text{prn}}\), 4.1. Extract visual feature descriptions \(f_{p}\) 4.1.1. Resize candidate region to 256 \(\times\) 256 4.1.2. Use the first 22 layers of ResNet50 and convolutional blocks to obtain \(f_{p}\) 4.2. Extract pose feature descriptions \(f_{p}\) 4.2.1. Resize candidate region to 256 \(\times\) 256 4.2. Use improved OpenPose with first 22 layers of ResNet50 to extract feature map F 4.2.3. Iteratively predict confidence map M and affinity field A to obtain pose information 4.2.4. Map pose information to 128-dimensional feature description \(f_{p}\) using a fully connected layer 4.3. Extract attention information \(f_{a}\) using pedestrian attention module 4.4.