# Research on Multi-Modal Pedestrian Detection and Tracking Algorithm Based on Deep Learning
## 5 Experimental Analysis
### Analysis of Results  
In this experiment, the proposed multi-modal pedestrian detection algorithm based on YOLOv4 using the channel stack fusion module is called Double-YOLOv4-CSE, and the common pedestrian detection algorithm based on single visible light mode is called visible-YOLOv4. All the above-mentioned detection algorithm models were trained, verified, and tested on the cleaned KAIST dataset. Through the analysis and comparison of the samples predicted by the experimental model, it was found that the detection method using the mixed mode feature fusion method proposed in this chapter had better detection effects, as shown in Figure 8 below. From this, it can be seen that the Double-YOLOv4-CSE proposed in this paper can more accurately detect dark pedestrians on the roadside in low-light scenes such as night and can effectively improve the ability to capture dark pedestrians. At the same time, to further verify the effect of introducing multi-modal and corresponding feature fusion methods on improving pedestrian detection performance in the whole scene. This algorithm was compared with the logarithmic average miss rate and average accuracy obtained by the current mainstream and mature multi-modal pedestrian detection algorithms ACF + T + THOG [18], Fusion-RPN [4], GFD-SSD [19], and AR-CNN [20] on the KAIST dataset. The AR-CNN algorithm serves as a mechanism for amalgamating RGB images and infrared images. It leverages the global feature maps \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Prediction Branch** & **Output Feature Map Size** & **Predict Prior Box Size** &
\begin{tabular}{c} **Output the Predicted Bounding** \\ **Box Number** \\ \end{tabular} \\ \hline Large target prediction layer & \(16\times 16\) & \((17\times 35)\); \((18\times 45)\); \((24\times 44)\) & \(3\times 16\times 16\) \\ Medium target prediction layer & \(32\times 32\) & \((22\times 57)\); \((28\times 63)\); \((33\times 77)\) & \(3\times 32\times 32\) \\ Small target prediction layer & \(64\times 64\) & \((99\times 95)\); \((47\times 126)\); \((63\times 170)\) & \(3\times 64\times 64\) \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Related data of nine kinds of anchor frames used in the experiment. Figure 8: Comparison of traditional single-mode pedestrian detection algorithm and multi-mode pedestrian detection algorithm. derived from both modalities to identify candidate boxes. By expanding the coverage area of candidate boxes identified by the region proposal network (RPN), feature maps corresponding to individual modalities are extracted. Employing the infrared mode as the reference, the algorithm predicts the relative offset of the RGB mode. Subsequently, alignment of the two feature maps is performed based on this offset. The default IoU threshold was 0.5, and the experimental results are shown in Table 3 below. From the data presented in the table, it is evident that the proposed Double-YOLOv4-CSE algorithm exhibits superior detection performance compared to other multimodal pedestrian detection algorithms. Compared with AR-CNN algorithm, the proposed model has certain advantages in average accuracy and log-average omission rate. Even when compared with the excellent Visible-YOLOv4 algorithm, the proposed Double-YOLOv4-CSE algorithm achieves a 5.0% improvement in accuracy and a 6.9% reduction in the logarithmic average missing rate. These results strongly indicate the outstanding performance of the Double-YOLOv4-CSE algorithm in pedestrian detection and tracking tasks. The core concept of YOLO algorithms is to treat object detection as a regression problem, achieving fast and accurate detection by directly predicting bounding boxes and object categories. This fundamental principle remains consistent across different versions of the YOLO algorithm. Consequently, strategies such as multimodal information fusion and contextual information enhancement employed in the Double-YOLOv4-CSE algorithm can also be implemented in other versions of the YOLO algorithm.