# Improved DeepSORT Algorithm Based on Multi-Feature Fusion
## 4 The Improved Feature Extraction Network  
In order to improve the performance of the feature extraction network, ResNet50 is selected as the backbone network. With the deepening of the network layer, the overall content of pedestrian appearance is richer, and the semantic information is more accurate. Figure 6: GIOU distance measurement effect display. (**a**) The detection frame is close to the prediction frame. (**b**) The detection frame is far from the prediction frame. Figure 7: Effect drawing of GIOU measurement in different overlapping modes. (**a**) Horizontal overlap. (**b**) Cross overlap. At the same time, regarding the integration of the features of different levels, improved richness of the pedestrian feature output from the feature extraction network improves tracking accuracy and deals with the complex external environment. At the same time, in order to reduce the number of parameters, the number of channels for network residual blocks is reduced. The backbone network structure of feature fusion is shown in Figure 8.  
From Figure 8, the residual module of the backbone network is divided into four stages according to the number of output channels. The first stage is composed of three identical residual structures. In the second to fourth stages, the convolution with size \(3\times 3\) in the residual module of the first layer in each stage has a stride of two; all other convolution strides are one. The dimension of the final output feature of the whole pedestrian feature extraction network is 512, which can express more-abundant pedestrian feature information than the 128 dimension of the original network. At the same time, at the end of the main network and at the end of the third and second stages, the output feature maps \(C_{0},C_{1},C_{2}\) of each layer are extracted and passed to the FPN network. The output feature information of the \(C_{0}\) layer has rich, high-level semantic information, which improves large-target pedestrian feature extraction but has a poor effect on small-target pedestrian feature extraction. Although the feature map information of \(C_{1}\) and \(C_{2}\) outputs are not as rich as that of \(C_{0}\), the output feature map information of \(C_{0}\) is not as rich. However, it has better extraction for smaller pedestrian targets. It may also retain some spatial information. Figure 8: Feature fusion backbone network structures. A Rectified Linear Unit (ReLU) is used as the activation function in the whole network structure, and a cross-entropy loss function is used for training. At the same time, each residual module is batch-normalized, which makes the model converge faster and improves the training efficiency and stability of the model. Because the data distribution is more uniform, it is less prone to over-fitting. The extracted \(C_{0},C_{1}\) and \(C_{2}\) are passed into the FPN module, and the network implementation process is shown in Figure 9.  
As shown in Figure 9, the feature maps \(C_{0},C_{1}\) and \(C_{2}\) of different levels output by the backbone network are fused through the FPN network. \(C_{0}\) operates through a convolution with a size \(1\times 1\). \(C_{0}\) is converted from 512 to 256 dimensions, and a concat function combines it with \(C_{1}\) after upsampling. After the concat function, the same convolution operation is used to turn it into 128 dimensions, and then after upsampling, the downsampled feature information is combined with \(C_{2}\). using a concat. After the concat, the output dimension is converted to 512 by a convolution operation with convolution dimensions of \(3\times 3\) and \(1\times 1\). Then, through a mean pooling layer with a size of \(5\times 5\) and a stride of two, the final pedestrian appearance eigenvector is obtained after full joining and batch normalization operations to calculate the appearance similarity measure between the track and the detection. ResNet50 was used as the backbone network to extract the appearance information of pedestrian targets.