# Non-Local Spatial and Temporal Attention Network for Video-Based Person Re-Identification
## 1 Introduction  
Person re-identification (Re-ID) aims to use computer vision algorithms for cross-camera tracking, which means finding the same person under different cameras. Person Re-ID intends to identify a probe person in a camera by matching his/her images or videos and has many practical applications, including intelligent surveillance and criminal investigation. Person Re-ID can be divided into image-based and video-based person Re-ID. Image-based person Re-ID has made significant progress in terms of both solutions [1, 2] and the construction of large benchmark datasets [3, 4]. Recently, more work [5, 6, 7] has begun to focus on video-based person Re-ID because of the richer information contained in video data as compared to image data. By extracting more spatial and temporal cues from video data, video person Re-ID has the potential to solve some of the challenges faced in image person Re-ID, e.g., the visual blocking of pedestrians as they walk. In the video-based person Re-ID task, the video-based dataset is composed of many consequent sequences of images rather than static images. Here, we need to declare that the video is composed of several sequences, and a sequence includes several frames of images in this article. The critical challenge is to make use of the temporal clues embedded in the sequences. Some previous work [5, 6, 7] has typically divided this task into two steps. In the first step, image-based convolutional neuralnetworks (CNNs) are used to extract features from each frame in the video to obtain frame-level features. The second step is to aggregate the features based on each frame to form a video-level feature that can represent the entire video. The distance between the video-level features of the input video is generally used to indicate whether the video contains the same target person or not. Ideally, the probability of including the same person in a video is higher if the distance between two video-level features is smaller and lower for two videos that are further apart. The main problem with video-based person Re-ID now lies in the second step, which is temporal feature aggregation. Depending on their ways of temporal feature learning, existing work can be roughly divided into the following three categories:  
1. Extraction of dynamic features from other CNN inputs, e.g., by optical flux [8]. 2. Extracting spatial and temporal features by regarding the video as 3-D data, e.g., via 3-D CNN [9; 10]. 3. Learning robust person representations by temporally aggregating frame-level features, e.g., through recurrent neural networks (RNNs) [5; 8; 11] and temporal pooling or weights [7; 12; 13]. The third category to which our work belongs is currently dominant in video-based person Re-ID tasks. Most existing methods represent the frame of the video as a feature map and then use an average or maximum pooling across frames to obtain a representation of the input video. However, this approach tends to fail when occlusions are frequent in the video because it processes all images in the video with equal importance. In order to distill the relevant information from a video and weaken the influence of noisy samples, some works have learned the temporal attention score of each frame in a given video by using recurrent neural networks (RNNs) to solve this problem. The limitation of the RNN method is that it requires sequential calculations to be performed. As a result, it is difficult to compute in parallel and make full use of the graphics processing unit (GPU) hardware. Additionally, a single recurrent operation could only calculate the dependency between the current and the latest frame. In general, it is difficult for RNNs to capture long-range dependencies. In this paper, we propose a non-local spatial and temporal attention network for video-based person Re-ID. We improve the non-local neural network [14] and apply it to the video-based person Re-ID task with excellent results.