# Two-Path Spatial-Temporal Feature Fusion and View Embedding for Gait Recognition
## 2 Related Work  
**Gait recognition:** Current deep-learning-based gait recognition methods can be broadly classified into two categories: model-based [13; 14; 15; 16] and appearance-based [4; 5; 6; 7; 8; 17; 18; 19; 20; 21; 22; 23; 24; 25; 26; 27; 28; 29; 30]. Model-based methods leverage the relationships between bone joints and pose information to create models of walking patterns and human body structures [13], such as OpenPose [14], HRNet [15], and DensePose [16]. These methods exhibit stronger robustness to clothing variation and carrying articles. However, model-based methods depend on accurate joint detection and pose estimators, which can significantly increase the computational complexity and may lead to inferior performance in certain scenarios. Conversely, appearance-based methods use gait silhouettes (the binary images shown in Figure 1) as the model's input and capture spatial and temporal characteristics from silhouettes by CNNs [17]. Gait silhouettes can describe the body state in a single frame at a low computational cost and more detailed information in each silhouette image can be preserved directly from the original silhouette sequences. The silhouettes are the basis of appearance-based methods, and the quality of silhouettes will directly affect the performance of the gait recognition system. Moreover, spatial feature representations can be obtained from the silhouette of an individual frame, which can represent appearance characteristics, while temporal feature representations can be captured from consecutive silhouettes, in which the relationship between adjacent frames can reflect the temporal characteristics and motion patterns. Therefore, some appearance-based methods have overcome the challenges of pose estimation and achieved competitive performance [4; 5; 6; 7; 8; 21; 22; 29]. Particularly, the first opensource gait recognition framework, named OpenGait ([[https://github.com/ShiqiYu/OpenGait](https://github.com/ShiqiYu/OpenGait)]([https://github.com/ShiqiYu/OpenGait](https://github.com/ShiqiYu/OpenGait)), accessed on 29 January 2023) [18], encompassed a series of state-of-the-art appearance-based methods for gait recognition. In this paper, our approach is categorized as appearance-based and we count on binary silhouettes as our input data without the need for pose estimation or joint detection. By focusing on silhouettes, we aim to reduce the influence of variations in subjects' appearance, thereby enhancing the accuracy and robustness of our gait recognition approach. Figure 1: Overview of the whole framework, which mainly consists of two modules as: two-path spatial-temporal feature fusion module and view embedding module. The two-path spatial-temporal feature fusion module consists of three component including multi-scale feature extraction (MSFE), frame-level spatial feature extraction (FLSFE) and multi-scale temporal feature extraction (MSTFE). View embedding module consists of two components: view prediction and HPP with view embedding. **Spatial feature extraction modeling:** Regarding the range of feature representations in spatial feature extraction modeling, two main approaches are commonly used: global-based and local-based methods. Specifically, global-based methods focus on exploring gait silhouettes as a whole to generate global feature representations [4; 5; 12; 17; 19; 20]. For instance, [PERSON] et al. [12] proposed the GEI and utilized 2D CNNs to obtain global gait feature representations from the GEIs. Similarly, GaitSet [4] and GLN [5] extracted global gait features at the frame-level by 2D CNNs. Conversely, local-based methods usually segment the silhouettes into multiple parts to establish the local feature representations [7; 21; 22], and focus more on learning the local information of different body parts. For example, [PERSON] et al. [21] split human gait into four distinct segments and adopted 2D CNNs to capture more detailed information from each of these segments. [PERSON] et al.