# Exploring Gait Recognition in Wild Nighttime Scenes
## 3 GaitSAT Framework
### Network Structure  
To capture the multifaceted features of human gait, our network integrates both 2D pose and bone data. This dual-input approach ensures a comprehensive representation of gait characteristics. The backbone network consists of two parallel streams, each dedicated to processing one of the two input types, allowing for specialized feature extraction from distinct gait representations. In each stream, we employ a series of SA-GC blocks to model the intrinsic relationships between joints and bones within individual frames of a gait sequence, effectively learning spatial features. However, the intrinsic connections within single skeleton frames cannot reflect the periodic patterns of gait. It is essential to capture the temporal dynamics of gait patterns. Consequently, we employ the proposed multi-scale temporal rhythm perception module to learn gait features at different time strides,  
Figure 5: Examples of pedestrians with varying walking speeds in GaitDN. (**a**-**c**) illustrate the process of a pedestrian taking a step, from the moment the left foot leaves the ground to the moment the right foot leaves the ground in the subsequent step. (**a**) The pedestrian takes one step in 12 frames. (**b**) The pedestrian takes one step in 16 frames. (**c**) The pedestrian takes one step in 18 frames.  
Figure 6: The framework of GaitSAT.  
enriching the temporal dimension of gait features. Finally, we perform pooling operations on the features learned at different time strides to obtain a robust and comprehensive final embedding that encapsulates both spatial and temporal gait characteristics. The network architecture of GaitSAT is illustrated in Figure 6. Suppose the output of the multi-scale temporal rhythm perception module is denoted as \(f_{\mathrm{S}}\in\mathbb{R}^{T\times N\times C^{out}}\), where \(C^{out}\) represents the number of output channels, \(T\) represents the number of frames in the sequence, and \(N\) represents the number of key points. The spatial feature mapping can be represented as  
\[f_{np}^{v}=\mathrm{ap}_{v}(f_{S})+\mathrm{mp}_{v}(f_{S}) \tag{4}\]  
where \(\mathrm{ap}(\bullet)\) refers to average pooling, \(\mathrm{mp}(\bullet)\) refers to max pooling along the \(N\) dimension, and \(v\) denotes the output of the \(v\)-th branch of the multi-scale temporal rhythm perception module. The average pooling operation captures the overall distribution of joint features, while max pooling identifies the most salient features across the joint dimension. By combining both average and max pooling along the joint dimension, we aim to create a comprehensive representation that captures both the general patterns and the distinctive features of an individual's gait across different joints and bones. Next, set pooling is employed to aggregate the temporal information:  
\[f_{tp}^{v}=\mathrm{sp}_{v}(f_{np}^{v}) \tag{5}\]  
where \(\mathrm{sp}(\bullet)\) refers to max pooling along the \(T\) dimension. Finally, the outputs from both branches (including the joint branch and the bone branch) are concatenated together to form the final recognition representation:  
\[\begin{split}& F_{branch}=\mathrm{cat}(f_{tp}^{1},...,f_{tp}^{v}) \\ & F_{final}=\mathrm{cat}(F_{joint},F_{bone})\end{split} \tag{6}\]  
This concatenation step integrates the complementary information learned from both joint and bone data, creating a comprehensive feature vector that encapsulates the full range of gait characteristics observed from different anatomical perspectives.