GLGait: A Global-Local Temporal Receptive Field Network for Gait Recognition in the Wild

Authors: guozhen  peng
yunhong  wang
shaoxiong  zhang
yuwei  zhao
annan  li
  glgait
Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-03-14 09:30:15

Abstract: Gait recognition has attracted increasing attention from academia and industry as a human recognition technology from a distance in non-intrusive ways without requiring cooperation. Although advanced methods have achieved impressive success in lab scenarios, most of them perform poorly in the wild. Recently, some Convolution Neural Networks (ConvNets) based methods have been proposed to address the issue of gait recognition in the wild. However, the temporal receptive field obtained by convolution operations is limited for long gait sequences. If directly replacing convolution blocks with visual transformer blocks, the model may not enhance a local temporal receptive field, which is important for covering a complete gait cycle. To address this issue, we design a Global-Local Temporal Receptive Field Network (GLGait). GLGait employs a Global-Local Temporal Module (GLTM) to establish a global-local temporal receptive field, which mainly consists of a Pseudo Global Temporal Self-Attention (PGTA) and a temporal convolution operation. Specifically, PGTA is used to obtain a pseudo global temporal receptive field with less memory and computation complexity compared with a multi-head self-attention (MHSA). The temporal convolution operation is used to enhance the local temporal receptive field. Besides, it can also aggregate pseudo global temporal receptive field to a true holistic temporal receptive field. Furthermore, we also propose a Center-Augmented Triplet Loss (CTL) in GLGait to reduce the intra-class distance and expand the positive samples in the training stage. Extensive experiments show that our method obtains state-of-the-art results on in-the-wild datasets, $i.e.$, Gait3D and GREW. The code is available at https://github.com/bgdpgz/GLGait.

Main paper: 
 introduction: 
 Gait recognition is a technique that identifies pedestrians by analyzing their walking patterns. Unlike other biometric characteristics such as the face and iris, gait can be captured from a distance without pedestrian cooperation, thus attracting increasing attention.Many advanced appearance-based methods [4, 12, 20, 28] using silhouette sequences as input have obtained successful performance on in-the-lab datasets such as CASIA-B [51] and OU-MVLP [40] . However, the performance of these methods drops dramatically on in-the-wild datasets, 𝑖.𝑒., Gait3D [52] and GREW [54] . The primary reason lies in that the scenario of existing in-the-lab datasets differs greatly from in-the-wild ones. In wild scenarios, pedestrians may walk at varying velocities or follow a non-straight routine, and even can be occluded by other pedestrians or objects. These noisy factors are challenging for methods designed according to in-the-lab datasets, resulting in performance degradation.To address the aforementioned issue, we first compare silhouette sequences between CASIA-B [51] and Gait3D [52] datasets. As shown in Figure 1 (a), multiple and evenly distributed gait circles can be clearly observed from in-the-lab sequences (see pedestrian #1). Thus, appropriate local temporal receptive field can facilitate the model in learning the pattern of a complete gait circle. While in wild scenarios, (see pedestrian #2) such ideal temporal segments are no longer available, since the variations of pace and walking directions have a great influence on the appearance. Therefore, a global temporal receptive field is necessary for aligning gait patterns. Recently, some methods [10, 11, 46] have been proposed to address the issue of gait recognition in the wild. These methods are mainly centered around Convolutional Neural Networks (Con-vNets). Compared with methods designed according to in-the-lab datasets, they often incorporate more convolutional operations to exhibit a larger receptive field. Given that gait silhouettes are typically quite small, such as 64 × 64 pixels, the spatial receptive field of these methods is already sufficient. However, the temporal receptive field obtained through convolutional operations is significantly insufficient. As shown in Figure 1 (b), the temporal receptive field usually can only cover around 50 silhouettes in a sequence, which is quite limited in contrast to typical frame number of in-the-wild gait sequences. The limited local temporal receptive field struggles to capture adequate information about pedestrian body shape changes, thus a global temporal receptive field is essential.Some works [2] use visual transformer block [7] to obtain a global temporal receptive field. However, directly replacing convolution blocks with transformer blocks cannot necessarily enhance a local temporal receptive field. Considering that a transformer block usually applies multi-head self-attention [44] (MHSA) to obtain a global receptive field, adding MHSA before temporal convolution operation in ConvNets is a possible resolution to obtain a globallocal temporal receptive field. However, due to using the output of ConvNets as input to MHSA, dimension explosion occurs in token size with large channels in ConvNets, such as 512, resulting in high memory consumption and computation cost. To address the issue, we propose Pseudo Global Temporal Self-Attention (PGTA). Compared with MHSA, PGTA reduces complexity in two aspects. First, considering the receptive field issue in the temporal dimension, we separate the spatial dimension, only calculating the patch size from the temporal dimension. Secondly, we separate the patch size from tokens in PGTA inspired by [33] , obtaining a pseudo temporal receptive field of each element in tokens. With the same temporal convolution kernel size and patch size, these pseudo global temporal receptive fields are naturally aggregated to a truly global one. We name the combination of PGTA and temporal convolution operation as Global-Local Temporal Module (GLTM).Based on GLTM, we design a Global-Local Temporal Receptive Field Network, named GLGait. The backbone of GLGait consists of a vision encoder and Global-Local 3D (GL-3D) Blocks. In the vision encoder, GLGait encodes a preliminary pedestrian representation. For GL-3D block, in the temporal dimension, GLTM is used to effectively obtain a global-local temporal receptive field. While in the spatial dimension, we use 2D convolution operations, the reason is straightforward: the spatial receptive field is already sufficient by convolution operations.Furthermore, inspired by center loss [17, 48] , we also propose a simple yet effective loss function, named Center-Augmented Triplet Loss (CTL) to assist in model training as a component in GLGait.Based on conventional triplet loss [18] , CTL additionally considers the class center as a positive sample for each input. This operation has two advantages: reducing the intra-class distance and expanding the positive samples in the training stage.The main contributions are summarized as follows: 4) Extensive experiments demonstrate that our approach obtains the state-of-the-art performance on in-the-wild datasets, 𝑖.𝑒., Gait3D [52] and GREW [54] . 
related works 2.1 model-based gait recognition: 
 Model-based methods [1, 13, 25-27, 35, 41, 42, 49] consider the physical structure of the human body and utilize pose information as input, such as 2D skeletons, 3D joints, and point clouds. For instance, PoseGait [27] utilizes the human body pose information to extract temporal-spatial features and employs ConvNets to extract high-level temporal-spatial features. GaitGraph [42] proposes a pose estimator to extract pose features and adopts graph convolutional neural networks [39] for gait recognition. GPGait [13] uses a unified pose representation as input. Then a part-aware graph convolutional network is proposed to enable efficient graph partition and local-global spatial feature extraction. LidarGait [38] leverages LiDAR to generate gait point clouds for gait recognition.Model-based methods are robust in some scenarios, such as clothes changed [5, 37] . However, pose information is not easy to calculate, thus these methods may be difficult to apply in a new scenario. [11, 14] , FC is the separate fully connected layers [4] , and BNN is BNNeck [32] . The final loss function is composed of a center-augmented triplet loss (CTL) and a cross-entropy loss.Appearance-based methods are employed to learn the body appearance without achieving explicit structures. Most methods [3, 4, 8-10, 12, 19, 23, 28, 45, 46, 50] use silhouette sequences as inputs and use deep neural networks to extract feature [16, 22, 36, 43] . Then, by comparing similarities with other silhouette sequences, the recognition is implemented. Some methods [47, 53] also utilize semantic parsing of pedestrians as the input, obtaining satisfactory performance.Previous works [4, 6, 12, 28] focus on in-the-lab scenarios, obtaining successful performance. However, with the recent development of in-the-wild datasets, 𝑖.𝑒., Gait3D [52] and GREW [54] , these methods are facing new challenges. To employ precise gait recognition in wild scenarios, some methods are proposed. GaitGCI [9] utilizes counterfactual intervention learning to eliminate the impact of confounder, focusing on the discriminative and interpretable regions effectively. DyGait [46] focuses on the extraction of dynamic features and proposes a dynamic augmentation module to learn part features automatically. Fan et al. [11] proposes a simple yet effective ResNet-like [16] framework, which is referred as GaitBase. Based on GaitBase, DGaitV2 [10] obtains higher accuracy by increasing the number of stacked blocks.Compared to pose information, appearance cues are relatively easy to obtain in a new scenario, exhibiting broader adaptability. However, appearance information is susceptible to the influence of pedestrian appearance, exhibiting low robustness in certain specific scenarios such as clothes change [5, 37] . Our GLGait belongs to the appearance-based category, using gait silhouette sequences as input and establishing a global-local temporal receptive field.Vision transformer [7] has achieved successful performance in many fields, such as classification [7] , objective detection [29] , and semantic segmentation [21] . Some works introduce transformer blocks into the gait recognition framework. TransGait [24] proposes a set transformer model with a temporal aggregation operation for obtaining set-level spatio-temporal features. SwinGait [10] utilizes convolutional blocks to extract silhouette feature and feed it to swinformer [30, 31] blocks. However, the former cannot enhance the local temporal receptive field, while the latter only obtains a window-global temporal receptive field. Differently, our GLGait employs Global-Local Temporal Module (GLTM) to both maintain global-local temporal receptive fields. 
method: 
 In this section, we first introduce the pipeline of GLGait. Then, we detail the vision encoder, Global-Local 3D (GL-3D) block, and centeraugmented triplet loss (CTL), respectively. Finally, we explain the optimization.The pipeline of the proposed GLGait is shown in Figure 2 (a) . A simple 2D convolution operation first initializes the silhouette sequences. Then we utilize the vision encoder to obtain a preliminary representation. After that, GL-3D blocks are used to extract both spatial information and global-local temporal information. Temporal Max Pooling operation (TP) and Horizontal Pooling operation [11, 14] are employed to aggregate the features. Finally, a combined loss function consisting of center-augmented triplet loss and cross-entropy loss is used to supervise the learning process.Since gait silhouette is binary, containing limited information [10] , we use a vision encoder to encode a preliminary pedestrian representation. Specifically, we use the conventional Pseudo 3D (P3D) blocks [16, 36] as the components. P3D block uses two 2D convolutions in the spatial dimension and a 1D convolution in the temporal dimension. Using x 𝑖𝑛 ∈ R 𝐶 ×𝑇 ×𝐻 ×𝑊 as input, where 𝐶 is the channels, 𝑇 is frame number, 𝐻 and 𝑊 are the silhouette height and width, respectively. The process can be formulated as, × , × × × , × × , × , × × Linear Norm T-ConvEQUATIONEQUATIONEQUATIONEQUATIONwherex 𝑜𝑢𝑡 ∈ R 𝐶 ×𝑇 ×𝐻 ×𝑊 is the output, C 1𝑑 (•) is the temporal convolution operation, C 2𝑑 (•)is the spatial convolution operation, R is the ReLU activation function.Operation. As shown in Figure 2 (b), GL-3D block consists of two 2D convolutions for spatial information and a Global-Local Temporal Modul (GLTM) for the temporal feature. Specifically, GLTM mainly contains Pseudo Global Temporal Self-Attention (PGTA) and temporal convolution operation as shown in Figure 2 (c). Different from normal multi-head self-attention [44] (MHSA), PGTA places its focus on the temporal dimension, reducing the memory and computation complexity in two aspects. 1) PGTA separates the spatial dimension from the patch size, which means it is only 1D rather than 3D. 2) PGTA separates the patch size from the token inspired by [33] . We exhibit the structure of PGTA in Figure 3 , where we only consider one head for simplicity. Given x 𝑖𝑛 ∈ R 𝐿×𝑇 ×𝐶 as input, where 𝐿 is 𝐻 ×𝑊 , the formula of PGTA is as follows:EQUATIONEQUATIONEQUATION)EQUATIONx 𝑔 = [x 1 ; x 2 ; ...;EQUATION)where 𝑖 ∈ {1, 2, ..., 𝑘 }, 𝑘 is the number of multi-heads in PGTA, 𝑃 is patch size, A is attention matrix, U 𝑞𝑘𝑣 and U 𝑚𝑠𝑎 are parameter matrices in PGTA. After PGTA, a temporal convolution operation is used to enhance the local temporal receptive field. Inspired by [15] , we also add skip connections and 1D convolution operation with the kernel size of one as linear layer after PGTA to obtain a sub-update for each element in tokens. The process is formulated asx 𝑙 = R (C 1𝑑 (x 𝑔 + x 𝑖𝑛 )), x 𝑙 ∈ R 𝐿×𝑇 ×𝐷 𝑚𝑖𝑑 (11) x 𝑜𝑢𝑡 = x 𝑙 U 𝑚𝑙𝑝 + x 𝑔 , U 𝑚𝑙𝑝 ∈ R 𝐷 𝑚𝑖𝑑 ×𝐶 (12)where x 𝑜𝑢𝑡 ∈ R 𝐿×𝑇 ×𝐶 is the output of GLTM, R and C 1𝑑 (•) are same as operations in Equation ( 1) and Equation ( 2), 𝐷 𝑚𝑖𝑑 is the channels of hidden layer, U 𝑚𝑙𝑝 is parameter matrix. Discussion. In this subsection, we undertake a memory and computation complexity analysis of PGTA compared with a normal Spatio-Temporal MHSA [2, 44] . Specifically, we consider two kinds of 𝐷 in Equation ( 6). One is that 𝐷 is the same as the token size.Another is that 𝐷 is a scalar less than 𝐶. The input is x 𝑖𝑛 ∈ R 𝐿×𝑇 ×𝐶 . Given token size 𝑃 × 𝐶, we observe that the token size does not affect the computation complexity in Equation ( 6) and Equation ( 9) as follows,EQUATION= O (𝐿𝑇𝐶𝐷).Therefore, we only consider the computation complexity in Equation (7) and Equation (8) . The formula of complexity in MHSA is asEQUATIONEQUATION)where 𝑁 is 𝐿𝑇 𝑃 , 𝐶 𝑀 𝑚𝑒𝑚 is the memory complexity in MHSA, and 𝐶 𝑀 𝑐𝑜𝑚 is the computation complexity in MHSA. The formula of complexity in PGTA is asEQUATIONEQUATION= O (𝐿𝑇 𝑁 𝐷),where 𝑁 is 𝑇 𝑃 , 𝐶 𝑃 𝑚𝑒𝑚 is the memory complexity in PGTA, and 𝐶 𝑃 𝑐𝑜𝑚 is the computation complexity in PGTA. First, we assume that 𝐷 is the same as the token size, which means information loss does not occur. For MHSA, given the token size 𝑃 𝑙 × 𝑃 𝑡 × 𝐶, where 𝑃 𝑙 is spatial dimension size and 𝑃 𝑡 is temporal dimension size, the memory complexity is O (𝑃 Secondly, we assume that 𝐷 is a scalar less than 𝐶, which means information loss occurs, especially when the token size is large. For MHSA, given the token size 𝑃 𝑙 × 𝑃 𝑡 × 𝐶, the memory complexity is O (𝑃 𝑙 𝑃 𝑡 𝐶𝐷) and the computation complexity is O (𝐿 2 𝑇 2 𝑃 2 𝑙 𝑃 2 𝑡 𝐷).For PGTA, given the token size 𝑃 𝑡 × 𝐶, the memory complexity is O (𝐶𝐷) and the computation complexity is O (𝐿 𝑇 2 𝑃 𝑡 𝐷). PGTA also reduces the memory complexity significantly, from O (𝑃 𝑙 𝑃 𝑡 𝐶𝐷) to O (𝐶𝐷). For computation complexity, the decrease lies in 𝐿 and 𝑃 2 𝑙 × 𝑃 𝑡 . Following the same set above, PGTA reduces the computation complexity to a quarter of MHSA. Furthermore, MHSA loses much information. Assuming the feature dimension is a measure, the information loss of PGTA is only O (𝐶 -𝐷). For MHSA, the information loss even reaches O (𝑃 𝑙 𝑃 𝑡 𝐶 -𝐷).Finally, considering the memory complexity, we set 𝐷 as a scalar in all our experiments.Operation. In contrast to the conventional triplet loss [18] , the center-augmented triplet loss (CTL) additionally incorporates class centers as positive instances for each sample, the formulation is given asEQUATIONEQUATIONwhere x ∈ R 𝐶 is input, x 𝑞 𝑖 is the 𝑖-th positive sample of x, x 𝑛 𝑗 is the 𝑗-th negative sample of x, D (•, •) is distance function, 𝑚 is margin, 𝑄 is the number of positive samples, 𝑁 is the number of negative samples, 𝐵 is the batch size (𝐵 = 𝑄 + 𝑁 + 1). Specifically, x 𝑞 𝑄+1 is the class center of x, and Euclidean distance is used in D (•, •). Discussion. In this subsection, we discuss the mechanism of centeraugmented triplet loss (CTL). Apart from the common effects of triplet loss [18] , CTL has two advantages.First, CTL reduces intra-class distance. As illustrated in Figure 4 (a), for sample x 1 , x 2 is a positive sample, and x 3 is a negative one. When the distance between (x 1 , x 2 ) exceeds that of (x 1 , x 3 ), the triplet loss computes the corresponding gradient represented as solid-line arrows, encouraging (x 1 , x 2 ) to move closer, and (x 1 , x 3 ) to move apart. However, as x 2 moves closer to x 1 , it keeps away the class center w 1 , thus increasing the intra-class distance. Conversely, as shown in Figure 4 (b), by treating w 1 as a positive sample for x 1 and w 3 as one for x 3 , CTL calculates the respective gradients depicted as dashed-line arrows, driving x 1 towards w 1 and x 3 towards w 3 . Due to the superposition of gradients from (x 1 , x 2 ) and (x 1 , w 1 ), x 1 moves closer to w 1 , indirectly preventing x 2 away from the class center w 1 , thus reducing the intra-class distance.Secondly, CTL can directly increase the number of positive samples without expanding the batch size. In gait datasets such as Gait3D [52] , many pedestrian silhouette sequences are limited in quantity, which means a lack of positive samples in a batch. CTL utilizes class centers as positive samples, adding computational complexity to the loss function without affecting the backbone computations of the model in the training stage and model inference in the test stage, presenting a good cost-effective trade-off.In the training stage of GLGait, a combined loss function consisting of center-augmented triplet loss (L 𝑐𝑡𝑙 ) and cross-entropy loss (L 𝑐𝑒 ) is calculated to supervise the learning process,EQUATIONwhere 𝛼 and 𝛽 are hyper-parameters to balance the contributions to the total loss L. 
experiments: 
 In this section, we first introduce the datasets and implementation details. Then, we compare our proposed GLGait with the latest gait recognition methods and analyze the results. Finally, extensive ablation experiments prove the effectiveness of each component in GLGait.The dataset information and implementation details in our experiments are as follows.Gait3D [52] is a large scale gait dataset. Within a supermarket, 39 cameras capture 1,090 hours of videos with 1,920×1,080 resolution and 25 FPS. Through processing, a total of 4,000 subjects, 25,309 sequences, and 3,279,239 frame images are extracted. 3,000 subjects are compiled as the training set, while the remaining 1,000 subjects form the test set. For the testing phase, the probe comprises one sequence from each subject, and the gallery consists of the rest sequences.GREW [54] is one of the largest gait datasets in the wild, including Silhouettes, GEIs, and 2D/3D human poses data types. The raw videos are collected from 882 cameras in large public areas. 7,533 video clips are used, containing nearly 3,500 hours of 1,920×1,080 streams. It has 26,345 subjects and 128,671 sequences, divided into two parts with 20,000 and 6,000 subjects as training set and test set, Table 1 : Comparisons of Rank-1, 5 and 10 accuracies, mean Average Precision (%), and Parameter size (M) on Gait3D [52] and GREW [54] respectively. Each subject in the test set has four sequences, two for probe and two for gallery. Implementation Details. Our experiments are implemented using PyTorch [34] . We design the network capacity referring to the baselines [10, 11] as shown in Table 2 . In Equation 22, 𝛼 and 𝛽 are both set to 1. The kernel size of all convolution operations is 3. In Equation 8, 𝑃 is set to 3, and 𝐷 is the same as 𝐶 in Stage-1. In Equation 11, 𝐷 𝑚𝑖𝑑 is the same as 𝐶 in all stages. For parameters, we only consider the backbone without FC [4] and BNN [32] layers for all experiments. Similar to DGaitV2 [10] , we also partition the model size into three segments: GLGait-Base (GLGait-B), GLGait-Large (GLGait-L), and GLGait-Huge (GLGait-H) to exhibit an appropriate compromise between accuracy and cost. They share identical architectures, except for the variation in the number of channels at each stage, which are (32, 64, 128, 256) , (64, 128, 256, 512) , and (128, 256, 512, 1024), respectively. During training, the input size of silhouettes is 64 × 44, and silhouettes are ordered in a sequence with a length of 30. The optimizer is the Stochastic Gradient Descent (SGD). The weight decay is 0.0005 and the momentum is 0.9. We adjust the learning rate, batch size, and the number of iterations to fit different dataset scales. 1) On Gait3D, we train the model for 120k iterations with a batch size of 32 × 4 (32 pedestrians, each containing 4 sequences). The learning rate starts at 0.1 and is subsequently decreased by a factor of 0.1 at iterations (40k, 80k, 100k). 2) On GREW, the model is trained for 180k iterations with a batch size of 32 × 4. The learning rate starts at 0.05 and is subsequently decreased by a factor of 0.2 at iterations (60k, 120k, 150k).Specifically, we train the models in GaitBase [11] and DGaitV2 [10] by ourselves. As shown in Table 1 , the results on the left side of the parentheses represent our results, while the results inside the parentheses originate from the original paper [10, 11] .We compare our approach with other recent appearance-based methods using silhouette sequence as input on in-the-wild datasets as shown in Table 1 . Specifically, the input to SMPLGait [52] is exclusively comprised of the silhouette. These methods can be divided into two categories based on the network backbone components: one category backbone predominantly consists of convolutional operations, while another category backbone is constituted of both convolutional operations and transformers.For the first category, the receptive field plays an essential role. For the spatial receptive field, as illustrated in Table 1 , despite DGaitV2-2D-B [10] having half the parameters of GaitBase [11] , its spatial receptive field substantially exceeds that of GaitBase, ultimately surpassing GaitBase in accuracy on Gait3D [52] and GREW [54] by 0.2% and 3.2%, respectively. The second consideration is the temporal receptive field. Under equivalent spatial receptive field conditions, DGaitV2-P3D-B outperforms DGaitV2-2D-B by 6.3% and 10.3% Rank-1 accuracy on Gait3D and GREW with only 0.44 MegaBytes parameters increase; a similar trend is observed with DGaitV2-P3D-L and DGaitV2-2D-L. However, no matter for DGaitV2-P3D or DGaitV2-3D, in comparison to sequences extending hundreds of silhouettes, their temporal receptive fields are significantly insufficient, merely extracting limited local temporal information.For the second category, we consider that a global-local temporal receptive field shall be important. As depicted in Figure 1 (a) , gait exhibits a certain cyclical pattern, and this cycle represents a local silhouette sequence within gait sequences. SwinGait-3D [10] utilizes a 3D residual block to encode a preliminary pedestrian representation, which is then fed into a 3D Swinformer [30, 31] Table 2 : Network backbone of the GLGait.Output Size Structure [𝑘 × 𝑘 × 𝑘, 𝑐] × 𝑏(𝑇 , 𝐶, 64, 44)EQUATIONStage-4 GL-3D Block (𝑇 , 8𝐶, 16, 11) block, achieving a window-global temporal receptive field. However, within these blocks, SwinGait-3D cannot obtain a true global one. In contrast, GLGait utilizes GLTM to exhibit a global-local temporal receptive field, thereby more effectively learning the cyclical motions of gait. With similar parameter counts, GLGait surpasses SwinGait-3D in Rank-1 accuracy both on Gait3D and GREW by 1.6% and 0.4%. Moreover, we conduct an extended experiment to further explore the performance of GLGait across varying sequence lengths. The results are shown in Table 3 , where the length distribution is illustrated in Figure 1 (b). GLGait-L outperforms DGaitV2-P3D-L 5.1% and 10.7% Rank-1 accuracy at lengths 301 to 400 and 401 to 500, respectively. This demonstrates that GLGait is effective in long sequences. Besides, we also observe that GLGait improves 2.9% Rank-1 accuracy at lengths 1 to 100, indicating that GLGait is even effective in short sequences rather than only in long sequences.1 × 3 × 3, 8𝐶 𝐺𝐿𝑇 𝑀, 8𝐶 1 × 3 × 3, 8𝐶 × 1Finally, with the incorporation of CTL, GLGait-H achieves stateof-the-art performance on both Gait3D and GREW, obtaining Rank-1 accuracy of 77.7% and 82.8%, respectively.Effectiveness of Center-Augmented Triplet Loss. To verify the effectiveness of CTL, we conduct ablation experiments on DGaitV2-P3D [10] and GLGait. As shown in Table 4 , CTL improves both DGaitV2-P3D and GLGait compared with conventional triplet loss [18] on Gait3D [52] and GREW [54] , demonstrating its generalizability and effectiveness. Meanwhile, we also compare CTL with center loss [48] (CL) and triplet center loss [17] (TCL) as shown in Table 5 . CTL outperforms them in GLGait-B and GLGait-L. The reason lies in that CL and TCL only focus on the connection between samples and class centers, ignoring the pair of samples to samples. In contrast, CTL considers the pair of both samples to samples and samples to class centers, reducing intra-class distance Table 4 : Performance gain from applying CTL on Gait3D [52] and GREW [54] with Rank-1 accuracy (%).Gait3D GREW [52] with Rank-1 accuracy (%), where TL is triplet loss [18] , CT is center loss [48] , TCL is triplet center loss [17] .Method TL CL TCL CTL Rank-1 GLGait-B √ - - - 73.3 - √ - - 72.8 - - √ - 73.3 - - - √ 73.9 GLGait-L √ - - - 76.6 - √ - - 76.1 - - √ - 76.2 - - - √ 77.6and expanding positive samples. CTL can seamlessly substitute conventional triplet loss [18] , and we substitute it with CTL in subsequent experiments.We exhibit ablation experiments in GLGait to prove the effectiveness of each component. Vision Encoder Size. To explore an appropriate vision encoder size, we conduct ablation studies within a controlled network, where the number of channels and blocks in per stage are fixed. Specifically, we employed the P3D block [16, 36] as the component of the vision encoder. As shown in Table 6 , the model demonstrates optimal performance when S-1 and S-2 are both employed as the vision encoder, at which point the vision encoder is capable of learning an effective preliminary representation of pedestrians. Utilizing only S-1 as the vision encoder fails to obtain a satisfactory preliminary pedestrian representation, diminishing the model's learning efficiency within the GL-3D block. Conversely, incorporating S-1, S-2, and S-3 as the vision encoder does not afford additional space for the GL-3D block, impeding the model's ability to learn an effective global temporal receptive field and consequently degrading model performance. Finally, we employ S-1 and S-2 as the vision encoder.Vision Encoder Component. We also exhibit the component ablation experiments on the vision encoder in Table 7 . When employing P3D block [16, 36] as the component, GLGait obtains a better result with fewer parameters compared with 3D block [16, 36] . The possible reason lies in that our GL-3D block also separates the spatial and temporal dimensions, which is similar to P3D block. Maintaining such a similar structure assists in model training. Meanwhile, for 2D block [16] , although it has fewer parameters, it is unable to process temporal information, which is essential in pedestrian representation, thus the performance significantly drops out. Table 7 : Ablation study of vision encoder components on Gait3D [52] with Rank-1 accuracy (%), and Params (M).ScoreGLGait-B 2D block [16] 72.4 3.52 3D block [16, 36] 73.1 4.12 P3D block [16, 36] 73.9 3.58GLGait-L 2D block [16] 75.2 14.07 3D block [16, 36] 76.4 16.43 P3D block [16, 36] 77.6 14.28Finally, we select P3D block as the component in the vision encoder to obtain a good accuracy and cost trade-off. Effectiveness of PGTA. To verify the effectiveness of Pseudo Global Temporal Self-Attention (PGTA), we compare it with other multi-head self-attention [44] methods, containing Spatio-Temporal MHSA [2] , Factorised self-attention [2] on temporal dimension, and MobileViT [33] self-attention. Specifically, we set patch size to 3 × 4. The results are shown in Table 8 . PGTA reduces half of the parameters compared with Spatio-Temporal MHSA and Factorised self-attention. Meanwhile, we observe that the Rank-1 accuracy of Spatio-Temporal MHSA and Factorised self-attention greatly drops out. The possible reason is that a large information loss occurs between the 3,072 token size (patch size×channels) and 256 channels. Compared with MobileViT self-attention, PGTA improves 1.9% Rank-1 accuracy with fewer FLOPs. Due to the issue of the receptive field lying in the temporal dimension, PGTA only focuses on the temporal dimension and separates the spatial dimension from tokens, thus effectively establishing a good solution.Effectiveness of Temporal Convolution after PGTA. To verify the effectiveness of temporal convolution after PGTA, we compare it with a normal linear operation. As shown in Table 9 , employing temporal convolution improves GLGait-B 1.6% and GLGait-L 1.1% Rank-1 accuracy than a normal linear operation with few parameters increase. Temporal convolution enhances the local receptive field, assisting the model in learning the motion process of gait. Besides, temporal convolution can also aggregate pseudo Table 8 : Ablation study of memory and computation complexity in self-attention [44] on Gait3D [52] with Rank-1 accuracy (%), Params (M), and FLOPs (G). To verify the effectiveness of GLGait in long sequences, we conduct visualization as illustrated in Figure 5 , where the score is model attention in temporal max pooling phase for each silhouette. GLGait can detect dynamic sub-sequences and give them high scores; for static sub-sequences, it selects representative silhouettes to give high scores and assigns low scores to the rest. This demonstrates that GLGait can align various gait patterns in long sequences, thus validating the effectiveness of global-local temporal receptive field. 
conclusion: 
 In this paper, to address the issue of temporal receptive field for gait recognition in the wild, we add the multi-head self-attention (MHSA) before temporal convolution operation in Convolutional Neural Networks (ConvNets), designing a Global-Local Temporal Receptive Field Network (GLGait) to obtain a global-local temporal receptive field. Due to the dimension explosion in MHSA, we propose a Pseudo Global Temporal Self-Attention (PGTA) to reduce the memory and computation complexity. Furthermore, a Center-Augmented Triplet Loss (CTL) is proposed to reduce the intra-class distance and expand the positive samples, seamlessly substituting conventional triplet loss. GLGait can effectively recognize pedestrians with limited memory and computation complexity in wild scenarios, thus this work can be applied to widespread surveillance gait recognition systems. Extensive experiments have been conducted on Gait3D and GREW. The results demonstrate that our approach outperforms state-of-the-art methods on in-the-wild datasets. 
 
 Appendix:  


Decision: All reviewers agree that this paper is well introduced, motivated and described, with enough level of novelty. Although reviewers provided some critiques/suggestions to improve clarity in some parts, all reviewers agree that the significance of the work and the provided evaluation are good for acceptance of the paper.

Reviews: review: paper_summary: paper_summary: This paper focuses on gait recognition applied in the wild instead of in the lab scenarios. According to the challenge of the wild scenarios (especially in the representation of temporal dimension), the authors propose a Global-Local Temporal Receptive Field Network (GLGait) to establish a global-local temporal receptive field. Furthermore, this paper also proposes a Center-Augmented Triplet Loss (CTL) in GLGait to reduce the intra-class distance and expand the positive samples in the training stage.
strength_weakness: strength_weakness: This paper proposes a novel Global-Local Temporal Receptive Field Network (GLGait) to obtain a global-local temporal receptive field for gait recognition in the wild. Besides, the paper proposes a Center-Augmented Triplet Loss (CTL) to assist in model training. Extensive experiments demonstrate the proposed approach obtains the state-of-the-art performance on in-the-wild datasets.
limitations: limitations: 1. This paper mentioned that the previous ConvNets-like methods is significantly insufficient to obtain the temporal receptive field. As we know, some operator in of ConvNets also can enlarge the receptive field to some extent，such as “Dilated Convolution”. Please provide the reason that employ “transformer” instead of “Dilated Convolution”?
2. In the subsection 3.5, the hyper-parameters α and β in formula (22) are utilized to balance the contributions to the total loss. How to determine the two hyper-parameters?
paper_summary: paper_summary: The paper introduces a Global-Local Temporal Receptive Field Network named GLGait, designed to tackle the issue of temporal receptive fields in gait recognition in-the-wilds. By integrating Pseudo Global Temporal Self-Attention (PGTA) with temporal convolution operations, GLGait effectively processes long gait sequences while reducing the model’s memory and computational complexity. Additionally, the paper proposes a novel loss function, the Center-Augmented Triplet Loss (CTL), to decrease intra-class distances and increase positive samples during the training phase. Extensive experiments demonstrate that GLGait achieves state-of-the-art performance on in-the-wild datasets.
strength_weakness: strength_weakness: a. This paper achieved state-of-the-art performance on the in-the-wild datasets Gait3D and GREW.
b. This paper demonstrates how to reduce the model’s complexity while maintaining technical accuracy by introducing PGTA and CTL. 
c.This paper articulates the methodology, experimental design, and results analysis with clarity.
limitations: limitations: a. Although GLGait introduces the concept of a global-local temporal receptive field and PGTA, these ideas may overlap with existing research. The integration of global and local features has been extensively explored in the field of deep learning.
b. This paper may not provide sufficient details to fully understand the implementation of PGTA and CTL, nor does it mention any intention to publish the code.
c. Robust gait recognition methods need to maintain accuracy and stability over time, and the paper does not discuss the model’s performance over time spans.
paper_summary: paper_summary: This paper is dedicated to the challenge of gait recognition in the wild. The authors propose the Global-Local Temporal Receptive Field Network (GLGait) to obtain more robust temporal information from gait sequences. The PGTA module is developed to alleviate computational complexity and reduce memory requirements. The experimental results show the effectiveness of the proposed method.
strength_weakness: strength_weakness: 1)	The PGTA module is designed to reduce the high memory cost and computation complexity of multi-head self-attention.
2)	The conducted experiments are comprehensive and demonstrate the effectiveness of the proposed method.
limitations: limitations: 1)	The title of the paper is about global-local temporal receptive field Network, but the method section lacks relevant descriptions of this content.
2)	The complexity analysis in the method section is too much, so it is suggested to trim it appropriately, and the related content can be moved to supplementary materials.
3)	In Figure 2 (a), both P3D and GL-3D are depicted twice, please note the difference in the figure and caption.
4)	In Figure 2, please indicate the meaning of the parameter p, and indicate whether the value of p is exactly the same in different places.
5)	Lines 327 and 336, C, H, and W should be distinguished by different symbols, such as lowercase letters.
6)	There is a lack of correspondence between Formulas 5-10 and Figure 3, which hampers reader comprehension. It would be beneficial to clearly mark corresponding variables such as $X_{in}$, $X_{i}$, etc., in Figure 3.
paper_summary: paper_summary: The paper is focused on the task of gait recognition in the wild, which aims to identify pedestrians by analyzing their walking patterns. The main idea of this paper is to solve the limited temporal receptive field of existing appearance-based methods for gait recognition in the wild. Therefore, this paper makes the following contributions: 1) a Global-Local Temporal Receptive Field Network (GLGait) for gait recognition in the wild, 2) the Pseudo Global Temporal Self-Attention (PGTA) to reduce the memory and computation complexity of multi-head self-attention (MHSA), and 3) a Center-Augmented Triplet Loss (CTL) for training GLGait. Experiments on two public datasets show the effectiveness of the proposed method.
strength_weakness: strength_weakness: 1. Novelty: This paper proposes a Global-Local Temporal Receptive Field Network with the Pseudo Global Temporal Self-Attention modules. This approach is novel in the community of gait recognition.
2. Technical correctness: The proposed Pseudo Global Temporal Self-Attention and Center-Augmented Triplet Loss are technically correct.
3. Clarity: The proposed GLGait network is presented in a clear and concise manner, with a detailed explanation of the components such as the Pseudo Global Temporal Self-Attention and Center-Augmented Triplet Loss.
4. Applications: The proposed GLGait network has potential applications in various gait-related domains such as surveillance and assistive technology for elderly people.

Overall, this paper presents a novel approach for gait recognition in the wild, which is based on a clear and concise approach and has been demonstrated to be effective through experiments on two in-the-wild datasets. The paper also provides a comprehensive comparison with other state-of-the-art methods for gait recognition in the wild, making it a valuable contribution to the field.
limitations: limitations: 1. Basically, this paper considers unimodal data in the gait recognition task and is less suitable for the conference.
2. The technical novelty of the Center-Augmented Triplet Loss is incremental to the existing triplet-like loss.
3. The experimental results are somewhat insufficient. This paper only provides quantitative results on the two datasets but no qualitative or visualized results to intuitively validate the claim of this paper, i.e., the proposed method can solve the problem of long-term temporal gait sequence modeling. I suggest the authors provide some examples as shown in Figure 1 to reflect that the propose method can capture the important silhouettes in a long sequence.