Title: Vocabulary-Guided Gait Recognition
Venue: NeurIPS 2025
Accepted: yes
Keywords: Gait Recognition

Vocabulary-guided Learning

Vision-Language Models
Abstract: What is a gait? Appearance-based gait networks consider a gait as the human shape and motion information from images. Model-based gait networks treat a gait as the human inherent structure from points. However, the considerations remain vague for humans to comprehend truly. In this work, we introduce a novel paradigm Vocabulary-Guided Gait Recognition, dubbed Gait-World, which attempts to explore gait concepts through human vocabularies with Vision-Language Models (VLMs). Despite VLMs have achieved the remarkable progress in various vision tasks, the cognitive capability regarding gait modalities remains limited. The success element in Gait-World is the proper vocabulary prompt where this paradigm carefully selects gait cycle actions as Vocabulary Base, bridging the gait and vocabulary feature spaces and further promoting human understanding for the gait. How to extract gait features? Although previous gait networks have made significant progress, learning solely from gait modalities on limited gait databases makes it difficult to learn robust gait features for practicality. Therefore, we propose the first Gait-World model, dubbed $\alpha$-Gait, which guides the gait network learning with universal vocabulary knowledge from VLMs. However, due to the heterogeneity of the modalities, directly integrating vocabulary and gait features is highly challenging as they reside in different embedding spaces. To address the issues, $\alpha$-Gait designs Vocabulary Relation Mapper and Gait Fine-grained Detector to map and establish vocabulary relations in the gait space for detecting corresponding gait features. Extensive experiments on CASIA-B, CCPG, SUSTech1K, Gait3D and GREW reveal the potential value and research directions of vocabulary information from VLMs in the gait field.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes Gait-World, a new paradigm for gait recognition that uses human-interpretable vocabulary and Vision-Language Models to guide feature learning. And proposed α-Gait, which bridges text and gait modalities via a Vocabulary Relation Mapper and Fine-grained Detector. Experiments on multiple datasets show improved robustness to clothing changes and occlusions, demonstrating how semantic vocabulary can enhance gait recognition while maintaining efficiency.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n(1) The illustrations in this paper are beautiful and can convey the meaning of the text.\n\n(2) The motivation of this paper is significant. It is quite convincing to utilize the universal features of VLMs to assist gait recognition.\n\n(3) The writing of this article is easy to understand, and the language is fluent.\n\n(4) The experimental results were extremely impressive.\n\nWeaknesses:  Please refer to Limitations.\n\nQUESTIONS:\n(1) In the problem description on lines 34-36, I think \"Ambiguous gait concepts\" would be a very appealing point, but the author didn't explain it clearly.  The same goes for the analysis in the abstract.\n\n(2) The paper claims that VLMs are insensitive to gait silhouettes, but only the detection results of GPT are shown in Figure 2. Can the detection results of other VLMs be provided to support this view? And can all VLMs identify the gait cycle from the silhouettes?\n\n(3) The predefined gait cycle actions appear biomechanically inspired. What empirical evidence supports that these 8 phases are optimal? Could a data-driven vocabulary discovery approach (e.g., clustering VLM embeddings) yield more discriminative terms? How does performance vary with vocabulary size/granularity?\n\n(4) In line 195,  the author points out that \"Horizontal Partitioning preserves temporal dimensions\", but VRM/GFD processes each frame independently.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-06-09"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces vocabulary to the gait recognition, which is a probably new recognition paradigm besides conventional appearance and model-based approaches. Specifically, a cross-attention based architecture, which is referred as α-Gait, is proposed for guiding gait feature learning with vocabulary cues.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The motivation is clear. Leveraging certain vocabulary bases to explore the potential integration of LLM in gait analysis is worthwhile.\n2. The design of vocabulary base is reasonable with respect to the gait cycle.\n\nWeaknesses:\n1. The proposed α-Gait lacks of novelty. Although effective, its main components, including Vocabulary Relation Mapper and Gait Fine-grained Detector, are actually existing cross-attention architectures.\n2. Vocabulary base lacks of modality-generalizability. The paper repeatedly claims that a-gait can complement existing paradigms such as in lines 237. However, the experiments only present results based on pedestrian silhouettes. The authors are encouraged to validate the generalizability of α-Gait by applying it to other modality like skeleton-maps or point clouds.\n3. Computational resource cost. 1) α-Gait introduces additional resource cost. 2) In Table 5, the block numbers of α-Gait are [2, 4, 4, 2] with channels [64, 128, 256, 512], which is larger than most compared methods. Therefore, comparisons of parameters size and FLOPs are recommended.\n4. It is better to make code publicly available.\n\nQUESTIONS:\n1. α-Gait integrates both components into a unified module. Treating them as separate and simple components diminishes the overall novelty of the method.\n2. Many but redundant datasets. Authors can remove some simple datasets such as CASIA-B [1] and add more modality-generalizability experiments.\n\n[1] Shiqi Yu, Daoliang Tan, and Tieniu Tan. A framework for evaluating the effect of view angle, clothing and carrying condition on gait recognition. In 18th International Conference on Pattern Recognition (ICPR’06), volume 4, pages 441–444. IEEE, 2006.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a novel paradigm called Gait-World, which aims to improve gait recognition by integrating human-defined vocabulary with Vision-Language Models (VLMs). Existing appearance-based and model-based gait recognition methods suffer from limitations in interpretability and generalization. This work proposes to bridge that gap by leveraging semantic vocabulary related to human gait to guide gait feature learning by VLMs. The proposed method was evaluated on CASIA-B, CCPG, SUSTech1K, Gait3D and GREW.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n(1) This paper introduces a new framework that uses human-understandable vocabulary (gait cycle phases) to guide gait representation learning.\n(2) By leveraging a text encoder, the semantic information may help the gait encoder learn better features. \n(3) Alpha-gait achieves strong results on benchmark datasets\n(4) The paper includes adequate ablation studies of the effect of vocabulary guidance, VLM type, and network components.\n\nWeaknesses:\n(1) The predefined vocabulary base only contain eight terms, which may not capture various gait patterns.\n(2) Despite framing the work as vocabulary-guided, all inputs share the same fixed set of textual prompts.\n(3) The encoder such as CLIP was not pretrained on gait-related data such as eilhouette, which may not align well with the words in the vocabulary.\n(4) There is no verification of semantic alignment of gait features and texts\n\nQUESTIONS:\n(1) Gait recognition requires fine-grained details, why the general phases text can help the gait recognition?\n(2) Gait of different people have same routine (initial contact, loading responses, ...), why the text can lead to better gait recognition performance?\n(3) Is it feasible to extend the gait vocabulary to further boost the performance?\n(4) Can the model take text related to appearance such as clothing, carrying, etc.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper has a new idea. It uses words about walking, like 'initial contact' or 'mid swing', to help computers recognise people from their walk (gait). They call this new method Gait-World. Their model, α-Gait, takes these words from a Vision-Language Model (VLM) and uses them to guide the main gait recognition network. This helps the network focus on the right parts of the walking style. They tested it on many datasets like CASIA-B, CCPG, and GREW, and showed that their model works very well, even better than many old methods.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\nOriginality is very high. The idea to use human vocabulary to guide gait recognition is very new. I have not seen this before. It connects two different fields, language and vision, for this problem.\n\nSignificance is good. This paper opens a new way to do research in gait recognition. Other people can now try using different words or better language models to improve results.\n\nQuality of experiments is strong. The authors tested their model α-Gait on five big datasets (CASIA-B, CCPG, SUSTech1K, Gait3D, GREW). The results in the tables show it is better than many state-of-the-art methods, especially in difficult situations like changing clothes (CCPG dataset). The ablation studies are also good to show why their idea works.\n\nClarity is good. The paper is written well. It is easy to understand the main idea. Figure 3 and Figure 4 are very helpful to see how the model works and what it is learning.\n\nWeaknesses:\n\nThe \n\nVocabulary Base is very small, only eight words for the gait cycle.  The paper does not say if they tried more words or different kinds of words. This feels a bit limited.\n\nThe paper does not talk about the bad side of this technology. Gait recognition can be used for surveillance and tracking people without them knowing. This is a big ethical problem and the authors should have discussed it. In the checklist, they just said Not Applicable for societal impact, which is not right.\nThe model architecture for α-Gait is quite simple. It uses standard blocks like attention. The main new thing is the idea (Gait-World), not a new complex model design.\n\nQUESTIONS:\n1. Your Vocabulary Base has only eight phrases, which are the technical terms for a gait cycle.  Why did you choose only these? Did you try using more simple or descriptive words like \"walking fast\", \"limping\", or \"carrying a bag\"?\n2. α-Gait model uses a Text Encoder which is a big model.  How much does this slow down the recognition time compared to a normal model like GaitBase? Can you give some numbers on inference speed?\n3. Your results on the CCPG dataset are much better than other methods , but on CASIA-B the improvement is not so big.  Can you explain why your method works so well for clothes-changing but is only competitive in normal view-changing scenarios?\n4. Gait recognition is a biometric technology and can be used for surveillance. This has a negative societal impact. Can you please add a discussion about these risks and how they can be managed?\n\nLIMITATIONS:\nNo. The authors have a small section on limitations , but it is only about future technical work, like using more words or making it a multimodal system.  They do not at all discuss the main limitation which is the potential negative societal impact of gait recognition technology (e.g., surveillance). This is a serious miss.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-03"}