PVTNL : Prompting Vision Transformers with Natural Language for Generalizable Person Re-identification (2025)

Abstract: Domain generalization person re-identification (DG-ReID) aims to train models on source domains and generalize to unseen target domains.While patch-based Vision Transformers have achieved success in capturing fine-grained visual features, they often overlook global semantic structure and suffer from feature entanglement, leading to overfitting across domains. Meanwhile, natural language provides high-level semantic abstraction but lacks spatial precision for fine-grained alignment.We propose PVTNL (Prompting Vision Transformers with Natural Language), a novel framework for generalizable person re-identification. PVTNL leverages the pre-trained vision-language model BLIP to extract aligned visual and textual embeddings. Specifically, we utilize body-part cues to segment images into semantically coherent regions and align them with corresponding natural language descriptions. These region-level textual prompts are encoded and injected as soft prompts into the Vision Transformer to guide localized feature learning. Notably, our language module is retained during inference, enabling persistent semantic grounding that enhances cross-domain generalization.Extensive experiments on standard DG-ReID benchmarks demonstrate that PVTNL achieves state-of-the-art performance. Ablation studies further confirm the effectiveness of body-part-level alignment, soft language prompting, and the benefit of preserving language guidance at inference time.

## introduction person re-identification (reid) aims to retrieve individuals with the same identity across a database, where the images are captured under different cameras, timestamps, and spatial locations @xcite @xcite @xcite . as a crucial task in computer vision, reid has been widely applied in video * lei xie is the corresponding author surveillance @xcite and security systems @xcite . with the rapid development of convolutional neural networks (cnns) @xcite , reid has achieved impressive progress @xcite @xcite . however, when the deployment scenario differs significantly from the training domain, performance degrades dramatically due to domain overfitting @xcite . to tackle this challenge, domain generalization person re-identification (dg reid) has been proposed to assess the model's robustness under domain shifts without accessing target domain data during training @xcite @xcite @xcite . existing dg reid methods have explored various directions, including domain-invariant feature learning @xcite , domain disentanglement @xcite , domain alignment @xcite @xcite , and meta-learning @xcite @xcite . while these techniques have shown promising results, they often rely on distribution similarity between domains and may yield unstable generalization performance @xcite @xcite . meanwhile, vision transformers (vits) @xcite have demonstrated stronger robustness to distribution shifts compared to cnns. several works have leveraged vits in dg reid @xcite @xcite , achieving enhanced generalization. however, pure visual models often struggle to distinguish fine-grained semantics under complex conditions such as varying poses or accessories. to address this limitation, we propose pvtnl (prompting vision transformers with natural language), a novel framework that introduces natural language prompts into vits for region-aware feature learning, as shown in fig. 1 . specifically, we first use body-part cues to segment each image into semantically consistent regions and align them with corresponding textual descriptions. these region-level texts are encoded and injected into the vision transformer as soft prompts, guiding the network to attend to meaningful local features. additionally, we employ a cross-attention mechanism to fuse local features, further enhancing the model's representational capacity. unlike prior works, our method retains the natural language module during inference, providing persistent semantic grounding that improves cross-domain generalization. pvtnl achieves state-of-the-art performance on several dg reid benchmarks, offering a new perspective on incorporating language as a generalization prior in vision tasks. the key contributions of this paper are summarized as follows: ## methodology in this section, we introduce pvtnl (prompting vision transformers with natural language), a novel framework designed for person reidentification by integrating local vision prompts with structured natural language. as shown in fig. 2 , pvtnl consists of four main modules: i) image segmentation and language-guided alignment; ii) prompt injection; iii) local features fusion; and iv) the overall training loss design. we describe each component in detail below. ## conclusion in this paper, we propose pvtnl (prompting vision transformers with natural language), a novel framework that integrates natural language prompts into vision transformers for region-aware feature learning in domain generalizable person reidentification (dg reid). we use body-part cues to segment each image into semantically consistent regions and align them with corresponding textual descriptions. we encode textual descriptions into prompts, inject them into the vision transformer for local feature extraction, and employ cross-attention to fuse local features for enhanced representation learning. experimental results across multiple datasets and different evaluation protocols demonstrate that our model achieves state-of-the-art performance in dg-reid. in future work, we will extend pvtnl to cross-modal scenarios (e.g., visible-infrared person reid) and explore its deployment under ethical guidelines to ensure responsible use.