Towards Interactive Chat-based Person Retrieval with Dialogue-aware Evolution
Chat-based person retrieval (ChatPR) aims to identify a target person through progressively evolving multi-turn dialogues. Building on our CVPR 2025 work, Chat-based Person Retrieval via Dialogue-Refined Cross-Modal Alignment (📄 Paper · 💻 Code), DiaNA+ further improves cross-modal alignment under sparse conversational cues and evolving dialogue states.
The implementation of DiaNA+ is available at: 👉 https://github.com/Flame-Chasers/DiaNA-Plus
📖 Overview
DiaNA+ aligns evolving dialogue queries with person images through:
- Adaptive Attribute Refinement (AAR) for aggregating sparse person-related cues across dialogue turns.
- Dialogue-aware Visual Calibration (DVC) for adapting visual representations to the current dialogue state.
- Random Round Retaining (R3) for improving robustness across different interaction stages.
The model uses Swin Transformer V2-B as the image encoder and Llama 3.2-1B as the dialogue encoder.
🤗 Model Zoo
| Checkpoint | Training data | Intended use |
|---|---|---|
pretrain |
MALS | Initialization for downstream finetuning |
ChatPedes-CUHK |
ChatPedes-CUHK | Evaluation on ChatPedes-CUHK |
ChatPedes-ICFG |
ChatPedes-ICFG | Evaluation on ChatPedes-ICFG |
ChatPedes-RSTP |
ChatPedes-RSTP | Evaluation on ChatPedes-RSTP |
🚀 Usage
Clone the implementation and download the model repository:
git clone https://github.com/Flame-Chasers/DiaNA-Plus.git
hf download byougert/DiaNA-Plus --local-dir checkpoints/DiaNA-Plus
Load a finetuned checkpoint:
from diana import load_diana
model, tokenizer = load_diana(
pretrained="checkpoints/DiaNA-Plus/finetune/ChatPedes-CUHK",
device="cuda",
)
model.eval()
To use another checkpoint, replace ChatPedes-CUHK with ChatPedes-ICFG or ChatPedes-RSTP. The pretrained checkpoint is located at checkpoints/DiaNA-Plus/pretrain.
✨ Citation
@InProceedings{bai2025chat,
author = {Bai, Yang and Ji, Yucheng and Cao, Min and Wang, Jinqiao and Ye, Mang},
title = {Chat-based Person Retrieval via Dialogue-Refined Cross-Modal Alignment},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
pages = {3952--3962},
month = {June},
year = {2025}
}
📄 License
This model is released under the MIT License.