Title: Automatic Synthetic Data and Fine-grained Adaptive Feature Alignment for Composed Person Retrieval
Venue: NeurIPS 2025
Accepted: yes
Keywords: Composed Person Retrieval

Synthetic Data Generation

Fine-grained Feature Alignment

Multimodal Retrieval
Abstract: Person retrieval has attracted rising attention. Existing methods are mainly divided into two retrieval modes, namely image-only and text-only. However, they are unable to make full use of the available information and are difficult to meet diverse application requirements. To address the above limitations, we propose a new Composed Person Retrieval (CPR) task, which combines visual and textual queries to identify individuals of interest from large-scale person image databases. Nevertheless, the foremost difficulty of the CPR task is the lack of available annotated datasets. Therefore, we first introduce a scalable automatic data synthesis pipeline, which decomposes complex multimodal data generation into the creation of textual quadruples followed by identity-consistent image synthesis using fine-tuned generative models. Meanwhile, a multimodal filtering method is designed to ensure the resulting SynCPR dataset retains 1.15 million high-quality and fully synthetic triplets. Additionally, to improve the representation of composed person queries, we propose a novel Fine-grained Adaptive Feature Alignment (FAFA) framework through fine-grained dynamic alignment and masked feature reasoning. Moreover, for objective evaluation, we manually annotate the Image-Text Composed Person Retrieval (ITCPR) test set. The extensive experiments demonstrate the effectiveness of the SynCPR dataset and the superiority of the proposed FAFA framework when compared with the state-of-the-art methods. All code and data will be provided at https://github.com/Delong-liu-bupt/Composed_Person_Retrieval.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper focuses on the topic of person retrieval. The author proposes **a new task** different from pure image retrieval or text retrieval, named composed person retrieval. In simple terms, this task uses an image and a text sentence describing changed clothing content as a combined query to search for images in the gallery, which is essentially clothing-changing REID. As a contribution to this new task, the author constructs **a pipeline for large-scale training data automatic synthesis**, thereby obtaining the SynCPR training dataset. The author also proposes **a new training framework** conducive to fine-grained alignment, named FIFA. In addition, the author constructs **a manually annotated test set** named ITCPR to more comprehensively and fairly evaluate the model's performance. The overall workload of this paper is relatively substantial.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. This paper proposes a new task named Composed Person Retrieval. The task uses both images and text as queries to search for images in the gallery. Although this task has been proposed in the general domain (composed image retrieval), such in-depth task definition and exploration of solutions are the first in the person retrieval field.\n2. This paper makes significant overall contributions, proposing a new task, a new training data synthesis strategy, a new training framework, and a new real-world test set.  \n3. The paper is well-written, fluent and easy to understand, with clear explanations of details, making it easy for readers to follow.\n\nWeaknesses:\n1. The experiments are somewhat insufficient. The authors mainly conducted two types of experiments: one is the performance comparison with previous methods, and the other is the ablation experiments to verify the effectiveness of each proposed component. In fact, I believe the focus of the experiments should lie in verifying the rationality and uniqueness of this new task. The authors should have a separate section for comprehensive analysis and experimental verification of this new task. For instance, what are the essential differences between this task and existing person retrieval tasks or traditional composed image retrieval? How much improvement can image-text combination queries bring? Etc. Conducting some qualitative and quantitative experiments to verify the practical application rationality of this task is, in my view, the most core and important part.\n\n2. The research object of this paper is persons, and it aims to utilize their biological characteristics for large-scale searches, which inevitably raises significant privacy concerns and negative social impacts. However, instead of discussing these issues or proposing reasonable control strategies against technical abuse, the authors arbitrarily claim that this research \"does not pose any obvious negative societal impacts.\"\n\n3. This paper seems to lack comparisons with some related works. As far as I know, in the field of person re-identification, there have been several studies on image-text combined retrieval, but the authors do not discuss or compare these methods. For example, relevant papers such as InstructReID and UNIReID.\n\nQUESTIONS:\nHere are some questions that interest me, and I would be grateful if the authors could provide answers.\n\n1. How is the identity consistency ensured in your data generation process? Limited by the capabilities of existing text-to-image models, can the method shown in Figure 3 truly ensure ID consistency? Is there any quantitative experiment to prove this?\n\n2. You only used the training set of CUHK-PEDES to fine-tune FLUX, but there are other datasets in the text person reid field, such as ICFG-PEDES and UFine6926. Should you attempt to fine-tune these datasets as well, perhaps to better enhance the real-world generalization ability of your generation model?\n\n3. The experiments in Table 1 seem unconvincing in demonstrating the effectiveness of your FAFA. You only fine-tuned the CIR method using your SynCPR dataset, while other methods are basically cross-domain experiments, making this comparison unfair. Additionally, the CIR method lacks designs for person-relevant aspects such as identity matching, so it is reasonable that your method outperforms CIR. Considering that TPR methods essentially support composed retrieval as well, you should focus on comparing with TPR methods by fine-tuning these TPR methods on your SynCPR dataset first, and then conduct a fair comparison.\n\n4. Conducting cross-domain and cross-dataset experiments can better demonstrate the effectiveness and generalizability of your method, rather than only evaluating its performance on the test set you proposed.\n\nOverall, this paper has made substantial contributions, but there are still some shortcomings. If you can address the questions raised above, I am willing to raise my score.\n\nLIMITATIONS:\nThe authors have barely discussed the limitations, especially the negative social impacts of their work. The authors should seriously and thoroughly discuss the social impacts of this technology in terms of privacy, and propose some control measures to prevent the improper use of this technology.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper explores Composed Person Retrieval (CPR), a new task that utilizes a compositional query, comprising a reference image and a textual description of modifications, to retrieve (re-identify) a target person. \n\nThe paper presents two main contributions: the construction of the SynCPR dataset and the Fine-grained Adaptive Feature Alignment (FAFA) framework. \nTo generate this high-quality synthetic dataset, a novel data synthesis pipeline is proposed, which uses a Large Language Model (Qwen2.5-70B) to generate textual quadruples, a fine-tuned generative model (Flux.1) to create identity-consistent images, and a Multimodal Large Language Model (Qwen2.5-VL) to filter for high-quality data. \nBased on this dataset, the FAFA framework is designed to improve fine-grained retrieval accuracy using three key components: Fine-grained Dynamic Alignment (FDA), Feature Diversity (FD) supervision, and bidirectional Masked Feature Reasoning (MFR). \n\nExperimental results on a newly annotated benchmark, ITCPR, confirm the effectiveness of both the SynCPR dataset and the FAFA framework when compared to existing methods.\n\nSTRENGTHS AND WEAKNESSES:\nOverall, I think the paper presented sufficient contributions, but I have some conerns.\n\n**Strengths**\n1. The synthetic dataset generation pipeline is well-designed and its effectiveness in CPR tasks has been experimentally verified in Table 1. Specifically, the process of generating consistent identity image pairs for a fine-grained person retrieval task is novel and effective. Also, the open-sourcing of related codes and datasets is appreciated.\n\n2. The ITCPR benchmark for the CPR task is also valuable, as its construction involved substantial manual annotation effort and noise-handling processes.\n\n3. The training pipeline and intuition of the FAFA framework seem reasonable, and it demonstrated strong performance in the experiments.\n\n**Weaknesses**\n\n1. As stated in L219-L222, the proposed framework seems to use multiple feature vectors to represent database images. More specifically, it uses a feature similarity defined in Eq(3) during inference. However, to my knowledge, most existing CIR approaches utilize a single feature vector during the evaluation stage. This implies that the proposed method requires more storage space and computational cost during the search stage. Please justify this trade-off.\n\n2. The usefulness of the proposed CPR task is questionable. General composed image retrieval has clear usage examples, such as product searches that reflect a user's intention on an e-commerce platform. However, what about composed person retrieval? Given that a primary application of person retrieval is in surveillance systems, it is questionable how useful CPR would be. I don't think this issue renders the paper's academic contributions meaningless, but its justification is a significant issue.\n\n3. Based on Table 2, the effectiveness of Feature Diversity (FD) and Masked Feature Reasoning (MFR) seems marginal compared to that of FDA. It would be better to have additional intuition and analysis on these auxiliary losses. The current manuscript primarily discusses the numerical results. The paper would be strengthened by further clarifying the behavior and intuition of these losses through additional qualitative analysis or comparison.\n\nQUESTIONS:\nBased on Weaknesses that I mentioned above, I'd like to suggest:\n\n1. Please justify using more fine-grained similarity (Eq.(3)) for evaluation. Compared to using the average pooled target image feature $\\bar{f_t}$ and other existing methods, how much more computational cost does ${f^i_t}^{N_t}_{i=1}$ require, and what is the performance difference?\n\n2. The justification for the practical usefulness of the proposed CPR task should be strengthened. The paper would be more compelling with a discussion of more diverse and concrete application scenarios to better motivate the task's significance.\n\n3. Please provide more comprehensive intuitions and additional analysis behind Feature Diversity (FD) and Masked Feature Reasoning (MFR).\n\nLIMITATIONS:\nI didn't see specific sections or sentences related to this and I hope this part will be further revised.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a novel Composed Person Retrieval (CPR) task, which leverages both a reference image and a textual relative caption to retrieve a target person image from a large-scale gallery. To support this task, the authors propose a scalable pipeline for automatic triplet synthesis, utilizing large language models (LLMs) and a fine-tuned generative model. Based on this pipeline, they construct SynCPR, a fully synthetic CPR dataset at million-scale. Furthermore, they present a Fine-grained Adaptive Feature Alignment (FAFA) framework tailored for the CPR task. To enable objective evaluation on real-world data, the authors also annotate existing clothes-changing person retrieval datasets with CPR-specific labels.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: \n1. Quality: ① The pipeline for constructing SynCPR is technically sound, with identity consistency achieved via innovative single-image pair synthesis and realistic images generated by a model fine-tuned specifically for this task. ② The experiments are comprehensive, including extensive ablation studies and comparisons with existing methods across multiple settings. ③ ITCPR provides rigorous evaluation on real clothes-changing datasets with false-negative mitigation, making the assessment more convincing.\n2. Clarity: The paper demonstrates clear writing and a coherent structure, with a fairly comprehensive description of the experimental implementation details.\n3. Significance: ① CPR complements existing retrieval tasks by combining text and image queries, demonstrating promising applicability in real-world scenarios. ② This work introduces two novel datasets: SynCPR, which eliminates the need for costly manual annotation, and ITCPR, which is derived from existing clothes-changing person retrieval datasets with additional annotations.\n4. Originality: ① First formalization of CPR, bridging IPR/TPR/CIR gaps. ② A fine-tuned generative model for images that closely resemble real-world persons generation.\n\nWeaknesses: \n1. Quality: Experiments focus on clothing changes (ITCPR) while performance on other variations (e.g., scene) is unexplored\n2. Clarity: The  icons in some figures, such as Figure 1 (b) and 3 are not distinguishable.\n3. Significance: ITCPR primarily focuses on clothing changes and lacks coverage of other types of variations.\n4. Originality: The distinction between CPR and CIR could be articulated more clearly.\n\nQUESTIONS:\n1. Regarding the triplet synthesis pipeline, the authors fine-tune DiT using different LoRA strength values β to introduce style diversity, but cause the images in SynCPR exhibit varying degrees of realism depending on β. However, it seems to lack experiments to verify whether using multiple β values leads to better performance compared to a single fixed β and examine whether the inclusion of less realistic images has any negative impact on model performance. Could you clarify this point？\n\n2. Among the four settings used for comparison with existing methods, the pretraining data varies across different settings, and some baselines were not trained for the text+image (T+I) task—particularly in the first two settings. Under such circumstances, can these comparisons reliably support the claimed superiority of the FAFA approach?\n\n3. In the FAFA framework, why are the fine-grained features derived from the target image rather than the reference image?\n\nLIMITATIONS:\n1. CPR appears to require both text and image queries simultaneously, which may limit its applicability in scenarios where only one modality is available.\n\n2. ITCPR focuses mainly on clothing change, thus leading to limited evaluation of other potential variations.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a novel task, Composed Person Retrieval, which integrates visual and textual information to enhance person retrieval accuracy.\n\nSTRENGTHS AND WEAKNESSES:\n- One weakness is the heavy reliance on the synthetic SynCPR dataset for training. While the generation process is impressive, there is no in-depth analysis of the domain gap between the synthetic training data and the real-world test data (ITCPR).\n- CPR task address the practical need to combine visual and textual queries for person retrieval, which is underexplored compared to image-based or text-based retrieval.\n- The paper is well-written and clearly structured, and The figures are effective at conveying the core ideas. The experiments and results are presented with clarity.\n- The proposed automatic data synthesis pipeline is scalable and innovative, generating a large-scale synthetic dataset to mitigate the lack of annotated data.\n- The originality of the FAFA framework is somewhat tempered by its reliance on existing architectures (e.g., BLIP-2), with incremental improvements over previous CIR methods.\n- Manually annotated ITCPR test set, containing 2,225 triplets, is relatively small, which might not be sufficient to draw robust conclusions about the generalization capabilities of the proposed methods.\n\nQUESTIONS:\n- Could the authors provide a detailed analysis of the computational complexity and resource requirements for the triplet synthesis pipeline, particularly regarding LLM and diffusion model inference times, to assess its practicality for large-scale use?\n- Can the authors clarify how the FAFA framework handles extreme cases, such as significant occlusions or low-quality reference images, and provide corresponding experimental results to demonstrate robustness?\n- Could the authors discuss the potential for extending the CPR task to include multiple reference images or textual descriptions, as mentioned in the conclusion, and provide preliminary results or insights?\n\nLIMITATIONS:\nPlease find the above section.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}