FlexiReID: Adaptive Mixture of Expert for Multi-Modal Person Re-Identification
Authors: Zhen Sun
Lei Tan
Yunhang Shen
Chengmao Cai
Xing Sun
Pingyang Dai
Liujuan Cao
Rongrong Ji
Venue: ICML 2025 poster (2025)
Keywords: ReID;MOE;Flexible Retrieval
Primary area: applications->computer_vision
TL;DR: We propose FlexiReID, a multimodal person re-identification framework that supports flexible retrieval across four modalitie.

Abstract: Multimodal person re-identification (Re-ID) aims to match pedestrian images across different modalities. However, most existing methods focus on limited cross-modal settings and fail to support arbitrary query-retrieval combinations, hindering practical deployment. We propose FlexiReID, a flexible framework that supports seven retrieval modes across four modalities: RGB, infrared, sketches, and text. FlexiReID introduces an adaptive mixture-of-experts (MoE) mechanism to dynamically integrate diverse modality features and a cross-modal query fusion module to enhance multimodal feature extraction. To facilitate comprehensive evaluation, we construct CIRS-PEDES, a unified dataset extending four popular Re-ID datasets to include all four modalities. Extensive experiments demonstrate that FlexiReID achieves state-of-the-art performance and offers strong generalization in complex scenarios.

FlexiReID: Adaptive Mixture of Expert for Multi-Modal Person Re-Identification
Zhen Sun
Lei Tan
Yunhang Shen
Chengmao Cai Rongrong Ji
Xing Sun
Pingyang Dai
Liujuan Cao
Abstract
T Text
The Renoooo
Gallery
Multimodal person re-identification (Re-ID) aims to match pedestrian images across different modalities. However, most existing methods fo- cus on limited cross-modal settings and fail to support arbitrary query-retrieval combinations, hindering practical deployment. We propose FlexiRelD, flexible framework that supports seven retrieval modes across four modalities: rgb, infrared, sketches, and text. FlexiReID introduces udaptive ffltture-of-expertt (MoE) mechanism to dynamically integrate diverse modality features and cross-modal query fusion module to en- hance multimodal feature extraction. To facilitate comprehensive evaluation, we construct CIRS- PEDES, a unified dataset extending four popular Re-ID datasets to include all four modalities. Ex- tensive experiments demonstrate that FlexiReID achieves saate-of-the-an performance and offers strong generalization in complex scenarios.
dedd com
Z20000
Uncertain
Descriptive Query
(a) Existing Methods
Gallery
FlexiRelD
T
Query: combinations
(b) Our Method
Figure Illustration our idea. Existing cross- moosl ReID mod- els primarily address single-modalig retrieval, which limits their practical applicability. In contrast, our method enables flexible retrieval across seven different modality combinations. The green boxes match the query.
1. Introduction
Pedestrian re-identification (ReID) is a critical technology in computer vision, focused on matching individuals across dif- ferent camera viewpoints. This capability is essential for var- ious surveillance and security operations, leading to diverse applications in fields such security, urban management, retail, and law enforcement. RelD tasks are classified into single-modal RelD and cross-modal RelD. Single-modal ReID focuses on retrieval between RGB images, depending on the extraction and matching of visual features. However, encounters several challenges in practical applications,
including variations in lighting, occlusions, differences in viewpoints, and changes in pedestrian poses, all of which can compromise recognition accuracy and robustness. In contrast, cross-modal pedestrian re-identification (ReID) presents significant advantages. Unlike single-modal ap- proaches, cross-modal ReID incorporates various modal- ities, including textual descriptions, infrared images, and sketches. By leveraging multimodal information, cross- modal ReID offers complementary features that sustain high recognition accuracy across diverse environmental condi- tions, such as lighting variations and nighttime scenarios, thereby improving the system's 's robustness and adaptability. For example, in low-light or nighttime conditions, infrared images can provide clearer outlines of pedestrians compared
'qqual contribution Key Laboratory Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China. Xiamen University, 361005, P.R. China. "National Uni- versity of Singapore. 'Tencent YouTu Lab. "Institute of Artificial Intelligence.Xiant Xiamen InniversityXiiamee China. Correspondence to: Pingyang Dai <pydai xmu.edu.en>.
Proceedings of the ndnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

FlexiReID: Adaptive Mixture of Expert for Multi-Modal Person Re-Identification
to RGB images, while textual descriptions and sketches can supply additional semantic and structural information when visual data is incomplete or ambiguous. However, existing cross-modal ReID, as shown in figurela, , has an inher- ent limitation: it typically supports retrieval only between specific pairs of modalities, which restricts its ability to accommodate various combinations of modalities. In real- world scenarios, we often receive information from multiple modalities simultaneously. If retrieval is confined single modality paired with RGB images, the potential of exist- ing multimodal information remains underutilized. This raises a critical question: Can we establish a unified cross- modal person re-identification framework that supports flexible retrieval across any combination of modalities?
effectiveness in complex scenes. Our main contributions are summarized as follows:
We introduce the concept of flexible retrieval in the field of person re-identification for the first time and propose FlexiRelI), which supports flexible retrieval with arbitrary modality combinations, opening up a new research direction.
In FlexiReID, we propose the AEA-MoE mechanism, which dynamically selects different numbers of experts based on the input features. Additionally, we design the CMQF module, which leverages learnable embedding features to compensate for missing modalities and fuse different modality features.
Based on this, we propose the FlexiReID frame- work, which spans four modalities (Text, Sketch, In- frared, RGB) and supports seven different combinations of multimodal retrieval: TeB, Sketch-to-RGB, Infrared-to-RGBB, Text=Sketch-to-RGi Text+Infrared-to- RGB, Iaffae--n RGB, and Sextt-Skecch-nfrrredd to-RGB, as shown in lb Inspired by CLIP, which is pre-trained on large-scale dataset of 400 million image-text pairs, FlexiReID utilizes a simple dual-encoder architecture for the extraction of visual and textual features. Notably, the three visual modalities share a single image encoder. In order to efficiently extract features of diverse modalities, we introduce an Adaptive Expert Allocation Mixture of Ex- perts (AEA-MoE) mechanism. Specifically, our proposed adaptive routing mechanism dynamically selects a varying number of expert combinations based on the attributes of the input features. Compared to the traditional Top-K routing mechanism, which selects fixed number of experts, our approach better leverages the strengths of the multi-expert system, thereby optimizing the extraction of multi-modal features. Additionally, we designed multi-modal feature fusion module called Cross-Modal Query Fusion (CMQF). This module accepts multi-modal feature inputs and uses learnable embedded features to compensate for missing modalities. Its superior feature fusion capability further en- hances the flexible retrieval performance of FlexiReID. In this study, We introduce the concept of flexible retrieval to the field of person re-identification for the first time, pio- neering a new research direction. The core idea of flexible retrieval is to accurately perform person retrieval using the existing modalities, even in the presence of missing modal- ities. To achieve this challenging objective, We expanded the modalities of the of the four datasets (CUHK-PEDES, ICFG-PEDES, RSTPReid, and SYSU-MMO1), construct- ing a unified dataset named CIRS- PEDES, which encom- passes four modalities: text, sketches, RGB images, and infrared images. Experimental results on the CIRS-PEDES show that the proposed FlexiRelD outperforms several other state-of-tte ofte-oof-hhe-rt methods, demonstrating its flexibility and
We construct a unified dataset, CIRS-PEDES, which contains four modalities. Extensive experiments demonstrate the effectiveness of FlexiReID.
2. Related Work
2.1. Vision-Language Pre- Pre-Trainig Models
Vision-language pre-training models, pre-trained on image- text corpora, have demonstrated significant potential in downstream vision and language tasks, such as few-shot classification et al., 2022; Gao et al., 2024; Yu et al., 2023), gross-modality generation(Nichol et al., 2021; Ramesh et al., 2022; Patashnik et al., 2021; Niu al., 2025b), and visual recognition( Wang et al., 2021). The pre- training approaches mainly contain the BERT-like masked- language and masked-region modeling methods(L_u et al., 2019; Su et al.,, 2019; Tan & Bansal, 2019; Chen et al., 2020), contrastive learning for learning a joint embedding space of vision and language et al., 2021; Jia et al., 2021; Li et al., 2021: Zhai et al., 2022a; Niu et al., 2025a), and vision-language multimodal autoregressive techniques(Cho et al., 2021; Ramesh et al., 2021). In this paper, we focus on the contrastive vision-language models (VLMs) that adopt a dual encoder to encode images and texts into the joint embedding space and use contrastive learning to align the visual and textual representations. representative work CLIP(Radford et al., 2021), which aggregates 400 million image-tex pairs from websites. It employs a dual-encoder architecture consisting of an image encoder and text en- coder, and showcases remarkable prompt-based zero-shot performance across diverse visual classification tasks by exploiting alignments between text and image features.
2.2. Cross-Modal Person Re-Identification
Person re-identification (ReID) focuses on retrieving all im- ages of a specific pedestrian across different devices, with an emphasis on learning distinctive pedestrian features. Based
2

---

FlexiReID: Adaptive Mixture of Expert for Multi-Modal Person Re-Identification
on the various modalities used to represent pedestrian infor- mation, RelD can be divided into single-modal RelIDCChen et al., 2022e; Ye et al., 2021c) and cross-modal ReID(Chen et al., 2022a; Ye et al., 2021b; Zhu et al., 2021). Cross-modal RelID, in particular, addresses unique situations where RGB images of pedestrians are not readily available. It suggests utilizing non-RGB modalities(such as infrared ihages(Chen al., 2022b; Yang et al., 2022; Ye et al., 2a), text descrip- tions(Ding et al., 2021; Gao al., 2021; Shao et al., 2022), and sketches(Chen et al., 2022a; Gui et al., 2020; Yang et al., 2020)) to represent pedestrian information, thereby broad- ening the application scope of RelD technology. Li et aL. et al., 2017) first propose to explore the problem of retriev- ing the target pedestrians with natural language descriptions for adaptation to real-world circumstances. Shao et al. et al., 2022) analyze the granularity differences between the visual modality and textual modality and propose granularity-unified representation learning method for text- based ReID. Pang et al. (Pang et al., 2018) first propose to use professional sketches as queries to search for the target person in the RGB gallery. They design cross-doman ad- versarial learning methods to mine domain-invarian feature representations. In order to explore the complementarity between the sketch modality and the text modality, Zhai et al.hii et al., 2022b) introduce a multi-modal Rell) task that combines both sketch and text modalities as queries for retrieval. However, existing methods are limited to retrieval between specific pairs of modalities and cannot be extended to support retrieval across various combinations of modali- ties, which fails to meet the demands of real-world scenar- ios. To address this limitation, we propose the FlexiRelI) framework, which spans four modalities and supports seven different combinations for multimodal retrieval.
included in the weighted averaging. In contrast the tra- ditional Top-K mechanism, the Adaptive Expert Activation (AEA-MOE) mechanism proposed in this paper employs an adaptive routing algorithm that dynamically adjusts the number of activated experts based on the complexity of the input data, thereby enabling more efficient utilization of computational resources.
3. Proposed Method
3.1. Framework
The overall framework is illustrated in figure2. The frame- work contains four types of modal data: Irgb. I.. Irr and T, corresponding to RGB, sketch, infrared, and text, respec- tively. We employ CLIP (ViT-B/16) as the backbone of our network. Specifically, the Image encoder is used to extract features from the three image modalities, while the Text encoder is used to extract features from the text modality.
For the Image encoder, the image is first divided into N patches, which are then mapped into embedding vectors through linear projection, with positional information added to enhance spatial awareness. Subsequently, a [CLS] token is introduced at the beginning of the embedding vectors to represent the global features of the image. These N+1 tokens are then fed into series of transformer blocks. In order to efficiently extract features of diverse modalities, the Adaptive Expert Allocation Mixture of Experts(AEi) (EEA- MOE) mechanism is introduced. After processing through the multi-head attention mechanism, an adaptive routing algorithm is employed to dynamically select the experts to be activated based on the confidence level of each expert. Unlike the traditional Top-K routing mechanism that selects fixed number of experts, the adaptive routing algorithm can dynamically adjust the number of activated experts based on the input feature attributes. Additionally, to ensure that the adaptive routing algorithm selects the smallest necessary set of experts, an adaptive loss is introduced.
2.3. Mixture-of-Expertt
Oxtture-of-Eppert. (MoE) has been extensively explored in computer vision (Riquelme et al., 2021), natural language processing(Shaeees et al., 2017), and vision-language pre- training(Chen et al., 2024). MoE learns a series of expert networks and a gating network, where the outputs of the expert networks are weighted by gating scores generated by the gating network before the weighting operation. In more recent works, some researchers(Eiger et al., 2013; Shazeer et al., 2017; Fedus et al., 2022; Lepikhin et al., 2020) use the gating scores as a criterion to sparsely select only one or a few experts. The sparse activation of experts enables a significant reduction of the computational cost when training large-scale models. To further enhance com- putational efficiency, TOP- (Oh-K(Shazeer et al., 2017) sparse gating mechanism is employed, selecting only subset of experts in each layer. This approach enables MoE models to scale linearly in size while maintaining manageable compu- tational requirements, depending on the number of experts
For the Text encoder, the text description T is tokenized using simple tokenizer with vocabulary of 49, 49,152 152 words, converting it into embedding vectors e. A [BOS] token is added at the beginning of the sequence as start token, and an [EOS] token is added at the end as an end token. Thus, the entire sequence can be represented as {Eboer C1, G and is fed into the transformer blocks. The subsequent processing is similar to that of the Image encoder and will not be repeated here.
The final image feature representations are denoted as Ui Uir, and the text feature representations are denoted as (tbos, fi...., ). where is used to indicate the three modalities within the image domain. Here, vas represents the global features of the image, and leos rep- resents the global features of the text. Since FlexiRelD
3

---

FlexiReID: Adaptive Mixture of Expert for Multi-Modal Person Re-Identificatio
SDM Loss
Multi-Head Artention
Aduptive
Patch Embedding
Router
RGB
CMQF
Adapters
TL_s
AEA-MOE
OR
TL_ir
1L_
Aduptive
Multi-Head Attention
Word Embedding
AEA-MOÉ
drewell light-caleral
S+IR
punds.
Adapters
TI_t
bag...
Adaptive
T
Router
Fine-tuned
Frozen
Learnable Embedding Fentures
Extructed Features
Fusion Features
Figure 2. The network architecture of the proposed FlexiReID. All visual modalities share single visual encoder. The incorporation of AEA-MOR facilitates the efficient processing of heterogencous modality data, CMQF is emplayed to scamlessly integrate diverse modality features, while learnable embedding features utilized to compensate for any missing modalities.
supports seven retrieval strategies across four modalities, we propose feature fusion module Cross-Modal Query Fusion(CMQF)), which accepts multi-modal feature input, complements the missing modalities with learnable embed- ding features, and finally outputs seven fused features de- noted as {fs: fir, ft, feair: faa, firs. fa.r..). These seven fused features and visual representation v are fnally in- teracted and calculated by Similarity Distribution Matching (SDM) which is an effective matching loss function across different modalities.
different modalities. Therefore, we propose an adaptive routing mechanism that dynamically selects the number of activated experts extract features from different modali- ties. Specifically, introduce threshold confidence level. After the routing mechanism calculates the confidence level for each expert, it first compares the highest expert confi- dence level with the threshold. If the highest confidence level exceeds the threshold, only the corresponding expert is activated for feature extraction, and no additional experts involved. If the highest confidence level does not exceed the threshold, experts confidence until the threshold is surpassed. This set of ex- perts is then used for feature extraction. The data processing procedure of AEA-MOE can be formally represented as follows:
activated in descending order of
To reduce the number of parameters during model train- ing, we freeze the Patch Embedding, Word Embedding, and Multi-Head Attention components of the pre-trained model, and only train the Adaptive Expert Allocation Mix- ture of Experts(AEA-MORR learnable embedding features and Cross-Modal Query Fusion(CMQF)) modules.
= Softmax(W,, x²))
(I)
3.2. Adaptive Expert Allocation Mixture of Experts (AEA-MOE)
9.(x) = Pr, Adapter; E S
(2)
Adapter $ S
In order to enhance the multimodal feature extraction capa- bility of the model, we introduced the AEA-MOE mecha- nism. The traditional MOE based on Top-K routing calcu- lates the confidence for each expert and activates the top K experts in confidence ranking. We argue that the activation strategy overlooks the diverse characteristics inherent in the input features. Different input features require the activa- tion of different numbers of experts, and a fixed number of experts cannot meet the needs of extracting features from
y/ Σ 9s Adapter; (x)
(3)
where W, represents the weights the gating network within the routing mechanism, and P vector of size n, where P. denotes the probability of selecting the i-th

---

FlexiReID: Adaptive Mixture of Expert for Multi-Modal Person Re-Identification
Case K=2 Threshold-0.65
3.3. Cross-Modal Query Fusion (CMQF)
Top-K Router Number of Experts-2
Aduptive Router 1.7-Threshold I166
To effectively extract multimodal features and achieve flex- ible retrieval with various modality combinations, pro- pose the CMQF module. The features of the combined modalities are input into the CMQF module, and learnable embedded features are used to compensate for any missing modalities. Specifically, the features of each modality are input into their respective transformer blocks, where the query features are the sum of the query features from the other modalities. Formally, this can be represented as:
Adapter1
Adapter2
Adapter3
Adapterl
Case K-2 Threshold-0.65
Top-K Router Number of Experts-2
Adaptive
1.3-0..-0.--Threeeee 0.65
Me = TLox((X, + X, X,WK, X,Wv) Vir TL.r(((X + Q,)
W,, (5)
y: TL, + XXWx,
AdapterI 0.15
Adapter2 0,25
Adapter3
Adapterd
0.3
Where X represents the input modality features, and W denotes the weight parameters used to generate the Query, Key, and Value. TL stands for the transformer block. Sub- sequently, all output features are concatenated and fed into shared transformer block (TL fu). Finally, the fused fea- tures are obtained through mean pooling operation. This process be formally represented follows:
Figure 3. Traditional Kop-K-based routing mechanisms are limited selecting fixed number experts. contrast, our proposed adaptive routing mechanism introduces threshold confidence level, allowing the number of selected experts to be dynamically adjusted based the input data.
Concat Vir, Us) = MeanPool WK (6)
=
expert computed by the gating network.S is the minimal set of experts whose confidence levels exceed the threshold. g: gx(x) indicates the probability of selecting the i-th expert as determined by the adaptive routing mechanism. Equation (3) illustrates the weighted summation process of experts and routing weights. The structure of the Adapter, inspired by (Chen et al., 2022d), consists of a down-sampling layer, an intermediate activation layer, and up-sampling layer.
After processing through the CMQF, we ob- tain seven types of modality- features: (fx, St. Ju.ir, Jirt, correspond to the following modalities: sketch, infrared, text, sketch fused with infrared, sketch fused with text, infrared fused with text, and the fusion of sketch, infrared and text features.
These features
To prevent the model from assigning small weights to each expert, resulting in the need to activate a large number of experts exceed the threshold confidence level, which con- tradicts the principle of MOE to enhance computational efficiency and performance by activating only few experts. We propose adaptive loss constrain the distribution of the probability calculated by the gating function. We in- troduce the concept of entropy for adaptive loss calculation, formally expressed as follows:
OR
TL
Suppin MeP
OR
Concat
Lada Σ Pc)
(4)
OR
TTL
Fasion Features
Extracted Features
Text
Leanable Embedding Feztures
the adaptive loss ensures that the minimum necessary set of experts is activated, thereby enhancing computational efficiency. Simultancously, it guarantees that the sum of the confidences of the activated experts exceeds the threshold confidence, and the activated experts can effectively extract features.
Figure 4. Our proposed CMQF Module.
3.4. Optimization and Inference
During the training phase, we utilize a parameter-Fe loss function called Similarity Distribution Matching
5

---

FlexiReID: Adaptive Mixture of Expert for Multi-Modal Person Re-Identification
(SDM))Jiang & Ye. 2023). Jiang and Ye incorporate the cosine similarity distributions of the N x N embeddings for image-texi pairs into the KL divergence to establish connection between the two modalities. This is formally represented as:
image modalities. This was achieved by training the model on pairs of visible light and corresponding infrared images, and then applying the trained model to each RGB image to produce infrared images that capture thermal radiation information, For the SYSU-MM01 dataset, which originally contains RGB and infrared images, we similarly used Style- GANr(Karras et al., 2021) to generate sketch modalities and employed the GPT-4 model to generate textual descriptions. This involved extracting features from each RGB image and generating corresponding textual descriptions that capture the main content and characteristics of the images. By incor- porating these additional modalities, our expanded datasets better simulate viding richer and more diverse data support for the training of the FlexiReid model. An overview of training and test set partitioning for each dataset can be found in the existing work(Ding et al., 2021; Zhu et al., 2021; Wu et al., 2020).
N
N
C = (L(pillq))
=
1 N
logg
Pi.i 96, +
(7)
1=1
Ludm fiiz + Lezi
(8)
multimodal data scenarios, pro-
where Pi, is the probability denoting the similarity between image-text pairs and Gi.j is the true matching probability. Ladm is the bi-directional SDM loss.
Our model supports seven retrieval methods, each of which employs the SDM loss for its loss calculation. Therefore, the total loss is the aggregate of the SDM losses across all seven retrieval methods. Formally, this is expressed as:
Evaluation Protocols. Following existing cross-modality RelD settings(Chen et al., 2022a; Ye et al., 2021b;c), we use the Rank-k matching accuracy, mean Average Preci- sion (mAP), and mean Inverse Negative Penalty (mINP) et al., 2021c) metrics for performance evaluation in our FlexiRelD.
Sasm
(9)
where Lyam denotes the SDM loss for the i-th retrieval method. Our framework is trained end-to-end, and the over- all optimization objective is formally defined as:
Implementation Details. We employ the Vision Trans- formerDDssvvitskiy,, 2020) as the visual modalities feature learning backbone, and the Transformer (Vasaan, 2017) as the textual modality feature learning backbone. Both backbones have pre-trained parameters derived from CLIP(Radford et al., 2021). During the training process, all parameters of the backhone networks are frozen. In batch, we randomly select 64 identities, each containing sketch, an infrared, a text, and an RGB sample. The image is resized to 384 x 128, and the length of textual token sequence is 77. We train our FlexiRelD model with the Adam optimizer for 60 epochs. And the initial learning rate is computed as lc-5 and decayed by cosine schedule. The threshold confidence level is set 0.6, and the number of experts is 6. The hyperparameter À that indicates the adaptive loss is set to 0.5. We perform experiments on single NVIDIA 3090 24GB GPU.
C= XLado
(10)
During inference, the trained network processes combina- tions of various modalities, assigning learnable Embedding Features to any missing modalities. It subsequently extracts integrated features across these modalities and computes their similarity with the RGB image embeddings. The Top- K candidates are then processed to derive the relevant evalu- ation metrics for each query.
4. Experiments
4.1. Experimental Setup
Datasets. In this study, we introduced four datasets: CUHK- PEDES, ICFG-PEDS, RSTPReid, and SYSU-MM01. To support the flexible retrieval capabilities of FlexiReid, which accommodates four modalities of pedestrian data, we ex- panded the modalities of these datasets. For the CUHK- PEDES, ICFG-PEDS, and RSTPReid datasets, which origi- nally contain RGB images and textual descriptions, we em- ployed StyleGAN3 et al., 2021) to generate sketch modalities. This involved preprocessing each RGB image and using a pre-trained StyleGAN3 model to convert it into a sketch, preserving the primary contours and structural information of the original image. Concurrently, we used InfraGAAN(Okkangllu & Ozer, 2022) to generate infrared
4.2. Performance Comparison
Results on CUHK-PEDES, ICFG-PEDES and RST- PReid In our experiments, we evaluated the model's perfor- mance across seven different test tasks: T - R, S - R, IR + R, - R, T + IR + R, S + I - R. and S+ IR + The results, as shown in table 1, demon- strate the model's performance on the CHUK-PEDES, ICFG-PEDES, and RSTPReid datasets. In the T + R task, our model achieved a Rank accuracy of 69.20% at R@ 1, 86.43% at R@5, and 91.41% at R@ 10 on the CHUK- PEDES dataset. On the ICFG-PEDES dataset, the model
6

---

FlexiReID: Adaptive Mixture of Expert for Multi-Modal Person Re-Identification
Table Comparison with the sfate-of-the ahts on CUHK-PEDES, ICFG-PEDES, and RSTPReid datasets. Rank (It) k accuracy (%)
reported. The best results are bold.
CURK-PEDES
ICFG-PEDES
RSTPReid
Tasks
Methods
Venne
R1
R55
R10
mAP
mINP
RS
R10
mAP
mINP
RI
RS
RIO
MAP
mINP
CMPMICIZZang La, 2018)
BCCVIS
49.37
71.99
79:27
43.51
65.44
74,25
MIA/Niu et 2020)
2020)
TIP20
46.89
67.14
75.18
VITAA/Wang 2020)
BCCV20
55.97
75.84
83.52
50.98
68.79
75.78
NAPSIGao at.., 2021)
arXiv21
59.36
79.13
80.00
54.07
DSSL,2hu al. 2021)
MM21
59.98
30:41
87.56
32.43
08
63.19
SSANiDing 2021)
arXiv21
61.37
80.15
54.23
72.63
79.53
43.50
Han et al. et 2021)
BMVC21
64.08
81.73
88.19
60.08
al. 2022b)
MM22
64.04
82.66
87.22
45.55
68.20
77.85
T-R
TIPCB/Chen ot al., 20122)
SARLL et al. 2022)
ICASSP22
Nauro22
64.26
64.13
X3.19
89.10
58.61
54.86
54.96
74.72
72.13
81.89
79.13
32.76
44.05
67.30
36.81
CATBC[Wang et al., 2022b)
MM22
64.43
81.87
87.35
47.35
69.55
79.00
AXM FasFoogg et al. 2022)
MM22
64.44
90.52
86.77
LGUR(SSaa et al., 2022)
MM22
65.25
83.12
89.00
59.02
75.32
81.55
SVTiShu al., 2022)
2022)
BCCV22
65.59
83.11
89.21
56,04
73.60
80.22
46.70
70.00
78.30
UNIReID(Chen et 2023a)
CVPR23
68.71
85.35
90.84
61.28
77.40
85.16
60:25
87.
CFine Yan et al., 2023)
TTP23
69.57
85.93
91.15
60.83
76.55
N2.42
50.55
72.55
$1.60
CSKTyLiu al., 2024b)
ICASSP24
69.70
86.92
91.80
62.74
56,90
77.31
83.56
33.87
57.95
81.30
88.35
46.43
RexxRelD(Churs)
69:20
96.43
91.41
62.47
61.34
78.41
83.92
35.73
7.53
55.79
79.62
86.48
45.37
26,25
Sketch Lran+tChenn et at., 2023b/
FAMIZ
81,39
90.61
93.54
73.72
04.72
74.83
80.75
91.52
38.04
5.08
61.37
50.15
58.29
48.94
25.73
S-R
DAL.Net al. 20046)
AAA12024
83.03
92.39
94,58
77.28
87.84
92.61
401.5
6.18
64.00
83.27
51
UNIKelDDCCha et al. 2023a)
CVP923
84.87
78.85
08.55
77.47
40.41
6.31
65.80
51.22
27.47
(exxRelD((hurs)
84.92
93.17
95.02
79.21
66.88
79.28
89.69
93.37
41,21
6.85
66
84.52
90.39
52.72
28.36
T+S-R
UNIRelD(Chen et aL, 2025a)
ResiReID((urs)
CVPR23
86.29
87.47
94.51
96.14
80.92
82.43
72.96
83.82
82.17
93.49
95.63
47.81
47.00
9.74
9.03
73:20
76. 10
80.72
94.31
64:73
58.72
41.24
34.61
GL G/Yng 2023)
ICCV
82.06
91.72
93.95
75.84
06.86
800.31
901.89
92.78
44.30
6.910
73.42
86.29
00:43
IR-R
SDCI/Ying et al., 2024)
CVPR34
84.57
92.73
94.58
77.32
68.30
81,36
91.83
94.07
45.81
7.92
74.67
87.94
93.16 16
62.75
39.95
ResiieeD(Ours)
85.26
93.25
95.31
79.13
69.39
82.03
92.19
94.27
46.76
8.47
75.36
$8.71
93.27
63:22
40.82
T-IR-R
ResiRelD(Ours
86.23
94.07
81.49
92.56
95.05
47.35
8.75
75.84
78
63.90
40.95
S+R -R
(exukelD(Churs)
85.97
93.52
95.89
81.02
69.668
82.57
92.74
95.28
47.45
8.93
75.94
89.47
94.23
64.99
41.18
TiS+1
HexiRelD(Uurs)
88.23
95.13
96.75
82.63
73.05
84.26
93.78
96.15
48.09
9.43
76.35
90.26
95.08
65.19
42.29
recorded a Rank-K K accuracy of 61.34% at R@ 1, 78.41% at R@5, and 83.92% at R @@I6 10. For the RSTPReid dataset, the Rank-K accuracy was 55.79% at R@1 1, 79.62% at R@5, and 86.48% at @@ 10. Notably, the model achieved state-of-the- art (SOTA) performance on the ICFG-PEDES dataset and demonstrated performance close to SOTA on the CHUK- PEDES and RSTPReid datasets. These results underscore the model's robust capability in text-to-image retrieval tasks. In the S-R task, our model achieved MAP metrics of 79.21 % 41.21% %, and 52.72% on the three datasets, re- spectively. These results represent state-of-the (SOTA) performance across all datasets.
modality cross-modal retrieval methods, making our ap- proach more versatile and widely applicable.
Table 2. Comparison with the statc-of-the-arts on SYSU-MM01 datasets. Rank (H) at k accuracy (%) is reported. The best results are bold.
Tucks
Methods
Venue
All-Search
R10 R200
Emloon-Search
SSFTL at. 2000)
CVPR20
61.6
89.2
91.9
97.7
22.6
DDAGYYG al., 2030
POCM30
54.8
90.4 95.8
53.0
68.0
DXI-VAF(h ctal, 2020)
MM30
90.5
91.8
58.5
CICL- 2021)
AAAI21
57.2
94.3
98.4
59.3
99.7
74.7
IR-R
VCD+VML 202))
CVP921
60.0
94.2
98.1
58.8
56.1
96.6
99.4
73.0
MiANNtWW al.
CVPR21
70.6
06.2 98.8
68
98
99
61.0
MCT NmmHoo 20.2000)
10CV3
91.3 97.1
62.0
97.0
06.6
SMCL/Wel at 20211
I0CV21
67.4 91.9 96.8
6188
58.5
96.6
98.8
73.5
In other tasks (T + S R,T T IR S IR + R T STTR R), our model also demonstrated robust per- formance. Compared to single-modality retrieval methods, the flexible combination of different modalities consistently achieved superior results. For instance, in the T+S+IR- task, our model achieved Rank-K metrics of 88.23% at R@ @@ 95.115.13 13%% at R@5, and 96.75% at R@ 10 on the CUHK- PEDES dataset, surpassing singlc-modality retrieval meth- ods. This indicates that combining various modalities pro- vides richer pedestrian detail information, thereby enhanc- ing overall retrieval performance
(letike(D(Oun))
57.9 93.4 97.5
62.5
59.2
91.2
99.1
73.8
T-R
UNIRalIXChen alll. 2023a)
CVP923
56.8 92.7
54.3 90.2
96.3
95.7
65.4
63.
54.2
56.7
93.3
91.3
96.5
97.4
67.6
66.9
S-iR
UNIKalEXC al, 20230l
FleuiRe/D/Oum))
CVPR25
64.7 90.9
66.4
92.7
54..5
95.2
60.3
39.2
16.7
68.5
06.7
95.4
97.5
98.2
75.3
74.00
T45-
UNIKalXX ct 20230)
CUPR25
65.7
66.3
54.2
95.1
07.9
98.5
67.2
66.
59.3
70,6
95.0
04.6
07.5
97.9
73.4
12.8
T-RR-R
(eem)
68.9
96.7
98.2
68.
71.2
08
99
25.2
SFik -R
69.0
98.
98.5
05.6
722
98.7
99.1
76.2
TrS-IK-1 -R
71.
97.2
98.7
693
73.3
98.9
99.6
77.3
Results on SYSU-MM01 We also evaluated our model on the SYSU-MM01 dataset, as shown in table 2. In the IR + RGB task under the full search mode, our model demon- strated competitive performance, approaching the state-of- the-art (SOTA) level. However, by flexibly combining other modalities, performance can be further enhanced. For in- stance, in the Seatrch-eearhhhaco mode for the T S IR - R task, our model achieved Rank-K K metrics of 73.2% on R@1, 1, 98.9% on R@ 10, and 99.6% on R@20 20. This improvement is attributed to the introduction of text and sketch modalities, which provide additional pedestrian features and enhance the model's 's comprehension capabilities.
Overall, most current cross- modal ReID methods focus on retrieving RGB modality from a non-RGB modality. Our approach not only achieves state-of-the-ar (SOTA) or near- SOTA performance in single-modality retrieval tasks but also supports broader range of retrieval methods. The flex- ible combination of various modalities outperforms single-
7

---

FlexiReID: Adaptive Mixture of Expert for Multi-Modal Person Re-Identification
Table Ablation study on R@ about each component of FlexiRelDD, The metric Avg. denotes the average @ across seven search modes.
Modille
M.P-Alligeer
Components AEA-MOE AL.
CMQF LEF
Arg.
Zero shot CLIP
12.58
244
3.79
1,02
2.73
3.72
4,51
4,49
MMP--Adgger
66.38
83.58
82,43
$5.00
80.58
+AEA MDBrwlo AL)
58.45
83.08
83.74
85.83
35.52
84.32
86.71
82.52
LAEA-MOEIW/ AL)
68.87
84.13
84.57
86.41
85.93
84.97
87.35
83.14
-CMQFFwwa LEE)
87.38
+CMQRww LEF)
89:20
84.92
85,26
87.47
86.23
85.97
88.23
83.90
4.3. Ablation Study
Fusion (CMQF) module, we conducted an ablation study comparing with several commonly used multimodal fea- ture fusion strategies, including concatenation, summation, and hierarchical fusion. Specifically, the concatenation strat- egy directly concatenates features from different modali- ties and feeds them into shared Transformer for fusion. The summation strategy aggregates features by summing them prior to Transformer processing. The hierarchical fu- sion approach first encodes each modality independently using separate Transformers and then performs fusion at shared layer. All methods were evaluated on the modality- augmented CUHK-PEDES dataset, and the average R @ ac- curacy across seven retrieval tasks was reported. As shown in Table 5, CMQF achieved the best overall performance. Compared to the aforementioned methods, CMQF lever- ages cross-modal attention mechanisms and incorporates learnable embedding features compensate for missing modalities, enabling more comprehensive and robust feature alignment. These results demonstrate the superior capability of CMQF in enhancing multimodal feature representations and improving retrieval accuracy.
A comprehensive ablation study for components of FlexiReID presented in Table 3, including the most critical accuracy metric R@1 and the average metric CUHK- PEDES datasets. The results in No.0 serve the back- bone baseline by zero-shot CLIP, where inference is per- formed directly on the original frozen CLIP model without adding any additional trainable modules. No.1 employs the traditional MOE method, while No.2 and No.3 utilize AEA-MOE. It is evident that AEA-MOE outperforms the traditional MOE in handling data from various modalities, thereby achieving superior performance. Furthermore, in- corporating adaptive loss further enhances performance, As demonstrated by No.4, CMQF plays pivotal role in the integration of features from different modalities, thereby improving the model's performance. Moreover, substitut- ing missing modalities with Learnable Embedding Features yields optimal performance. These results indicate that each component of FlexiReID significantly contributes the model's isssi overall performance, working in concert achieve optimal outcomes.
Study on Routing Strategies To evaluate the ef- fectiveness of our proposed adaptive routing mechanism, we conduct a comprehensive ablation study by comparing it with several widely adopted routing strategies, including Top-K K routing, soft routing, and hash routing. As shown in Table 4, our adaptive routing achieves the best overall performance. These results demonstrate that our method benefits from dynamically selecting both the number and combination of experts based on the input features, rather than relying on fixed or probabilistic routing strategies. This ablation study highlights the critical role of adaptive expert allocation in enhancing feature expressiveness and improv- ing downstream retrieval performance.
Ablation
Table 5. Comparison of different feature fusion strategies
Method
Avg.
Concatenation
83. 14
Summation
82.24
Hicrarchical Fusion
83.51
CMQF (Ours)
83.90
4.4. Hyper-Parameter Analysis
The Number of Experts. As shown in Figure 5a, to investi- gate the impact of the number of experts n, we sample n as 2, 4, 6, 8 and 10, to evaluate the R@] accuracy and mAP under different numbers of experts. A constant threshold confidence level fixed to 0.4 in the overall experiment. When n less than average R @ performance gradually increases with an increasing number of experts. However, when exceeds 6, larger leads to decrease in perfor- mance. we observe that although increasing n can propor- tionally enhance the model's 's information capacity, larger n does not necessarily lead to better performance. This sug- gests that the model's capacity cannot grow indefinitely. We ultimately determine as a practical choice.
Table 4. Comparison of different routing strategies. Method
Avg.
Top-K Routing
80.58
Soft Routing
81.80
Hash Routing
83.11
Ours (Adaptive Routing)
83.90
Ablation Study on Feature Fusion Strategies To vali- date the effectiveness of the proposed Cross-Modal Query
8

---

FlexiReID: Adaptive Mixture of Expert for Multi-Modal Person Re-Identification
Threshold Confidence Level As shown in Figure 5b, set the number of experts n and explore the R@ 1 curacy and mAP with the change of threshold confidence level. When the confidence level is low, the model's per- formance is poor. However, as the threshold confidence level increases to 0.4, the performance reaches its peak, As the threshold confidence level increases further, the model's performance essentially reaches a plateau without additional improvement. Therefore, the threshold confidence level is set to 0.4.
Acknowledgements
This work was supported by the National Science Fund for Distinguished Young Scholars (No. (20.62025,,3) the Na- tional Natural Science Foundation of China (No. U2IB2037, No. U22B2051, No. U23A20383, No. U21A20472, No. 62176222, No. 62176223, No. 62176226, No. 62072386, No. 62072387, No. 62072389, No. 62002305 and No. 62272401), and the Natural Science Foundation of Fujian
Province of China (No. 2021J06003, No.2022106001).
Impact Statement
845
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
83.5
87.5
Avg Acuuncyh))
82.5
81.5
References
81
+ Aug Rg]
666
Chen, C., Ye, M., Qi, M., and Du, B. Sketch transformer: Asymmetrical disentanglement learning from dynamic synthesis. In Proceedings of the 30th ACM International Conference on Multimedia, pp. 4012-40200 2022a.
80.5
Number of Adagters
(a) Ablation Study the Number of Experts.
84.5
68
Chen, C., Ye, M., Qi, M., Wu, J., Jiang, J., and Lin, C.- W. Structure- positional transformer for visible- infrared person re-identification. IEEE Transactions Image Processing. 31:2352-2366 2022b.
84
Ang RG@ Acconsc((%)
83.5
67.5
82.5
Chen, C., Ye, M., Qi, M., Wu, J., Liu, Y., and Jiang, J. Saliency and granularity: Discovering temporal coher- ence for video-based person re-identification. IEEE Trans- actions on Circuits and Systems Video Technology. 32 (9):610-66112 2022c.
81.5
01
80.5
0.2
0.4
0.6
0.8
Threshold confidence level
(b) Ablation Study on Threshold Confidence.
Chen, C., Ye, M., and Jiang, D. Towards modality-agnostic person re-identification with descriptive query. In Pro- ceedings of the IEEF/CVF Conference on Computer Vi- sion and Pattern Recognition, pp. 15128-15137, 2023a.
Figure 5. Ablation Study on the Number of Experts and Threshold Confidence.
Chen, C., Ye, M., Qi, M., and Du, B. Sketchtrans: Disentan- gled prototype learning with transformer for sketch-photo recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, (6(5):2950-296) 2023b.
5. Conclusion
In this paper, we propose FlexiRelD, a multimodal person re-identification framework that supports flexible retrieval across four modalities: text, sketches, RGB, and infrared, as well as any combination thereof. We design the AEA- MoE mechanism to dynamically select expert networks and introduce the CMQF module to optimize modsl fea- ture fusion. Based on the expanded CIRS-PEDES dataset, experimental results demonstrate that FlexiRelD outper- forms existing methods in complex scenarios, validating its flexibility and effectiveness, and opening new research directions for multimodal person re-identification.
Chen, J., Guo, L., Sun, J., Shao, S., Yuan, Z., Lin, L., and Zhang, D. Eve: Efficient vasion-language pre-training with masked prediction and modality-awan moe. In Proceedings of the AAAI Conference Artificial Intelli- gence, volume 38, pp. 1110-1119, 2024.
Chen, S., Ge, C., Tong, Z., Wang, J., Song, Y., Wang, J., and Luo, P. Adaptformer: Adapting vision transformers for scalable visual recognition. Advances in Neural Informa- tion Processing Systems, 16666-6666677 2022d.

---

FlexiReID: Adaptive Mixture of Expert for Multi-Modal Person Re-Identification
Chen, Y., Zhang, G., Lu, Y., Wang, Z., and Zheng. Y. Tipeb: A simple but effective part-based convolutional baseline for text-based person search. Neurocomputing, 494:171- 181, 2022e.
Jia, C., Yang. Y., Xia, Y., Chen, Y.-T., Parekh, Z., Pham, H., Le, Q., Sung, Y.-H., Li, Z., and Duerig, T. Scaling up visual and vision-language representation learning with noisy text supervision. In International conference on machine learning, pp. 4904-4916. PMIR, 2021.
Chen, Y.-C., Li, L.,, Yu, L., EI Kholy, A., Ahmed, F., Gan, Z., Cheng, Y., and Liu, J. Uniter: Universal image-text repre- sentation learning. In European conference on computer vision, pp. 104-120. Springer, 2020.
Jiang, D. and Ye, M. Cross-modal modal implicit relation reason- ing and aligning for text-to-image person retrieval. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 7787-2797, 2023.
Cho, J., Lei, J., Tan, H., and Bansal, M. Unifying vision-and- language tasks via text generation. In International Con- ference on Machine Learning, pp. 1931-1942. PMLR, 2021.
Karras, T., Aittala, M., Laine, S., Härkönen, E., Hellsten, J., Lehtinen, J., and Aila, T. Alias-free generative adversarial networks. Advances in neural information processing systems, 34:852-863, 2021.
Ding, Z., Ding, C., Shao, Z., and Tao, D. Semantically self-aligned network for text-to-image part-aware person re-identification. arXiv preprint arXiv:2107. 12666, 2021.
Langley, P. Crafting papers on machine learning. In Langley, P. (ed.), Proceedings of the 17th International Conference on Machine Learning (ICML 2000). pp. 1207-1216, Stan- ford, CA, 2000. Morgan Kaufmann.
Dosovitskiy, An image is worth 16x 16 words: Trans- formers for image recognition at scale. arXiv preprint arXiv:2010. 1929, 2020.
Lepikhin, D., Lec, H., Xu, Y., Chen, D., Firat, O., Huang, Y., Krikun, M., Shazeer, N., and Chen, Z. Gshard: Scaling giant models with conditional computation and automatic sharding. arXiv preprint arXiv:2006. 16668, 2020.
Eigen, D., Ranzato, M., and Sutskever, I. Learning fac- tored representations in a deep mixture of experts. arXiv preprint arXiv:1312.433: 2013.
Farooq, A., Awais, M., Kittler, J., and Khalid, S. S. Axm- net: Implicit cross-modal feature alignment for person re- identification. In Proceedings of the AAAI conference on artificial intelligence, volume 36, pp. 4477-4485, 2022.
Li, J., Selvaraju, R., Gotmare, A., Joty, S., Xiong, C., and Hoi, S. C. H. Align before fuse: Vision and language representation learning with momentum distillation. Ad- vances in neural information processing systems, 34: 9694-9700 2021.
Fedus, W., Zoph, B., and Shazeer, N. Switch transformers: Scaling trillion parameter models with simple and ef- ficient sparsity. Journal of Machine Learning Research, 23(120):1-39,200 2022.
Li, S., Xiao, T., Li, H., Zhou, B., Yue, D., and Wang, X. Person search with natural language description. In Pro- ceedings of the IEEE conference on computer vision and pattern recognition, pp. 1970-1979, 2017.
Gao, C., Cai, G., Jiang, X., Zheng, F. Zhang, J., Gong, Y., Peng. P., Guo, X., and Sun, X. Contextual non-local alignment over full-scale representation for text-based person search. arXiv preprint arXiv:2101.0303 2021.
Li, S., Cao, M., and Zhang, M. Learning semantic-aligned feature representation for text-based person search. In ICASSP 2022-2022 IEEE International Conference Acoustics, Speech and Signal Processing (ICASSP), pp. IEEE, 2022.
Gao, P., Geng, S., Zhang, R., Ma, T., Fang. R., Zhang, Y., Li, H., and Qiao, Y. Clip-adapter: Better vision-language models with feature adapters. International Journal of Computer Vision, (822))581-595 2024.
Liu, X., Cheng, X., Chen, H., Yu, H.,, and Zhao, G. Differ- entiable auxiliary learning for sketch re-identification. In Proceedings of the AAAI Conference on Artificial Intelli- gence, volume 38, pp. 3747-3755, 2024a.
Gui, S., Zhu, Y., Qin, X., and Ling, X. Learning multi- level domain invariant features for sketch re-identification. Neurocomputing, 403:294 303, 2020.
Liu, Y., Li, Y., Liu, Z., Yang, W., Wang, Y., and Liao, Q. Clip-based synergistic knowledge transfer for text-based person retrieval. In ICASSP 2024-2024 IEEE Interna- tional Conference on Acoustics, Speech and Signal Pro- cessing (ICASSP), pp. 7935-7939. IEEE, 2024b.
Han, X., He. S., Zhang. L., and Xiang. T. Text- based person search with limited data. arXiv preprint 10Xiv:2110.10880 2021.
Hao, X., Zhao, S., Ye, M., and Shen, J. Cross-modality per- son re-identification via modality confusion and center aggregation. In Proceedings of the JEEE/CVF Interna- tional conference on computer vision, pp. 16403-16412, 2021.
Lu, J., Batra, D., Parikh, D., and Lee, S. Vilbert: Pre- training task-agnostic visiolinguistic representations for vision-an diioo-and-las aad-language tasks. Advances in neural informa- tion processing systems, 32, 2019.
10

---

FlexiReID: Adaptive Mixture of Expert for Multi-Modal Person Re-Identification
Lu, Y., Wu, Y., Liu, B., Zhang, T., Li, B., Chu, Q., and Yu, N. Cross-modality person re-identification with shared- specific feature transfer. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 13379-13389 2020.
Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., and Chen, M. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06115, 1(2):3 2022.
Riquelme, C., Puigcerver, J., Mustafa, B., Neumann, M., Jenatton, R., Susano Pinto, A., Keysers, D., and Houlsby, N. Scaling vision with sparse mixture of experts. Ad- vances in Neural Information Processing Systems, 34: 8583-8595, 2021.
Nichol, A., Dhariwal, P., Ramesh, A., Shyam, P., Mishkin, P., McGrew, B., Sutskever, I., and Chen, M. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprint arXiv:21 10741, 2021.
Shao, Z., Zhang, X., Fang. M., Lin, Z.., Wang, J., and Ding, C. Learning granularity uniffed representations for text- to-image person re- re-identification In Proceedings of the 30th acm international conference on multimedia, pp. 5566-5574, 2022.
Niu, H., Hu, J., Lin, J., Jiang, G., and Zhang, S. Bov- seg: Efficient open- vecn-oobaullry panoptic segmentation. In Proceedings of the AAAI Conference on Artificial Intelli- gence, volume 39, pp. 6262, 2025a.
Shazeer, N., Mirhoseini, A., Maziarz, K., Davis, A., Le, Q.. Hinton, G., and Dean, J. Outrageously large neural networks: The sparsely-gate mixture-of of---ofeeeepee effeext layer. arXiv preprint arXiv: 006638,,,..,,.. 2017.
Niu, H., Xie, L., Lin, J., and Zhang, S. Exploring semantic consistency and style diversity for domain generalized se- mantic segmentation. In Proceedings of the AAAI Confer- ence on Artificial Intelligence, volume 39, pp. 1245-6253, 2025b.
Shu, X., Wen, W., Wu, H., Chen, K., Song, Y., Qiao, R., Ren, B., and Wang, X. See finer, see more: Implicit modality alignment for text-based person retrieval. In European Conference on Computer Vision, pp. 624-641. Springer, 2022.
Niu, K., Huang, Y., Ouyang, W., and Wang, L. Improv- ing description- based person re- re-identification by multi- granularity image-text alignments. IEEE Transactions on Image Processing, 29:5542-5556, 2020.
Su, W. Zhu, X., Cao, Y., B., Lu, L.,, Wei, F., and Dai, J. VI-bert: Pre-training of generic visual-linguistic represen- tations. arXiv preprint arXiv: 1908 085300 2019.
Özkanoglu, M. A. and Ozer, S. Infragan: A gan architecture to transfer visible images to infrared domain. Pattern Recognition Letters, 155:69-76, 2022.
Tan, H. and Bansal, M. Lxmert: Learning cross-modality encoder representations from transformers. arXiv preprint arXiv:
Pang, L.., Wang, Y., Song, Y.-Z., Huang, T., and Tian, Y. Cross-domain adversarial feature learning for sketch re- identification. In Proceedings of the 26th ACM interna- tional conference on Multimedia, pp. 609-617, 2018.
2019.
Tian, X., Zhang, Z., Lin, S., Qu, Y., Xie, Y., and Ma, L. Farewell to mutual information: Variational distillation for cross-modal person re- re-identification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1522-1531, 2021.
Patashnik, O., Wu, Z., Shechtman, E., Cohen-Or, D., and Lischinski, D. Styleclip: Text-driven manipulation of stylegan imagery. Proceedings of the IEEE/CVF inter- national conference on computer vision, pp. 2085-2094, 2021.
Vaswani, A. Attention is all you need. Advances in Neural Information Processing Systems, 2017.
Pu, N., Chen, W., Liu, Y., Bakker, E. M., and Lew, M. S. Dual gaussian-based variational subspace disentangle- ment for visible-infrared person re-identification. In Pro-
Wang. M., Xing, J., and Liu, Y. Actionelip: A new
paradigm for video action recognition. arXiv preprint arXiv:2109.08472, 2021.
ceedings of the 28th ACM international conference on multimedia,, pp. 2149-2158, 2020.
Wang, Z., Fang, 7.,, Wang, J., and Yang. Y. Vitaa: Visual- textual attributes alignment in person search by natural language. In Computer Vision-ECCV 2020: 16th Eu- ropean Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XII 16, pp. 402-420. Springer, 2020.
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. Learning transferable visual models from natural language supervision. In International conference machine learning, pp. 8748-8763, PMLR, 2021.
Wang, Z., Zhu, A., Xue, J., Wan, X., Liu, C., Wang, T., and Li, Y. Caibe: Capturing all-round information beyond color for text-based person retrieval. In Proceedings of the 30th ACM international conference on multimedia, pp. 5314-5322 5322, 2022a.
Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Rad- ford, A., Chen, M., and Sutskever, I. Zero-shot text-to- image generation. In International conference on ma- chine learning, pp. 8821-8831 Pmir, 2021.

---

FlexiReID: Adaptive Mixture of Expert for Multi-Modal Person Re-Identification
Wang. Z., Zhu, A., Xue, J., Wan, X., Liu, C., Wang, T., and Li, Y. Look before you leap: Improving text-based person retrieval by learning a consistent cross-modal common manifold. In Proceedings of the 30th ACM international conference on multimedia, pp. 1984-1992, 2022h.
Ye, M., Chen, C., Shen, J., and Shao, L. Dynamic tri-level relation mining with attentive graph for visible infrared re- identification. IEEE Transactions on Information Foren- sics and Security, 17:386-399 -3,,,,,,,, 2021a.
Ye, M., Ruan, W., Du, B., and Shou, M. Z. Channel aug- mented joint learning for visible-infrared recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 13567-13576, 2021b.
Wei, Z., Yang, X., Wang, N., and Gao, X. Syncretic modal- ity collaborative learning for visible infrared person re- identification. In Proceedings of the IEEE/CVE inter- national conference on computer vision, pp. 225-234, 2021.
Ye, M., Shen, J., Lin, G., Xiang, T., Shao, L., and Hoi, S. C. Deep learning for person re-identification: A survey and outlook. IEEE transactions pattern analysis and machine intelligence, 44(6))2872-288) 2021c
Wu, A., Zheng, W.-S., Gong, S., and Lai, J. Rgb-ir person re- identification by cross-modality similarity preservation. International journal of computer vision, 128(6):1765- 1785, 2020.
Yu, T., Lu, Z., Jin, X., Chen, Z., and Wang. X. Task residual for tuning vision-language models. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 10899-10909, 2023.
Wu, Q., Dai, P., Chen, J., Lin, C.-W., Wu, Y., Huang, F., Zhong, B., and Ji, R. Discover cross-modality nuances for visible-infrared person re-identification. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4330-4339, 2021.
Zhai, X., Wang, X., Mustafa. B., Steiner, A., Keysers, D., Kolesnikov, A., and Beyer, L. Lit: Zero-shot transfer with locked-image text tuning. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pp. 18123-18133, 2022a.
Yan, S., Dong, N., Zhang, L., and Tang, J. Clip-driven fine- fineggainned text-image person re-identification. IEEE Transactions Image Processing, 2023.
Zhai, Y., Zeng, Y., Cao, D., and Lu. S. Trireid: Towards multi-modal person re-identification via descriptive fu- sion model. In Proceedings of the 2022 International Conference on Multimedia Retrieval, pp. 63-71, 2022b.
Yang. B., Yc, M., Chen, J., and Wu. Z. Augmented dual- contrastive aggregation learning for unsupervised visible- infrared person re-identification. In Proceedings of the 30th ACM International Conference on Multimedia, pp- 2843-2851, 2022.
Zhang. Y. and Lu, H. Deep cross-modal projection learning for image-text matching. In Proceedings of the European conference on computer vision (ECCV), pp. 686-701, 2018.
Yang, B., Chen, J., and Ye, M. Towards grand unified representation learning for unsupervised visible-infrared person re-identification. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 11069- 11079, 2023.
Zhao, Z., Liu, B., Chu, Lu, Y., and Yu, N. Joint color- irrelevant consistency learning and identity-aware modal- ity adaptation for visible-infrared cross modality person re-id re-identificatinn. In Proceedings of the AAAI confer- ence on artificial intelligence, volume 35, pp. 3520-3528, 2021.
Yang, B., Chen, J., and Ye, M. Shallow-deep collabora- tive learning for unsupervised visible-infrared person re- identification. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pp. 16870-16879, 2024.
Zhou, K., Yang, J., Loy, C. C., and Liu, Z. Learning to prompt for vision-language models. International Jour- nal of Computer Vision, 130(9))2377-2348 2022.
Yang, F., Wu, Y., Wang, Li, X., Sakti, S., and Nakamura, S. Instance-level heterogeneous domain adaptation for limited-labelee sketch-to-phote retrieval. IEEE Transac- tions Multimedia, 23:2347-2360, 2020.
Zhu, A., Wang, Z., Li, Y., Wan, X., Jin, J., Wang, T., Hu, F., and Hua, G. Dssl: Deep surroundings person separation learning for text-based person retrieval. In Proceedings of the 29th ACM international conference on multimedia, pp. 209-217, 2021.
Ye, M., Shen, J., J. Crandall, D., Shao, L., and Luo, J. Dynamic dual-attentive aggregation learning for visible- infrared person re-identification. In Computer Vision- ECCV 2020: 16th European Conference, Glasgow, UK August 23-28, 2020, Proceedings, Part XVII 16, pp. 229- 247. Springer, 2020.
12

---

FlexiReID: Adaptive Mixture of Expert for Multi-Modal Person Re-Identification
A. Ablation Study on the Impact of Applying Adaptive Moe to Different Encoders.
To further investigate the applicability and effectiveness of our proposed AEA-MOE mechanism within the FlexiReID framework, we design an ablation study presented in the appendix. In this study, the AEA-MOE module applied individually the image encoder, the text encoder, and jointly to both, aiming to clarify the performance contribution of this mechanism across different modality-specific processing modules. As shown in the table 6, Experiment No.1 incorporates AEA-MOE only into the image encoder, No.2 applies it to the text encoder, and No.3 integrates it into both encoders. Based on the Rank @ accuracy across seven retrieval tasks, we observe that AEA-MOE improves performance in both the image and text encoders. However, the enhancement is more significant on the image side, and the best overall performance is achieved when applied to both encoders simultaneously.
These results validate the generalizability of the proposed AEA-MOE module. can be effectively adapted to both image and text modalities and demonstrates complementary enhancement effect in multimodal retrieval scenarios.
Table Ablation results the CUHK-PEDES dataset evaluating the impact of applying the AEA-MOB mechanism to the image encoder, text encoder, or both. Method
No.
T > R
> R
IR >
R
T
>
R
T
IR
1R
R
IR > R
Avg.
MLP Adapter
66,38
81.47
81.93
83.58
83,26
82.43
85.03
80,58
68.24
83.73
83.81
85.87
85.39
84.46
86.88
82.63
No.O+AEA-MME(Teel
67.25
82.66
82.78
84.52
84.06
83.19
85.74
81.46
NuO=AAEA-ME(/maee...
68.87
84.13
84.37
86.41
85,93
84.97
87.35
83.14
B. Experiments on PKU-Sketch
To further validate the effectiveness of the FlexiRelD framework in handling sketch-based scenarios, conduct additional experiments on the PKU-Sketch dataset, which consists of two modalities: sketches and RGB images. To address the issue of missing modalities, we adopt Learnable Embedding Features (LEF) strategy, wherein each missing modality is compensated by corresponding learnable embedding vector. These embedding vectors are jointly optimized with the main network parameters during training, enabling semantic completion and consistency preservation for the missing modalities. This allows the model to perform robust cross-modal retrieval even when the input modality combination incomplete.
As shown in the figure FlexiReID achieves superior performance compared to existing methods on the PKU-Sketch dataset. This demonstrates that FlexiReID exhibits strong robustness and practical applicability in complex scenarios with incomplete multimodal inputs.
Table 7. Sketch-based retricval performance on the PKU-Sketch datasct.
Method
Reference
mAP
Rank @
Rank @5
Rank @ 10
CCSC
MM22
83.7
86.0
98.0
100.0
Sketch Trans+
PAMT2023
85.8
96.0
99.0
DALNet
AAA12024
86.2
90.0
98.6
100.0
FlexiReID(Ours)
91.2
93.5
99.3
100.0
C. Experiments on Market-1501 and MSMT17
To further evaluate the performance of the FlexiRelI) framework on the traditional RGB-to-RGB single-modal person re-identification task, we conduct experiments on two widely used RelD datasets: Market-1501 and MSMT17. This experiment aims verify whether our proposed framework can still exhibit strong visual feature modeling capabilities without relying auxiliary modality information. Notably, to maintain architectural consistency with multimodal retrieval settings, utilize learnable embedding features replace the missing modalities. This mechanism ensures that even when only RGB images are available, the model can still receive a unified four-modality representation, thereby preserving the semantic generalization ability inherent in the multimodal design.
As shown in the table 8, FlexiReID outperforms existing representative methods on both Market- 1501 and MSMT17. These results clearly demonstrate that our method not only excels in complex cross-modal retrieval scenarios but also delivers outstanding performance in standard RGB-RGB settings, highlighting its strong generalizability and practical applicability.
13

---

FlexiReID: Adaptive Mixture of Expert for Mudti-Modal Person Re-Identification
Table 8. Evaluation results on the Market-1501 and MSMT17 datasets for the RGB-1o-RGB person person re-identification task.
Market-1501
MSMT17
Methods
Reference
Rank @
mAP
Rank@1 1
mAP
FastRelD
ACM MM23
95.4
88.2
83.3
59.9
BPBreID
WACV23
95.1
87.0
MV12P
Inf Fusion24
95.2
87.0
80.4
56.4
FlexiReID(Ours)
96.0
92.1
83.7
67.5
14


Reviews:
[{"review_id": "EoYpk1lkJi", "reviewer": "Reviewer_dFYK", "summary": "FlexiReID is a novel framework for multimodal person re-identification that enables flexible retrieval across various single or combined modalities—including text, sketches, RGB, and infrared images—thereby addressing the limitations of existing methods that focus on only one or two modality pairs. By introducing an adaptive mixture of experts (MOE) mechanism, FlexiReID dynamically integrates outputs from different expert networks, leveraging each modality’s strengths to enhance retrieval performance. Additionally, a cross-modal query fusion module refines the fused features to optimize their representational quality. To evaluate the framework comprehensively, the authors construct a unified dataset called CIRS-PEDES, derived from four existing ReID datasets (CUHK-PEDES, ICFG-PEDES, RSTPReID, and SYSU-MM01) and enriched with text, sketches, RGB, and infrared data.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your careful and valuable comments. We will explain your concerns point by point.\n\n> Table1 illustrates FlexiReID achieves promising accuracy in the T+S+IR→R situations. However these sketches are generated from RGB images, while in real-world scenarios, sketches is a front-facing portrait sketched from memory, which leads to a substantial discrepancy between the sketches depicted in the paper and the official version.\n> \n\n> The paper focuses on the multi-modal person re-ID, but these multi-modalities are generated via AI tools which is not align with real-world scenarios. Thus, there remains a gap compared to practical applications.\n> \n\nA1: In response to the two similar concerns you raised, we provide the following explanation. It is indeed true that sketches and other multimodal data generated using intelligent tools may differ from those in real-world scenarios. However, this approach still holds significant value. Currently, publicly available person re-identification datasets that span more than two modalities are scarce. Leveraging such generative methods can substantially enrich the modality diversity of datasets, facilitating the construction of unified multimodal datasets and providing more comprehensive support for model training.Moreover, we have also conducted experiments on real-world test sets. For instance, as shown in Table 2, SYSU-MM01 is a real near-infrared (NIR) to RGB retrieval dataset. The inclusion of generated modalities led to notable performance improvements. In addition, we evaluated our method on the PKU-Sketch dataset, which contains real sketch modality data (as shown in the table1 below), and on the RegDB dataset, which includes real infrared modality data (see Table 4 in our response to reviewer YSVq), further validating the effectiveness of our approach in real-world applications.\n\nWe believe that FlexiReID can serve as a foundational framework, which can be further fine-tuned with data from real-world scenarios to enhance the model's adaptability to practical environments. Therefore, although some modality discrepancies exist at this stage, this reflects a common characteristic of research in its developmental phase. Our work introduces new perspectives and methodologies for the field of multimodal person re-identification and lays a solid foundation for future research in real-world applications.\n\nTable 1: Experiments on PKU-Sketch\n\n| Methods | Reference | mAP | Rank@1 | Rank@5 | Rank@10 |\n| --- | --- | --- | --- | --- | --- |\n| CCSC | MM22 | 83.7 | 86.0 | 98.0 | 100.0 |\n| Sketch Trans+ | PAMI2023 | - | 85.8 | 96.0 | 99.0 |\n| DALNet | AAAI2024 | 86.2 | 90.0 | 98.6 | 100.0 |\n| FlexiReID（OUrs） | - | 91.2 | 93.5 | 99.3 | 100.0 |\n\n> In Line433, the authors claims \"No.2 employs the traditional MOE method, while No.3 utilizes AEA-MOE.\" which seems not consistent with Table 3?\n> \n\nA2: Thank you for your correction. There was indeed a labeling error in the manuscript. In fact, No.0 corresponds to the zero-shot CLIP backbone baseline, No.1 adopts the conventional MoE approach based on the Top-K mechanism, while No.2 and No.3 employ the proposed AEA-MoE method. We will make the necessary corrections in the revised version.\n\n> The authors propose a flexible multimodal re-ID framework, but why do the authors ignore RGB-to-RGB retrieve, which is the main stream of re-ID?\n> \n\nA3: Thank you for your constructive suggestion. Following your advice, we have included RGB-to-RGB retrieval in our evaluation. We assessed the performance of FlexiReID on the MSMT17 and Market-1501 dataset, and the corresponding results are presented in the table2 below:\n\nTable 2: Experiments on Market-1501 and MSMT17\n\n|  | Market-1501 |  | MSMT17 |  |\n| --- | --- | --- | --- | --- |\n| Methods | Rank@1 | mAP | Rank@1 | mAP |\n| FastReID(ACMMM23) | 95.4 | 88.2 | 83.3 | 59.9 |\n| BPBreID(WACV23) | 95.1 | 87.0 | - | - |\n| MVI2P(Inf Fusion24) | 95.2 | 87.0 | 80.4 | 56.4 |\n| FlexiReID(Ours) | 96.0 | 92.1 | 83.7 | 67.5 |\n\n> In Figure2, why do the authors apply SDM loss to features of the same images, in which the model cannot retrieve persons across various situations?\n> \n\nA4: I may have misunderstood your question. Are you asking why the SDM loss is applied to images of pedestrians with the same pose in the same scene during training? In fact, during the data processing stage of training, we construct modality pairs using images of the same identity captured in different scenes and with different poses, then compute the SDM loss based on these pairs. During testing, the model is also capable of retrieving images of the same pedestrian taken in different scenes from the query. We will refine the illustration in Figure 2 to eliminate this ambiguity. If you have any further questions, we look forward to continued discussions.", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "There do not exist theoretical claims.", "experimental_designs_or_analyses": "Table1 illustrates FlexiReID achieves promising accuracy in the T+S+IR→R situations. However these sketches are generated from RGB images, while in real-world scenarios, sketches is a front-facing portrait sketched from memory, which leads to a substantial discrepancy between the sketches depicted in the paper and the official version.", "relation_to_broader_scientific_literature": "This work is related to multimodal retrieval, demonstrating that combining multiple modalities can lead to stronger feature embeddings and higher accuracy.", "essential_references_not_discussed": "None", "other_strengths_and_weaknesses": "Strengths:\n1. The paper considers muliti-modal person re-ID and the proposed AEA-MOE is proved to be effective in muliti-modal person re-ID.\n\nWeaknesses:\n1. In Line433, the authors claims \"No.2 employs the traditional MOE method, while No.3 utilizes AEA-MOE.\" which seems not consistent with Table 3?\n2. The paper focuses on the multi-modal person re-ID, but these multi-modalities are generated via AI tools which is not align with real-world scenarios. Thus, there remains a gap compared to practical applications.", "other_comments_or_suggestions": "None", "questions_for_authors": "1. The authors propose a flexible multimodal re-ID framework, but why do the authors ignore RGB-to-RGB retrieve, which is the main stream of re-ID?\n2. In Figure2, why do the authors apply SDM loss to features of the same images, in which the model cannot retrieve persons across various situations?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "None"}}, {"review_id": "vXtdwyexpE", "reviewer": "Reviewer_YSVq", "summary": "The paper proposes the FlexiReID framework to support person retrieval across seven different modality combinations (such as text, sketches, infrared images, RGB images, and their combinations). The framework comprises an AEA-MoE mechanism for dynamically selecting varying numbers of expert networks according to input features, and a CMQF module that is capable of effectively integrating features from different modalities and compensating for missing modalities through learnable embedding features. To support the study, the paper constructs a dataset named CIRS-PEDES which unifies four modalities. Extensive experiments show FlexiReID's efficacy in multimodal person re-identification.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your thoughtful feedback. We'll address each of your concerns in detail.\n\n> No. The paper misses model comparison experiments for multi-modal retrieval on benchmark datasets.\n> \n\n> In Table 1, except for T-R task, there are few comparison results for other task, which cannot demonstrate the superiority of the proposed method. Similarly, in Table 2, there are no comparison results for six tasks. And the compared methods for IR-R task are not SOTA, the proposed method demonstrates relatively average performance on the dataset.\n> \n\n> Based on the experimental results presented in the paper, the performance of the proposed method is relatively modest.\n> \n\nA1: There are few models for multi-modal retrieval, with UNIReID being one of the representative works. Our work is the first to explore flexible compositional retrieval, so no existing methods are directly comparable for the many-to-one setting. If you have any recommended works (preferably with open-source code，due to time constraints), we’d be glad to include them. Nevertheless, We provide additional comparisons with S-R and IR-R methods in Table 1 below, and report UNIReID’s performance on three tasks in Table 2 below. It is important to note that we did not employ any task-specific module designs or training strategies for individual retrieval tasks, which may limit performance in single-task scenarios. However, compared to traditional single-modality cross-modal retrieval, FlexiReID supports a wider range of retrieval modes and demonstrates better generalization capabilities.\n\nTable1: Supplementary experiments on CUHK-PEDES\n\n| Tasks | Methods | Venue | R1 | mAP |\n| --- | --- | --- | --- | --- |\n| S—R | Sketch Trans+ | PAMI2023 | 81.39 | 73.72 |\n|  | DALNet | AAAI2024 | 83.03 | 75.39 |\n|  | FlexiReID(Ours) | - | 84.92 | 79.21 |\n| IR—R | GUR | ICCV2023 | 82.06 | 75.84 |\n|  | SDCL | CVPR2024 | 84.57 | 77.32 |\n|  | FlexiReID(Ours) | - | 85.26 | 79.43 |\n\nTable2: Supplementary experiments on SYSU-MM01\n\n|  | SYSU-MM01 | ALL-Search |  | Indoor-Search |  |\n| --- | --- | --- | --- | --- | --- |\n| Tasks | Methods | R1 | mAP | R1 | mAP |\n| T—R | UNIReID | 54.6 | 52.8 | 56.3 | 63.5 |\n|  | FlexiReID(Ours) | 56.8 | 65.4 | 58.2 | 67.6 |\n| S—R | UNIReID | 64.2 | 57.7 | 65.8 | 73.8 |\n|  | FlexiReID(Ours) | 66.4 | 60.3 | 68.5 | 75.3 |\n| T+S—R | UNIReID | 66.9 | 65.9 | 67.9 | 72.7 |\n|  | FlexiReID(Ours) | 68.7 | 67.2 | 70.6 | 73.4 |\n\n> It is suggested to compare CMQF with other routing mechanism.\n> \n\nA2: We added comparisons between CMQF and other routing mechanisms (Table 3). Our adaptive routing shows better average performance than the alternatives.\n\nTable3: Comparison experiment of routing mechanisms\n\n| Routing | Avg. |\n| --- | --- |\n| Top-K | 80.58 |\n| Soft Routing | 81.80 |\n| Hash Routing | 83.11 |\n| Ours(Adaptive Routing) | 83.90 |\n\n> The paper focuses on unified cross-modal person re-identification framework, could you explain the motivation behind this idea? Given that RGB -RGB is the most common task, why not integrate it into a more unified framework?\n> \n\nA3: Our unified framework aims to handle diverse real-world inputs beyond fixed modality pairs (e.g., Text-RGB, IR-RGB). In practice, users may provide multiple modalities (e.g., text, sketch, IR), which existing models struggle to integrate. FlexiReID supports seven modality combinations, improving retrieval performance and robustness. Following your suggestion, we also added RGB-to-RGB retrieval, with results on Market-1501 and MSMT17 shown in Table 2 of our response to reviewer dFYK.\n\n> Could you assess the impact of synthetic data on model performance through comparative experiments? And it would be more convincing to evaluate the approach on other real-life datasets.\n> \n\nA4: We use generative models to fill missing modalities, enabling flexible retrieval despite incomplete dataset coverage. To evaluate the impact of synthetic data, we tested on real-world datasets including SYSU-MM01 (Table 2 in the article), RegDB (Table 4 below), and PKU-Sketch (Table 1 in our response to reviewer dFYK). The results confirm FlexiReID’s practical effectiveness as a unified, adaptable framework with strong real-world deployment potential.\n\nTalbe4: Experiments on RegDB(IR-R)\n\n| Method | Venue | R1 | mAP |\n| --- | --- | --- | --- |\n| SFANet | TNNLS23 | 70.2 | 63.8 |\n| CAJ | TPAMI24 | 84.9 | 77.8 |\n| DARD | TIFS24 | 85.5 | 85.1 |\n| FlexiReID | - | 88.6 | 87.4 |\n\n> Could you also analyze the computational complexity and inference speed of the proposed method?\n> \n\nA5: We analyzed the computational complexity and inference speed of FlexiReID. With a frozen CLIP encoder and lightweight AEA-MoE and CMQF modules, it runs at 19 GFLOPs and 14 ms/query on a single NVIDIA 3090 GPU—comparable to UNIReID (17 GFLOPs, 11 ms/query), which supports only three fixed modality combinations. In contrast, FlexiReID supports seven flexible combinations with similar efficiency, making it more adaptable and deployable.", "claims_and_evidence": "No.\nIn L68-71, the claim  ’ which supports flexible retrieval with arbitrary modality combinations’ is problematic. FlexiReID only supports seven different modality combinations and misses the other modalities such as thermal, LiDAR and event data.", "methods_and_evaluation_criteria": "No.\nThe paper misses model comparison experiments for multi-modal retrieval on  benchmark datasets.", "theoretical_claims": "No proofs.", "experimental_designs_or_analyses": "In Table 1, except for T-R task, there are few comparison results for other task, which cannot demonstrate the superiority of the proposed method. Similarly, in Table 2,  there are no comparison results for six tasks. And the compared methods for IR-R task are not SOTA, the proposed method demonstrates relatively average performance on the dataset.", "relation_to_broader_scientific_literature": "The key contributions of the paper related to unified person ReID model that can handle multi retrieval task in a unified modal, such as UNIReID", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "Strength: \nThe paper introduces, for the first time, the concept of flexible retrieval, which supports seven different modality combinations for retrieval.\n\nWeakness:\nBased on the experimental results presented in the paper, the performance of the proposed method is relatively modest.", "other_comments_or_suggestions": "It is suggested to expand Table 1 and Table 2 by adding additional comparative results, particularly from more recent studies. As for ablation study, it is suggested to compare CMQF with other routing mechanism.\n\n-----------------\nThe author addressed most of my comments, so I will raise my score.", "questions_for_authors": "1.\tThe paper focuses on unified cross-modal person re-identification framework, could you explain the motivation behind this idea? Given that RGB -RGB is the most common task, why not integrate it into a more unified framework?\n2.\tThe paper employs generative models to synthesize missing modalities, could you assess the impact of synthetic data on model performance through comparative experiments? And it would be more convincing to evaluate the approach on other real-life datasets. \n3.\tCould you also analyze the computational complexity and inference speed of the proposed method?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "I16FQB50xk", "reviewer": "Reviewer_Tfk2", "summary": "This paper firstly introduces  the concept of flexible retrieval in the field of person re-identification and propose a corresponding method FlexiReID which supports flexible retrieval with arbitrary modality combinations. The authors also constructed a unified dataset by existing ReID datasets.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> I am curious about the comparative experiment. This paper only provides the comparison with SOTA methods on Text-to-RGB task. However, there exists previous methods for other dual modalies retrieval. But this paper do not provide the comparison results.\n> \n\nA1: Thank you for the valuable question. Currently, there are still few works focusing on multi-modal retrieval. Among them, UNI-ReID is one of the representative methods that support retrieval across multiple modalities (text, sketch, and their combination). Our work is the first to systematically explore the novel query paradigm of *flexible compositional retrieval*, and we are the first to propose the many-to-one retrieval setting. Therefore, there are no existing methods that can be directly compared with ours under this setting. If the reviewer has any recommended works (preferably with open-source code，due to time constraints), we would be glad to include them in our discussion.\n\nNonetheless, we have conducted comprehensive comparisons with existing methods on several dual-modality retrieval tasks. For example, we compared with recent Sketch-to-RGB methods on the PKU-Sketch dataset (see Table 1 in our response to reviewer dFYK), and with recent Infrared-to-RGB methods on the RegDB dataset (see Table 4 in our response to reviewer YSVq). In addition, we have included the latest methods for the S→R and IR→R tasks in Table 1 of the paper (see Table 1 in our response to reviewer YSVq). These results demonstrate that FlexiReID achieves competitive performance across multiple tasks, further validating its advantages as a unified and flexible framework for multimodal retrieval.\n\n> The author used generative models (StyleGAN3, InfraGAN, GPT-4) to extend the modality of sketches, infrared images, and text descriptions. Although this approach is effective, it may result in significant differences between the generated data and the actual collected data.\n> \n\nA2: Thank you for the valuable question. It is true that using generative models such as StyleGAN3, InfraGAN, and GPT-4 to synthesize modalities like sketches, infrared images, and textual descriptions may introduce certain differences compared to real-world collected data. However, this approach is particularly meaningful at the current stage, as publicly available person re-identification datasets spanning three or more modalities are extremely limited. The inclusion of synthetic modalities greatly enriches the diversity of data modality, facilitates the construction of a unified multimodal dataset, and provides essential support for training our proposed flexible compositional retrieval framework.\n\nTo assess the practical effectiveness of our method, we have also conducted evaluations on several real-modality datasets. For instance, as shown in Table 2 of the article, SYSU-MM01 is a real NIR-to-RGB retrieval dataset. Additionally, we evaluated our method on the PKU-Sketch dataset, which contains real sketch modality data (see Table 1 in our response to reviewer dFYK), and on RegDB, a dataset with real infrared images (see Table 4 in our response to reviewer YSVq). These experiments further validate the effectiveness and applicability of FlexiReID in real-world scenarios.\n\nWe believe FlexiReID serves as a solid and flexible foundation, which can be further fine-tuned using real-world data to enhance its adaptability. While some discrepancies exist between synthetic and real data, this reflects a necessary stage in the research process. Our work offers new insights and methodologies for multimodal person re-identification and lays a foundation for future practical deployment.", "claims_and_evidence": "The concept is reasonable.", "methods_and_evaluation_criteria": "The authors propose AEA-MoE mechanism to dynamically selects different numbers of experts and CMQF module to leverage learnable embedding features to compensate for missing modalities and fuse different modality features. It seems reasonable to solve this new problem. \nThis paper provide the widely used metrices like Rank-K accuracy、mAP、mINP.", "theoretical_claims": "This article does not involve strict theoretical proofs, nor does it propose clear theoretical propositions or theorems. The main contributions in the article are focused on method design, module implementation, experimental verification, and qualitative analysis, so there is no issue of reviewing theoretical proofs.", "experimental_designs_or_analyses": "I am curious about the comparative experiment. This paper only provides the comparison with SOTA methods on Text-to-RGB task. However,   there exists previous methods for other dual modalies retrieval. But this paper do not provide the comparison results.", "relation_to_broader_scientific_literature": "This work is releated to  Cross-modal Person Re-identification, Mixture-of-Experts and Vision-Language Pre-training Models.", "essential_references_not_discussed": "NA", "other_strengths_and_weaknesses": "Strengths:\nThe method is practical and has clear application value.\n\nWeakness: \nThe author used generative models (StyleGAN3, InfraGAN, GPT-4) to extend the modality of sketches, infrared images, and text descriptions. Although this approach is effective, it may result in significant differences between the generated data and the actual collected data.", "other_comments_or_suggestions": "NA", "questions_for_authors": "NA", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gvpdnRjqLh", "reviewer": "Reviewer_NWLb", "summary": "This paper presents the FlexiReID framework, which addresses the issue of modality combination retrieval that has been largely overlooked in the current cross-modal person re-identification (ReID) field. Specifically, traditional approaches in cross-modal ReID typically use a single modality as the query to match the corresponding person. This paper introduces, for the first time, the concept of flexible combination retrieval, where the query can be replaced with different modality combinations (e.g., text, sketch, infrared), supporting seven distinct retrieval methods, thereby enhancing the model's applicability across diverse scenarios.\nThe approach proposed in this paper includes the innovative introduction of an adaptive mixture of experts (MoE) mechanism, applied during the modality feature extraction process. In contrast to the traditional Top-K routing mechanism used in MoE models, the adaptive routing mechanism introduced here allows for flexible selection of the number of experts based on the complexity of the features. To ensure that the model selects only the minimal number of necessary experts, the paper also incorporates an adaptive loss function. This enables the model to handle different modality extraction tasks without increasing computational burden.\nAdditionally, this paper introduces a novel feature fusion method, CMQF, which employs a two-layer fusion structure consisting of an interaction layer and a fusion layer. To address the issue of missing modalities, the method uses learnable feature vectors as substitutes, thereby effectively considering all possible modality combinations. The final model is optimized using contrastive loss.\nIn the experimental validation section, the paper constructs the CIRS-PEDES dataset, spanning four modalities. Compared to methods that only support single modality cross-modal retrieval, FlexiReID achieves optimal performance. Furthermore, the combination retrieval methods supported by FlexiReID further enhance retrieval performance, demonstrating that the use of multiple modalities for person re-identification leads to superior results.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> The ablation experiment section lacks a comparison between the adaptive routing mechanism and the Top-K routing mechanism. It would be helpful to include this comparison.\n> \n\nA1: In fact, we have already compared a method using the Top-K mechanism in our ablation study, specifically in Row No.1 of Table 3. Additionally, we have supplemented our work with comparative experiments of CMQF combined with other routing mechanisms(see Table 3 in our response to reviewer YSVq). These ablation studies on routing strategies further validate the effectiveness of our proposed adaptive routing.\n\n> The paper could explore different feature fusion methods, test their performance, and compare them with the CMQF feature fusion method proposed in the paper to highlight the advantages of the proposed approach.\n> \n\nA2: Following your suggestion, we introduced three fusion strategies for comparison with our CMQF, as shown in the table below. The three strategies are: Concatenation—concatenating features from different modalities and feeding them into a Transformer module for fusion; Summation—summing features from different modalities and then using a Transformer for fusion; and Hierarchical Fusion—passing each modality through its own Transformer module first, followed by a shared Transformer for final fusion. It can be seen from Table 1 below that our CMQF achieves the best performance among all methods.\n\nTable1: Comparative experiment of feature fusion methods\n\n| No | Method | T—R | S—R | IR—R | T+S—R | T+IR—R | S+IR—R | T+S+IR—R | Avg. |\n| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |\n| 1 | Concatenation | 68.87 | 84.13 | 84.37 | 86.41 | 85.93 | 84.97 | 87.35 | 83.14 |\n| 2 | Summation | 67.85 | 83.20 | 83.51 | 85.42 | 85.06 | 84.08 | 86.54 | 82.24 |\n| 3 | Hierarchical Fusion | 68.93 | 84.47 | 84.82 | 86.94 | 86.11 | 85.55 | 87.76 | 83.51 |\n| 4 | CMQF(Ours) | 69.20 | 84.92 | 85.26 | 87.47 | 86.23 | 85.97 | 88.23 | 83.90 |", "claims_and_evidence": "I believe the effectiveness of the FlexiReID method proposed in the paper is indeed validated through experiments on multiple datasets. Specifically, tests were conducted on four datasets with expanded modalities, evaluating seven different person re-identification methods. The results show that, in single-modality retrieval tasks, FlexiReID achieves optimal performance compared to the current state-of-the-art cross-modal ReID methods. Moreover, FlexiReID supports combination modality retrieval methods, which are not supported by mainstream approaches. It also demonstrates that using multiple modality combinations as queries can lead to superior performance, as opposed to relying on a single modality as the query. This highlights the significant practical implications of FlexiReID, which enables flexible matching of persons across various modalities in real-world applications.", "methods_and_evaluation_criteria": "I believe the FlexiReID framework proposed in this paper holds significant importance for the development of the person re-identification (ReID) field. ReID has evolved from single-modality matching to cross-modal matching, during which many excellent recognition methods have emerged. However, these methods all belong to the one-to-one identification category, where a model supports only one modality as a query to match another modality. This paper transcends the traditional one-to-one ReID paradigm by introducing the concept of many-to-one, where multiple modality combinations serve as queries to match the target modality, thus opening up a new research direction. Moreover, with the advancement of various surveillance technologies, acquiring multiple modalities of pedestrians has become increasingly common. Therefore, the need for a generalizable model that supports both cross-modal and combination modality retrieval has become more pressing. For these reasons, I consider the FlexiReID method proposed in this paper to be of great significance for complex real-world applications. On the other hand, this paper also expands current ReID datasets by incorporating additional modalities. I believe this creates the necessary data conditions for the emerging research direction of combination retrieval.", "theoretical_claims": "The method proposed in this paper is generally sound. Specifically, the paper applies two innovative techniques: adaptive mixture of experts (MOE) and modality fusion. Regarding adaptive MOE, the paper proposes an adaptive routing mechanism to overcome the limitation of the traditional MOE model, which uses the Top-K routing mechanism to select a fixed number of experts. The proposed adaptive routing mechanism customizes the number of experts based on the complexity of the input modality features.\nAs for the modality fusion technique, it employs a two-layer structure, consisting of an interaction layer and a fusion layer, with ablation experiments providing strong evidence of its effectiveness. I believe the authors could further explore additional modality fusion strategies to better highlight the advantages of their proposed method.", "experimental_designs_or_analyses": "I have thoroughly reviewed the experimental design and analysis section of this paper, and I believe it is generally well-reasoned. In the performance evaluation section, the paper presents results from models evaluated on four datasets spanning RGB, sketch, infrared, and text modalities, and compares them with current state-of-the-art cross-modal ReID methods. Overall, FlexiReID achieves optimal performance across various cross-modal ReID tasks on different datasets. Additionally, the combination modality retrieval methods it supports further enhance its performance.\nIn terms of ablation experiments, the paper conducts ablations on the adaptive MOE and feature fusion modules, as well as on the two hyperparameters: the number of experts and the threshold confidence. I believe the ablation experiments could be further supplemented by including an ablation of the adaptive routing mechanism versus the Top-K mechanism, which would provide a more comprehensive demonstration of the advantages of the proposed adaptive MOE.", "relation_to_broader_scientific_literature": "The concept of flexible combination modality retrieval proposed in this paper represents a further development of previous cross-modal person re-identification (ReID) work. Typical examples of traditional cross-modal ReID tasks include text-to-visible modality and infrared-to-visible modality person re-identification. A classic paper on text-to-visible person re-identification is Cross-Modal Implicit Relation Reasoning and Aligning for Text-to-Image Person Retrieval, which introduces an implicit reasoning and alignment model called IRRA. This model leverages cross-modal implicit local relation learning for global alignment without requiring any additional supervision or reasoning costs.\nOn the other hand, for infrared-to-visible re-identification, the paper Multi-Stage Auxiliary Learning for Visible-Infrared Person Re-Identification proposes MSALNet, which first applies grayscale histogram transformations to infrared and visible light images and trains the model in two stages to reduce color-related effects. It then uses the HFCL module to fuse cross-modal information and the MSR module to suppress low-similarity feature locations. Finally, the DCA loss function is used to optimize the distance between samples and cross-modal class centers, reducing intra-class variation.\nI believe the framework proposed in this paper not only encompasses the functionality of traditional ReID tasks but also incorporates the capability for combination modality retrieval, which previous frameworks lacked, thus enhancing its versatility.", "essential_references_not_discussed": "I believe the FlexiReID method proposed in this paper unifies various traditional cross-modal ReID tasks within a single model, while also supporting flexible combination modality retrieval. This significantly enhances the versatility of a single model. The concept of flexible combination modality retrieval has not been addressed in previous ReID literature, and this method opens up a new exploratory direction in the field of ReID.", "other_strengths_and_weaknesses": "Strengths：\n1）This paper introduces the concept of combination modality person re-identification (ReID) for the first time, opening up a new research direction in the ReID field. I believe that this approach, which integrates various cross-modal ReID tasks into a single framework, holds significant importance for complex real-world application scenarios.\n2）This paper proposes the adaptive MOE method, which, in contrast to traditional MOE methods based on the Top-K routing mechanism, offers a more flexible expert selection strategy. By introducing adaptive loss, this module can automatically learn and select the minimum number of necessary experts, further optimizing both the model's performance and computational cost.\n3）I appreciate the illustrations in this paper; they are clear and highly readable. Each figure effectively aids in understanding the method proposed by the authors, allowing readers to quickly grasp the key concepts. The details and annotations in the figures are also well-executed, enhancing both the professionalism and comprehensibility of the illustrations. Overall, the design of the figures significantly contributes to the presentation of the paper, greatly improving its overall readability.\n\nWeaknesses\n1）The ablation experiment section lacks a comparison between the adaptive routing mechanism and the Top-K routing mechanism. It would be helpful to include this comparison.\n2）The paper could explore different feature fusion methods, test their performance, and compare them with the CMQF feature fusion method proposed in the paper to highlight the advantages of the proposed approach.", "other_comments_or_suggestions": "I believe that the greatest contribution of this article is the proposal of the concept of combinatorial modal person re-identification, which caters to the era of the increasing development of various monitoring technologies. The constructed FlexiReID framework, as a unified model integrating various cross-modal ReID tasks, has strong practical significance. In addition, the proposed adaptive MOE method is also quite innovative.\nIt is hoped that in the future, the performance of the adaptive routing mechanism can be compared with that of the Top - K routing mechanism to further demonstrate its advantages.", "questions_for_authors": "1）The paper proposed a two - layer feature fusion method and achieved good results. Could you try to compare other feature fusion methods and illustrate the advantages of the method you proposed?\n2）Could you conduct an additional ablation experiment to compare the performance of the adaptive routing mechanism you proposed with that of the traditional Top - K routing mechanism?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2wVD1cf5Q0", "reviewer": "Reviewer_33pg", "summary": "This article innovatively introduces the concept of combined modality pedestrian re-identification. Compared with traditional cross-modal identification, it demonstrates greater flexibility in dealing with complex scenarios. Centered around this concept, the article constructs the FlexiReID framework, which integrates a variety of cross-modal and combined modality pedestrian identification tasks, and exhibits extremely strong universality, enabling it to adapt well to diverse application scenarios.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> In the related work section, the author could include a review of the literature on feature fusion.\n> \n\nA1: Thank you for the suggestion. We appreciate your insight and will include a review of relevant literature on feature fusion in the related work section in the revised version.\n\n> The authors could conduct additional ablation studies to independently analyze the impact of applying the adaptive MoE mechanism to the text and image modalities on performance.\n> \n\nA2: Thank you for your suggestion. As requested, we have added the corresponding ablation experiments, with the results shown in the table below. In Row No.1, the adaptive MoE mechanism is applied on the image modality side, while in Row No.2, it is applied on the text modality side. As shown, applying the adaptive MoE to the image modality yields a greater performance improvement. This is because the image branch contains three modalities, making it more suitable for the adaptive MoE mechanism.\n\nTalbe1: Ablation study on the impact of applying adaptive moe to different encoders\n\n| No. | Method | T—R | S—R | IR—R | T+S—R | T+IR—R | S+IR—R | T+S+IR—R | Avg. |\n| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |\n| No.0 | MLP_Adapter | 66.38 | 81.47 | 81.93 | 83.58 | 83.26 | 82.43 | 85.03 | 80.58 |\n| No.1 | No.0+AEA-MOE(Image) | 68.24 | 83.73 | 83.81 | 85.87 | 85.39 | 84.46 | 86.88 | 82.63 |\n| No.2 | No.0+AEA-MOE(Text) | 67.25 | 82.66 | 82.78 | 84.52 | 84.06 | 83.19 | 85.74 | 81.46 |\n| No.3 | No.0+AEA-MOE(Image,Text) | 68.87 | 84.13 | 84.37 | 86.41 | 85.93 | 84.97 | 87.35 | 83.14 |\n\n> In the feature fusion module, have you considered directly fusing two modalities without using learnable features as replacements when a modality is missing?\n> \n\nA3: Thank you for your question. In fact, as shown in Row No.4 of the ablation study in Table 3 of the paper, we conducted an experiment without using learnable features to replace the missing modalities. The comparison shows that using learnable features to fill in missing modalities is beneficial for performance improvement.\n\n> In the ablation experiment section, how is feature fusion performed when the feature fusion module is not used?\n> \n\nA4: Thank you for your question. In the ablation study, when the feature fusion module is not used, we concatenate the features and pass them through a Transformer module to extract global features for fusion. This corresponds to the fusion method in Row No.1 of Table 1 in our response to reviewer NWLb. Additionally, the same table includes ablation experiments comparing different fusion strategies, which clearly demonstrate the performance advantages of our CMQF method.", "claims_and_evidence": "This work's claims are intuitive, convincing, and supported by its experiments.", "methods_and_evaluation_criteria": "The FlexiReID framework proposed in this paper is highly forward-looking. It breaks away from the traditional single-task mode and constructs a comprehensive and multi-level recognition system by integrating various identification tasks. In practical application scenarios, pedestrian data features are rich and diverse. Single cross-modal recognition is difficult to comprehensively capture key information. However, the combined modality recognition advocated by FlexiReID can fully explore the complementarity among different modality data, greatly improving the recognition accuracy. The fusion of multiple modality data makes the recognition more accurate. This innovative concept not only meets the complex real-world needs but also opens up a new path for the widespread application of ReID technology. It is expected to become the core direction of future industry development.", "theoretical_claims": "The theory proposed in this paper is highly compelling and well-supported by solid evidence. Taking the attention allocation mechanism in the model as an example, it innovatively introduces a dynamic weighting strategy based on the distribution of data features. This strategy adaptively adjusts the weight proportions of submodules according to the degree of dispersion in the input data features. Furthermore, in the feature fusion stage, the paper adopts a progressive fusion architecture. Initially, features from different modalities are processed separately, followed by cross-mapping and fusion between features, gradually integrating information across modalities. This hierarchical, multi-stage fusion approach effectively captures the correlations between different modality features.", "experimental_designs_or_analyses": "Overall, the experimental design in this paper demonstrates a high level of rationality. The study selects datasets that encompass diverse scene characteristics and conducts comprehensive evaluations on a variety of ReID tasks. The experimental results clearly indicate that the proposed method exhibits significant advantages in key performance metrics such as accuracy and recall. Furthermore, through carefully designed ablation studies, the paper convincingly validates that multimodal collaborative processing achieves superior recognition accuracy compared to unimodal independent operation, highlighting the powerful efficacy of multimodal fusion.", "relation_to_broader_scientific_literature": "The FlexiReID framework proposed in this paper expands the research paradigm of cross-modal ReID, extending from traditional single-modal queries (such as text-to-image or infrared-to-image) to multi-modal combination retrieval, an aspect that has not been fully explored in existing literature. Previous studies mainly focused on cross-modal feature alignment and modality-invariant feature learning, such as modality transformation methods based on adversarial learning or Transformers, but none considered the possibility of multi-modal joint queries. Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer first introduced the MOE model based on the Top-K routing mechanism. In contrast, this paper proposes an adaptive MoE mechanism, which is more dynamic than the traditional Top-K MoE method and can flexibly select the number of active experts. Additionally, the feature fusion method designed in the paper, combined with a learnable feature filling strategy, effectively enhances retrieval performance in the case of missing modalities, which is uncommon in previous research based on simple feature concatenation or shared representation learning. Overall, I believe FlexiReID is a more flexible, universal, and efficient solution for person re-identification.", "essential_references_not_discussed": "In the related work section, the author could include a review of the literature on feature fusion.", "other_strengths_and_weaknesses": "Strengths：\n\n- The research findings of this paper are not limited to the field of cross-modal person re-identification. For instance, its proposed concept of modality combination and adaptive mechanisms provide new insights and methodological references for addressing challenges in complex data fusion and dynamic model adjustment across various related domains.\n\nWeaknesses：\n\n- The authors could conduct additional ablation studies to independently analyze the impact of applying the adaptive MoE mechanism to the text and image modalities on performance.\n\n- In the related work section, the authors could further supplement the discussion with a comprehensive review and literature survey on feature fusion research.", "other_comments_or_suggestions": "The FlexiReID framework proposed in this paper is truly distinctive in the field of cross-modal person re-identification (ReID), offering a novel perspective for addressing the challenges of ReID in complex scenarios.\n\nHowever, in the experimental validation section, while the paper conducts ablation studies to verify the effectiveness of each module within the overall framework, the specific role of the adaptive MoE mechanism in processing different modalities could be further explored. For instance, additional ablation studies could be conducted to independently analyze the impact of applying the adaptive MoE mechanism on the text and image modalities, providing deeper insights into its contributions to overall performance.", "questions_for_authors": "- In the feature fusion module, have you considered directly fusing two modalities without using learnable features as replacements when a modality is missing?\n\n- In the ablation experiment section, how is feature fusion performed when the feature fusion module is not used?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: Five experts in the field reviewed this paper. Their recommendations are 3 Weak Accepts and 2  Accepts. Overall, the reviewers appreciated the paper because the new FlexiReID method integrates an adaptive MOE and CMQF feature modality fusion. The AEA-MOE mechanism dynamically selects different numbers of experts. It can be versatile and effective in multimodal person re-ID, and supports seven different modality combinations for retrieval. Moreover, the results are convincing, showing that FlexiReID can achieve promising accuracy. 
   Some reviewers initially raised concerns regarding the paper, but the author provided a detailed and insightful response that addressed most of these concerns. For example, there is concern that sketches and other multimodal data generated using AI tools (StyleGAN3, InfraGAN, GPT-4) to synthesize missing modalities that may not align with those captured in real-world scenarios. However, the authors argue that multimodal datasets are limited, and have shown that such generative methods can improve the diversity of datasets, and model accuracy. 
    I recommend accepting this paper based on the reviewers’ feedback and the authors’ satisfactory rebuttal. Overall, FlexiReID is motivated, holds significant novelty, and the experimental validation is convincing. However, the reviewers have raised some concerns that should be addressed in the final camera-ready version of the paper. In particular, the ablations lacks comparisons with different adaptive routing mechanisms and feature fusion methods. This paper only provides the comparison with SOTA methods on cross-modal ReID tasks. Based on the experimental results, the performance of the proposed method may be relatively modest.
Decision: Accept (poster)