MDReID: Modality-Decoupled Learning for Any-to-Any Multi-Modal Object Re-Identification
Authors: Yingying Feng
Jie Li
Jie Hu
Yukang Zhang
Lei Tan
Jiayi Ji
Venue: NeurIPS 2025 spotlight ()
Keywords: Multi-modal Person Re-identification; Modality Decoupled Learning
Primary area: applications

Abstract: The challenge of inconsistent modalities in real-world applications presents significant obstacles to effective object re-identification (ReID). However, most existing approaches assume modality-matched conditions, significantly limiting their effectiveness in modality-mismatched scenarios. To overcome this limitation and achieve a more flexible ReID, we introduce MDReID to allow any-to-any image-level ReID systems. MDReID is inspired by the widely recognized perspective that modality information comprises both modality-shared features, predictable across modalities, and unpredictable modality-specific features, which are inherently modality-dependent and consist of two key components: the Modality Decoupling Module (MDM) and Modality-aware Metric Learning (MML). Specifically, MDM explicitly decomposes modality features into modality-shared and modality-specific representations, enabling effective retrieval in both modality-aligned and mismatched scenarios. MML, a tailored metric learning strategy, further enhances feature discrimination and decoupling by exploiting distributional relationships between shared and specific modality features. Extensive experiments conducted on three challenging multi-modality ReID benchmarks (RGBNT201, RGBNT100, MSVR310) consistently demonstrate the superiority of MDL. MDReID achieves significant mAP improvements of 9.8\%, 3.0\%, and 11.5\% in modality-matched scenarios, and average gains of 3.4\%, 11.8\%, and 10.9\% in modality-mismatched scenarios, respectively.

MDReID: Modality-Decoupled Learning for Any-to-Any Multi-Modal Object Re-Identification
Yingving Feng'; Li ; Jie Hu², Yukang Zhang?,
Lei Tan Jiayi Ji
Northeastern University 'Xiamen University 'National University of Singapore fengyingyinngestuall www.neu.eeeee heu.edu..om cn, tanlei @nus edu. sg
Abstract
Real-world object re-identification (ReID) systems often face modality inconsis- tencies, where query and gallery images come from different sensors (e.g.. RGB, NIR, TIR). However, most existing methods assume setlality-maccher conditions, which limits their robustness and scalability in practical applications. To address this challenge, we propose MDReID, a flexible any-to-any image-level RelD frame- work designed to operate under both moddlltt-matcheer and modality-mismatehe scenarios. MDReID builds on the insight that modality information can be de- composed into two components: modality-shared features that predictable and transferable, and modality-specific features that capture unique, modality- dependent characteristics. To effectively leverage this, MDRelD introduces two key components: the Modality Decoupling Learning (MDL) and Modality-awarr Metric Learning (MML). Specifically, MDL explicitly decomposes modality fea- tures into modality-share
and modality-specific representations, enabling effective retrieval in both modality-aligned and mismatched scenarios. MMI., tailored met- ric learning strategy, further enforces orthogonality and complementarity between the two components enhance discriminative power across modalities. Extensive experiments conducted on three challenging multi-modality ReID benchmarks (RGBNT201, IGBNT100, MSVR310) consistently demonstrate the superiority of MDReID. Notably, MDRelD achieves significant mAP improvements of 9.8%, 3.0%, and 11.5% in general modality-matched scenarios, and average gains of 3.4%, 11.8%, and 10.9% in modality code is available at: http://gghuubccom//oooo661131/11
modallty-mismatchnn mismatched scenarios, respectively. The
Introduction
Object Re-Identification (ReID) focuses on identifying and retrieving specific objects across non- overlapping camera views. Conventional object re- re-identification (ReID) methods [1, 2, 3, 4, 6, 7] primarily rely on RGB images. However, RGB-based approaches often face significant limitations under challenging environmental conditions, such poor illumination, shadows, and low image resolution. These adverse conditions frequently lead to the extraction misleading features, thereby diminishing discriminative capability. To overcome these limitations, multi-modal RelD [8, 9, 10, 11, 12, 13], which integrates complementary information from multiple spectrums, has emerged as promising approach. By leveraging the inherent strengths and complementary features of different modalities, multi-modal ReID significantly enhances feature representations, enabling more robust and accurate identification in complex real-world scenarios.
"Tese authors contributed equally this work.
Corresponding Author: Lei Tan Prooecc Leader
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Modaltty-mathhed
Modulity-mismattinn
Modality Medality-speefffe
(RGBB]
TIR) JRGB, TIRI (RGI, TIRI
TiKi
Uncertain Query
Uncertaia Gallery
Modality-tared Feature
[RGB]
[RGB, T(M) [RGB, INIR,
(a) Examples of condality-(miismmttte ReID scenarios.
(b) The motivation of proposed MDReID.
Figure 1: Illustration of the motivation of MDReID. (a) Though the availability of spectral modalities (e.g., RGB, NIR, TIR) varies across queries and galleries, recent methods only focus on the modality-matchee matthed scenarios, which limits their practical applicability. (b) MDReID overcomes the rigidity of modality constraints by disentangling modality-sharee and modality-speciffc features, enabling effective matching between queries and galleries from arbitrary modalities.
The introduction of multi-modal data has significantly enhanced the performance of RelD in chal-
lenging scenarios, thereby stimulating extensive research efforts in this field [8, 9, 10, 11, 12]. EDITOR [11] selects diverse tokens from Vision Transformer (ViT) to mitigate the impact of irrele- vant backgrounds and narrow the gap between modalities, achieving strong detection performance. TOP-ReID [12] incorporates cyclic token permutation module to aggregate tures and a complementary reconstruction module minimize the distribution gap across different image spectra. These innovations enable TOP-ReID to achieve robust performance under both modality comppltt--compete and modallty-miissigg scenarios. Although these methods effectively leverage the characteristics of multi-modal data, they are developed based on fundamental assumption that all modalities within the dataset are strictly aligned (as illustrated in Figure (a) Modality-matched) Ideally, every camera would simultaneously provide RGB, NIR, and TIR modalities. However, due to practical constraints such as device differences and deployment diversity, achieving fully matched retrieval conditions is challenging in weal-world scenarios. Therefore, is crucial de- velop a flexible image-level any-to-any ReID framework capable effectively operating under both modality-matched matched and miimatchee scenarios (as illustrated in Figure (a)).
spectral fea-
To address this limitation, we propose MDReID, flexible any-to-an, ReID framework designed to support retrieval tasks involving arbitrary combinations of query and gallery modalities. The core challenge this problem lies in learning effective representations that remain robust under both modality- mootaedt-matched and modality-mismateh
conditions. A straightforward solution, adopted by TOP-ReID [12], attempts to predict the missing modality representation from the available one, thereby enabling modality- aligned retrieval. However, as highlighted by RLE [14], predicting cross- spectrum features solcly from visual inputs constitutes an ill- ill-posed problem. The modality-speciffi characteristics that inherently unpredictable often lead to suboptimal learning. To this end, Learning (MDL), which introduces a modality-shane shared
MDRelD introduces a and -ppecit token into the ViT architecture, leveraging multi-layer attention within the transformer to extract shared and specific features from multi-mode inputs. MDL explicitly disentan- gles the modality representations into two complementary components: modality-sharee component that captures the predictable cross-modal representation and a modality-speciftt component that preserves the unpredictable modality characteristics. This decoupling enables the model to better generalize across diverse modality combinations while maintaining the modality-specific cues.
Furthermore, enhance the disentanglement between the two components, propose Modality- aware Metric Learning (MML) strategy. MML comprises two complementary objectives: repre- sentation orthogonality loss (ROL) and knowledge discrepancy loss (KDL). ROL is applied at the channel level to both promote the aggregation of modality enforce orthogonality between shared and specific components, ensuring they capture distinct and non-overlapping information. KDL, on the other hand, encourages representational complementarity between shared and specific components by ensuring that the combined representation more discriminative than either component alone.
features across modalities and
To sum up, the main contributions of this paper are as follows:

---

We propose MDReID, a flexible any-to-any object re-identification framework that supports retrieval across arbitrary query-gallery modality combinations, addressing the practical limitations of strictly aligned multi-modal datasets.
We introduce a Mlodality-Dccouppe Learning (MDL) strategy, coupled with Modality- aware Metric Learning (MML) strategy. MDL explicitly disentangles modality and modality-specific representations, while MML further strengthens this disentangle- ment by introducing a representation orthogonality loss and knowledge discrepancy loss, encouraging the two components to encode distinct and complementary information.
Extensive experiments on multi-spectral object ReID datasets (RGBNT201, RGBNTI00, MSVR310) demonstrate the superior adaptability and performance of the MDReID across diverse scenarios. MDReID achieves significant mAP improvements of 9.8%, 3.0%, and 11 1.55% in general modality-matcher scenarios, and average gains of 3,4%, 11.8%, and 10.9% in moddalty-mismatcchie scenarios, respectively.
2 Related Work
Compared to the general EGB-to-RGit object re-identification [15, 16, 17, 18, 19, 20, 21], multi- modal object re- re-identification (RelD) has demonstrated significant practical value, achieving widespread applications in cross-scenario systems [22, 23] (e.g., security surveillance, intelligent transportation, etc.). Current methods [24, 9, 25] primarily enhance re-identification accuracy by effectively leveraging the complementarity and integration of multi-modal features. For example, Zheng et al. [26] proposed the cross cooss-iiretiinal consistency network (CCNet), which overcomes discrepancies in both modality and sample aspects, thereby achieving more effective multi-moda data collaboration. To address the modal- missing problem, Zheng et al. [27] proposed the DENet model, which incorporates a feature transformation module to recover information from missing modalities and dynamic enhancement module to improve multi-modality representation. Li et al. [8] proposed HAMNet, which automatically fuses different spectral features using a specially designed heterogeneous score coherence loss. He et al. [28] proposed the GPFNet model, which utilizes graph learning to fuse multi-modal features.
Notably, models based on Vision Transformer (ViT) [29, 30, 10, 31, 32, 33] for object re-identification have achieved excellent results in recent years and gained widespread attention. Pan et al. [34] proposed the H-ViT model, which alleviates challenges caused by heterogeneous molti-modalities and reduces feature deviations arising from modal variations. Additionally, they introduced a progressively hybrid transformer (PHT [35]) that effectively fuses multi-modal complementary information through random hybrid augmentation and a feature hybrid mechanism. Crawford et al. [36] addressed the issue of modality laziness in multi-modal fusion by proposing the UniCat model based VIT architecture. By selecting diverse tokens from VIT to mitigate the impact of irrelevant backgrounds and narrow the gap between modalities, Zhang et al. [11] developed the EDITOR model. Wang et al. [12] designed the Top-RelD model, which achieves state-of-the-art accuracy by incorporating cyclic token permutation module aggregate multi-spectral features and complementary reconstruction module to minimize the distribution gap across different image spectra.
Despite advancements in multi-modal object ReID, existing methods rely on modality-aligned conditions, limiting their adaptability to real- wolldwrldd scenarios. To address this, we introduce MDReID, which disentangles modality-sharee and modality-sseeciff both matched and mismatched conditions. It incorporates Modality-Decoupler Learning (MDL) to refine feature separation and Modality-aware Metric Learning (MML) enhance discrimination through orthogonality and knowledge discrepancy losses. As a result, MDRelD enables robust object re-identification across diverse modality configurations.
features for effective retrieval across
3 Methodology
3.1 MDRelD: Any-to-any Object ReID
In ideal multispectral person re-identification (ReID) settings, complete RGB, NIR, and TIR modal- ities are available for both query and gallery (RNT-to-RNT). However, real-world deployments often suffer from different modalities due to heterogeneous sensor availability and deployment
3

---

Shared token
Lits + Liri
RGB
Visiten TrarsforTIES Block.
Liwas Projection
Uag yhr fling)
(
[A N Gol
NIR
US lingg)
Dap for fing!
TIR
Tmngl
Dip. fing)
Modality--awar aware Learning Representatinn Orthogonality Loss (ROL)
Knowledge Discrepancy Loss (K.D.)
A nchhor
Posilise
Negative
Targyt:
[0 12 12.
Target cross-corelation
Shored Fosture Un Hb) Specific Feature (.p) Combined Feature (11
(app)lal)
Figure 2: Overall framework of MDReID, MDReID designed to support retrieval across arbitrary modality combinations. It disentangles features into shared and specific components to boost performance in both matched and mismatched scenarios. Additionally, by leveraging representation orthogonality loss (ROL) and knowledge discrepancy loss (KDL), MDReID refines feature separation and enhances retrieval robustness.
constraints. To address this fundamental challenge, we propose MDReID, a flexible, any-to-any ReID framework that supports arbitrary combinations of input and output modalities. As illustrated in Fig. 2. MDReID adopts a Vision Transformer (ViT) backbone and integrates two core compo- nents: Modality -eceoullee Learning (MDL) and Modality
Metric Learning (MML). MDL explicitly disentangles each modality's representation into modality-shane features (crucial for cross-modality retrieval) and modality-specifft features (essential for preserving discriminative in matched scenarios). MML further enhances this decoupling by enforcing metric-level consistency across modalities. This design enables robust and flexible object re- re-identification in diverse and environments, significantly advancing the applicability of RelD in practical
settings.
3.2 Modality Decoupled Learning
Generally, with ViT as the backbone, we process input images from different modalities (M {R, N, 7, corresponding to RGB, NIR, and TIR). Standard ViT procedure involves segmenting the image into non-overlapping patches, which are then linearly projected into patch embeddings as:
M [[ IM IIM I
where M E (,,N
(I)
To explicitly model and decouple modality characteristics crucial for cross-modal ReID, we deviate from the standard use of single [CLS] token. Instead, prepend two distinct learnable tokens to the patch embedding sequence for each modality: a boddality-specinng token and a modality-share token. This augmented sequence is then processed by the ViT encoder to get:
IMM (17 I I,M, J.
}),, where M (R, N,T).
(2)
where M denotes the modality-specifit features, M denotes the modality-shared features, and 1 represents the encoded features.
Leveraging these decoupled features from different modalities (I INI.N.IT unified representation for each sample using a fixed-dimensional structure and associated availability mask. This allows consistent handling across different modality presence scenarios. Specifically, define potential full feature vector, Vfulls by concatenating all possible specific and shared components, assuming all three modalities (RGB, NIR, TIR) are present:
IT IR..I ap- sh- IT.) we construct
Ufull
[1p.- 1 1 1h 1 .17]
(3)

---

Correspondingly, a binary availability mask, mask indicates the presence of each component: mask full = [1, 1, 1, 1,
(4)
where 'I' signifies that the feature component at that position is present and valid, and "0" (as introduced below) signifies absence or invalidity.
The actual feature vector e and mask mask used for any given input sample are derived from this full structure based on the modalities available for that specific sample. If a modality M E (R, N, T) is missing for a sample, the corresponding modality specificc feature Ind and the modality shared feature T.M in the vector are replaced with zero vector O. Also, the entries in the mask mask: corresponding to these zeroed-out features (I and ( are set to O.
For example, if the TIR modality (T) is missing for sample, its feature vector and mask mask would be:
[Tp
7. sp
O. FR IN 0) mask = [1, 1, 1,0,
(5)
This structured representation (v and mask) is used consistently across all scenarios, including both modality-matched and modality-mismatceed retrieval. In modality-matchel scenarios (e.g., RN-to-RN), both the query and gallery samples will have the same modalities present (or absent), resulting in identical structures for their masks (e.g., [1, 1, 0, 1, 1, 0] for both if T is missing). In scenarios (e.g., R-to-N), the query and gallery samples will inherently have different available modalities. Their respective u and mask representations are generated according to the rules above based on their own available data. For instance, the RGB query sample would have lig = [TR O, O, T O, 0J and maski = [1. 0, 0. 1, O, 0j. And the NIR gallery sample would have fgg 0, sp 0, O, T 0) and maskig = (0, 1,0,0
modality-mismatchhe
The subsequent retrieval process utilizes both the feature vectors (Vg, 1g) and their corresponding availability masks (maskg. maskg) to compute a similarity score that respects the decoupled nature of the features and handles missing modalities robustly. We define the total similarity, Sim( (Vg 0g). as the combine of two components, i.e., the modality-specific similarity Sim.p and the modality- shared similarity Simgh- The modality-speciff similarity compares modality-specifee features between identical modalities only. The similarity is calculated as the sum of dot products between corresponding specific features, masked for joint availability:
Simap sum(masksppa) swm (ummkmskssss))
1
Σ
(U (T -ask sp.q
mask. sp.g"
(E(..,
The modality-shared similarity computes the similarity between all pairs of available shared features across modalities. Define Vsh.g be a matrix with shared features IJ IN IT (or zero vectors sh.qq shiq" sh.g
unavailable) as columns, and similarly for Vah..g masksh.g and masksh.g. We first compute the matrix of all pairwise shared feature similarities:
Sah. VT
sh.q Vah.g
This results in a 3 x 3 matrix where (San)ij is the dot product (T ( for i,j E (R, N, T)- To account for missing modalities, we define pairwise availability mask matrix with the outer product of the shared masks:
Mah = mask maskah.g
(8)
where (Mat_ pair) ) is if and only if both the i-th shared feature of the query and the j-th shared feature of the gallery are present. The total shared similarity the sum of all valid pairwise similarities, obtained by Hadamard product of Ssh and Msh pair:
1
Simsh sum summMsh_pair
Σ (Sah @ Mah-pair)ij.
Finally, the total similarity score is:
Simrotal vgy) = (Simep + Sim,a)/2.
(10)
This approach, based on a fixed-size zero-padded feature vector and an explicit availability mask,
provides a flexible and robust mechanism for handling arbitrary combinations of modality presence and absence, effectively adapting to all modality-mate matched and mismatched object RelD scenarios addressed by our method.
5

---

3.3 Modality-aware Metric Learning
After MDL, the features are separated into modality speccifc and poddality-shared components. To achieve more complete and thorough feature decoupling, we propose a modality metric learning approach. Note that, during the training phase, we assume all the modalities for the samples are available. The core objective of this metric learning is twofold: 1) Enhance Shared Feature Consistency: Promote high similarity between modality-sharee features irrespective of their originating modality, which is crucial to bridge the modality gap in mismatched retrieval scenarios. 2) Reinforee Specific Feature Purity: Ensure modality-speciffe features are distinet from each other and also orthogonal to all modality shared features. This preserves unique modality characteristics and prevents leakage between specific and shared representations.
To achieve these goals simultaneously, we define representation orthogonality loss, LROL . that minimizes the discrepancy between the computed pairwise similarities of the decoupled features and a predefined target similarity structure. Within our framework, we operate on the feature vector v and its availability mask mask generated for each sample, as defined previously. We compute the 6 x pairwise similarity matrix V sim based on the Tzz-nrmmlized vector u:
Vsim (i.j) of
T Uj for i,j E {L,.... 6}.
(II)
We then define the ideal target similarity matrix as:
1 0
0
0
0
1
0
0
0
0
0
1
0
0
0
A
(12)
0
0
1
0
0
0
1
0
0
1
Finally, the loss LROL, then formulated the sum of squared errors between the computed
similarities Vsim and the target similarities A, masked by Mpair:
LROL- ΣV Σ ((Vsim (1. A(t,)))²)
(13)
Additionally, the concept of triplet loss is incorporated further enhance the model's feature decoupling capability. In other words, the goal of feature decoupling is to achieve synergistic effect and prevent information from collapsing into modality-specifff representations. To this end, as shown in Fig. 2, we introduce knowledge discrepancy loss (KDL) to enforce the complementarity between shared and modality-specific features, ensuring that their combination yields better retrieval performance than using either alone. Specifically, within a batch of samples, given an anchor a, the positive samples Pa denotes the samples with the same label as a while the negative samples N₂ denotes the opposite samples. The distances using specific feature from the anchor a to the batch of positive or negative samples are computed as follows:
dg T) (() Ts(t)lla It),)
(14)
where T E (Pa, N.). S E (sp sh, sp, sh:
where S indicates the features used (modality -specifcc one, modality-shared one, or the combined one). Note that we omit the superscript representing the modality for simplicity, considering the modalities are completed, and the features of RGB, NIR, TIR modalities are concatenated to calculate the distance. Then, for the combination of shared and modality-specifff features, the maximum positive sample distance should be as small as possible compared to using a single type of feature, and the minimum negative sample distance should be large as possible compared to using single type of feature. As shown in Fig. 2, this formulation can be described as: max(dsp (), > max(dap p)), max (a, p)) and min sh n)) < min (a, n)); min(dss (a, n)). There- fore, we define the knowledge discrepancy loss LKDS as:
LKDL = D|p Oll1 + D| - 11/1,
max (a, p))
where Dp max (dep+ (a, p)) + max (dup (), + max(dat max(dds((a.p))
(15)
min(dep+eh (a, n))
D, min (dap-sh (a, n)) + min (dap (a, n)) + min(dgn n))
6

---

It is worth noting that detach operations are applied to dsp(a, n) and di (a, 12) to avoid gradient computation. Thus, minimizing D, reduces the farthest positive sample distance for the combined shared and modality-specifie features, while minimizing |,, 1|11 increases the nearest negative sample distance. Consequently, improving the retrieval capability of the combined features enhances the performance of model. In summary, the loss function for modality-aware metric learning is defined as follows:
LMML = w1 x LROL + W2 X LKDL.
(16)
3.4 Objective Function
As illustrated in Fig. 2. our model incorporates two primary loss functions: one for the VIT backbone and another for modality-awand aware metric learning. For the ViT backbone, adopt label smoothing cross-entropy loss Lee and triplet loss Lyrs following previous research to optimize the representation. In summary, the total loss function is as follows:
L Lee + Lort + LMML-
(17)
4 Experiments
4.1 Datasets and Evaluation Protocols
We evaluate the proposed MDReID framework on three publicly available datasets spanning both person (RGBNT201 [24]) and vehicle re-identification (RGBNT100 [8] and MSVR310 [26]) tasks. For evaluating modality mismatch scenarios, we focus on representative cross-modal configurations, including RT-to-NT, RT-to-N, R-to-N, and R-to-NT. Here, NIR is the second most widely used modality in surveillance systems after RGB, and RGB-to -GBR research continues attract significant interest. Therefore, evaluate our method in four R-to-N-based application scenarios. Performance measured using mean Average Precision (mAP) and Cumulative Matching Characteristics (CMC) at ranks 1, 5, and 10 (R-1, R-5, R-10). We also report key complexity indicators, including the number of trainable parameters and floating point operations (FLOPs), to quantify the efficiency of the model.
RGBNT201 [24] is a multi-modality person re-identification dataset designed to overcome the limitations of single-modal imaging in challenging surveillance scenarios. It was captured on university campus using four non-overlapping camera views with a synchronized triple-camera system that simultaneously records RGB, near-infrared (NIR), and thermal-infrare (TIR) images. The dataset contains 201 identities, each represented by at least 20 non-adjacent image triplets, resulting in 4,787 images per modality. The collection covers range of real-world conditions, including severe illumination changes, occlusion, viewpoint variations, and background clutter. The dataset is split into 141 identities for training, 30 for validation, and 30 for testing, with the entire test set serving as the gallery and 10 records per identity sampled as probes. RGBNT201 thus provides robust benchmark for evaluating multi-modal fusion strategies and addressing missing-modde issues in person re-identification.
RGBNT100 [8] contains 17,250 spatially aligned image triples from 100 vehicles captured under uniform conditions. Each triple includes RGB, NIR, and TIR images. Fifty vehicles (8,675 triples) are used for training, and the remaining 50 vehicles (8,575 triples) form the testing/gallery set, with 1,715 triples randomly selected as queries. The RGB and NIR images are recorded at a resolution of 1920x 1080, while the TIR images are captured at 640 x 480, all at consistent frame rate of 25 fps. TIR images provide thermal information that is robust to illumination changes, further enhancing the multi-spectral data for challenging vehicle re- re-identification scenarios.
MSVR310 [26] is high-quality multi-spectral vehicle re-identification benchmark captured under diverse, challenging conditions. It contains 2,087 triplet samples (6,261 images in total) from 310 vehicles. Each sample includes spatially aligned images from three modalities: RGB, NIR, and TIR. RGB images are captured by 360 D866 camera during the day and by Mi8 mobile phone at night, NIR images are obtained with the 360 D866 in near-infrared mode, and TIR images are recorded with a FLIR SC620 camera at 640x480 resolution. The number of samples per vehicle ranges from 2 to 20, and time labels are provided for cross-time matching. The dataset divided into 1,032 samples from 155 vehicles for training and 1,055 samples from 155 vehicles for testing/gallery, with 591 samples
7

---

Table 1: Performance comparison RGBNT201, RGBNT100, and MSVR310. The best and second results are in bold and underlined, respectively.
RGBNT201
RGBNT100
MSVR31O mAP
Method
mAP
R-1
R-5
R-10
Method
MAP
R-1
R-1
Single
MUDeep [39]
23.8
19.7
33.1
44.3
DMML [40]
58.5
82.0
19.1
31.1
HACNN [41]
21.3
19.0
34.1
42.8
BoT [42]
78.0
95.1
23.5
38.4
MLFN [43]
26.1
24.2
35.9
44.1
Circle Loss [44]
59.4
81.7
22.7
34.2
PCB [45]
32.8
28.1
37.4
46.9
HRCN [46]
67.1
91.8
23.4
44.2
OSNet [47]
25.4
22.3
35.1
44.7
TransReID [1]
75.6
92.9
18.4
29.6
CAL [48]
27.6
24.3
36.5
45.7
AGW [16]
73.1
92.7
28.9
46.9
Multi
HAMNet [8]
27.7
26.3
41.5
51.7
GAFNet [25]
74.4
93.4
PFNet [24]
38.5
38,9
52.0
58.4
GraFT [49]
76.6
94.3
JEEE [9]
47.5
44.4
57.1
63.6
GPFNet [28]
75.0
94.5
[27]
42.4
42.2
55.3
[35]
79.9
92.7
UniCat [36]
57.0
55.7
UniCat [36]
79.4
96.2
[10]
71.1
73.4
CCNet [26]
77.2
96.3
36.4
55.2
EDITOR [1I]
66.5
68.3
81.1
88.2
EDITOR [11]]
82.1
96.4
39.0
49,3
RSCNet [31]
68.2
72.5
RSCNet [31]
82.3
96.6
39.5
49.6
TOP-ReID [12]
72.3
76.6
84.7
89.4
TOP-ReID [12]
81.2
96.4
35.9
44,6
MDReID (Ours)
82.1
85,2
90.3
92.6
MDReID (Ours)
85.3
95.6
51.0
68.9
(from 52 vehicles) used as queries. MSVR310 provides comprehensive platform for addressing intra-class appearance variations and modality differences support robust vehicle re-identification.
4.2 Implementation Details
All experiments are implemented in Py Torch and conducted on a single NVIDIA RTX 4090 GPU with CUDA 12.5 and Python 3.8. We adopt the CLIP-Base [37] visual encoder the backbone. Input images are resized to 256 x 128 for RGBNT201 and 128 x 256 for RGBNT100 and MSVR310, We employ random horizontal flipping, cropping, and erasing [38] for data augmentation. The model optimized using Adam with a batch size of 64. The base learning rate is initialized at 3.5 x 10-1, while the visual encoder is fine-tuned with reduced rate of 10-6, Training is performed for 50 epochs.
4.3 Comparison with State-of-the-Art Methods
Multi-spectral Object ReID (RNT-to-RNT) We compare our MDRelD against range of state-of- the-art single-spectral and multi-spectral approaches on RGBNT201, RGBNT100, and MSVR310. For person RelD on RGBNT201, as shown in Table 1, the TOP-ReID [12] achieves 72.3% mAP, 76.6% R-1, 84.7% R-5, and 89.4% R-10. comparison, our MDRelD outperforms these results by 9.8%, 8.6%. 5.6%, and 3.2%, respectively, highlighting the effectiveness of proposed MDReID. For vehicle re- ID. MDRcID achieves the highest mAP on RGBNTI00 (85.3%) and surpasses the next- best approach on MSVR310 10 by significant margin of 1 1.5% in mAP and 13.7% in R-1. These results clearly demonstrate the superior discriminative power and robustness our modality-decouple design in multi-spectral re-ID scenarios where all modalities are available.
Evaluation on Missing-spectral Scenarios. In practical applications, sensor limitations or environ- mental factors often lead to incomplete modality inputs. Following the setting of TOP-ReID [12], we evaluate the robustness of our method under six possible missing-modality configurations across the RGB, NIR, and TIR channels using the RGBNT201 dataset. As shown in Table 2, our MDRelD consistently outperforms the TOP-ReID, achieving an average improvement of 10.0% in mAP and 9.4% in Rank-1 1 accuracy across all missing-modality scenarios. These results demonstrate that our decoopt framework remains highly effective even when partial modality information is absent, highlighting its strong generalization ability in incomplete multispectral settings.
Evaluation on Modality-mismatcher Scenarios. evaluate the generalizability our approach under realistic deployment conditions, we benchmark MDRelD across four representative modality- mismatched scenarios on three public datasets. We compare against two strong baselines, TOP- RelD [12] and EDITOR [11], by reproducing their results using official open-source implementations. As shown in Table 3, EDITOR designed exclusively for modality- settings, which exhibits limited effectiveness when modalities differ across query and gallery. Although TOP-ReID attempts
8

---

Table 2: Performance of missing-modality settings RGBNT201. "M (X)" means missing the X image modality. The best and second results are in bold and underlined, respectively.
Methods
MRRGB
M (NTR)
(TIR)
M NRGBENIR)
M
M (NIR-TIR)
Average
mAP
R-1
mAP
R-1
mAP
R-1
MAP
R-1
mAP
R-1
mAP
R-1
MAP
R-1
MUDeep [39
19.2
16.4
20.0
17.2
18.4
14.2
13.7
11.8
11.3
6.5
12.7
8,3
15.9
12.9
HACNN [41]
12.5
11.1
20.5
19.4
16.7
13.3
9.2
6.2
6.3
2.2
14.8
12.0
13.3
10.7
Single
MLFN [43]
20.2
18.9
21.1
19.7
17.6
11.1
13.2
12.1
8,3
3.5
13.1
9.1
15.6
12.4
PCB [45]
23.6
24.2
24.4
25.1
19.9
14.7
20.6
23.6
11.0
6.8
18.6
14.4
19.7
18.1
OSNet [47]
19.8
17.3
21.0
19.0
18.7
14.6
12.3
10.9
9.4
5.4
13.0
10.2
15.7
12.9
PFNet [24]
31,9
29.8
25.5
25.8
26.4
23.4
Muld
DENe [27]
35.4
36.8
33.0
35.4
32.4
29.2
TOP: ReID [12]
54.4
57.5
64.3
67.6
51.9
54.5
35.3
35.4
26.2
26.0
34.1
31.7
44.4
45.4
MDRReID (Ours)
67.0
67.9
75.8
80.9
61.7
59.8
48.6
51.1
31.4
29.2
42.1
40.0
54.4
54.8
Table 3: Performance of modality- misaaitte-mmmttthee Scenarios. We show the best average score in bold.
RT-to-NT
RT-to-N
R-to-N
R-to-NT
Average
Methods
mAP
R-1
mAP
R-1
MAP
R-1
mAP
R-1
mAP
R-1
EDITOR [1]]
27.3
27.9
2.80
0.0
4.0
2.3
4.3
4.1
8.5
7.5
RGBNT201
TOP-ReID [12]
43.0
44.6
14.4
13.3
15.4
14,0
11.9
8.7
18.2
18.0
MDRelD (Ours)
53.1
$1.7
16.7
13.8
16.6
11.1
15.1
10.9
21.6
19.1
EDITOR [1]]
42.1
59.9
2.6
0.8
2.8
1.5
2.7
1.5
11.9
15.5
RGBNT100
TOP-ReID [12]
59.0
81.8
21.9
28.5
26.2
34,0
21.7
25.4
26.8
36.1
MDReID (Ours)
69.4
85.5
39.6
47.9
45.4
56.4
37,6
41.8
38.6
47.4
EDITOR [1]]
6.4
11.0
2.2
2.5
1.6
0.2
1.7
1.5
2.5
3.4
MSVR310
TOP-ReID [12]
18.4
30,5
12.9
19.5
13.7
21.3
12.6
18.8
11.2
17.8
MDReID (Ours)
35.1
52.5
24.7
34.5
28.6
39.8
27.0
36.0
22.1
31.7
to address missing modalities via reconstruction, its performance remains constrained due to the ill-posed nature the reconstruction problem. contrast, our MDReID framework consistently improves the average mAP by 3.4%, 11.8%, and 10.9% the RGBNT201, RGBNT100, and MSVR310 datasets, respectively. In addition, we train four expert models on the RGBNT201 dataset for four specific scenarios. Results show that our method outperforms these experts by 9.2% on average in mAP and by 8.9% in R-1. Importantly, our single-model approach adapts to all four scenarios and offers greater flexibility. In summary, these results highlight the robustness and adaptability of MDReID in handling modality mismatty- ReID, affirming its effectiveness capable of supporting arbitrary modality combinations in both query and gallery inputs.
4.4 Ablation Study
To evaluate the effectiveness of each component in our proposed MDReID framework, we conduct comprehensive ablation studies the RGBNT201 dataset. To ensure a thorough evaluation across both modality-maten
and modality-mismatchen scenarios, we report the average performance over 8 evaluation settings, including RNT-to-RNT, RT-to-RU RT, RT-to-NT, RT-to-N, R-to-N, and R-to-NT. The detailed results are presented in Table 4 (a). Note that configuration "I" corresponds to using single classifier for all modalities, while configuration "3" corresponds assigning separate classifier to each modality. Ablation results show that, compared with one classifier, modality-specinf classifiers effectively improve performance, increasing mean mAP by 13.4% and mean R-1 by 13.7%.
Modality Decoupled Learning Our base framework integrates a shared vision transformer. uti- lizing label smoothing cross-entropy along with triplet loss for optimization. To better address modality-mismatcher object re-identification, we design a modality decoupling learning (MDL) that separates features into shared and specific representations. Experimental results demonstrate that this module boosts performance, with an average increase of 11.5% in mAP and 11.1% in Rank- 1. This demonstrates that modality decoupling not only benefits re-identification in mismatched settings but also enhances retrieval performance in matched scenarios.
Modality-aware Metric Learning To further decouple shared and specific features, we design modality-awan metric learning (MML) loss function. This loss consists of two components: the first aligns shared features across multiple modalities, while the second enhances the decoupling of features based on the aligned representation. Experimental results show that introducing ROL improves average MAP and Rank- by 1.8% and 2.7%, respectively. Similarly, introducing KDL. increases mAP by 0.5% and Rank-1 by 1.7% When both components are combined, the average mAP and Rank-1 improve by 3.8% and 4.1 1%, respectively. These results indicate that ROL significantly

---

Table Ablation study and hyper-parameter settings of MDReID. We show the best average score in bold.
(a) Ablation study. MDL,, LOO,, and TXDL indi- cate the Modality Decoupled I Learning, Represen- tation Orthogonality Loss (ROI)), and Knowledge Discrepancy Loss (KDL). respectively.
(b) Performance under different w1- The opti- mal performance achieves when wi is to 1.5.
(c) Performance under different w2- The opti- mal performance achieves when wa is set to 5.25.
Index
MDL
LROL
LKDL
mAP
R-1
w1
W2
mAP
R-1
X
27.8
0.5
39.6
38.1
4.5
41.7
41.1
2
X
X
39,4
38.2
1.0
40.3
40.0
5.0
40.2
39.6
3
J
X
41.2
40.8
1.5
41.2
40.8
5.25
43.2
42.3
V
39,9
40.9
2.0
38.9
38.1
5.5
41.6
41.3
5
V
43.2
42.3
3.0
38.2
37.0
6.0
41.9
40.4
enhances re-identification accuracy in mismatch scenarios. Moreover, while KDL alone provides limited improvement, its combination with ROL effectively boosts overall model performance.
Modality Decoupled Learning We design the KDL objective to enforce a knowledge gap between specific and shared features. Without KDL, the model may degrade into relying only on shared features. Therefore, we ensure that combining specific and shared features yields higher retrieval performance than using either feature alone. To achieve this, we apply formulation 15 to impose distance constraints in the feature space based on the described relations. As shown Table (a), index 3 and 5, adding KDL improves mAP by 2.0% and Rank-1 by 1.5% across multiple scenarios. In summary, the MDReID framework, comprising the modality decoupled learning and modality-aware metric learning, achieves average performance of 43.2% mAP, 42.3% Rank-1, 50.2% Rank-5, and 54.9% Rank-10 across various scenarios, including both modality- matched and modality-minn mismatheed cases. Its modular design effectively decouples shared and specific features, enhancing object re- identification accuracy under multi-spectral conditions while maintaining robust adaptability across diverse modalities.
4.5 Discussions
Hyper-parameter settings of MDReID In Eq. (16), we introduce two hyperparameters w1 and W2 to balance the importance of ROL and KDL. Therefore, this part evaluates the performance under different hyperparameter settings and shows the results Table 4. We first vary wi without KDL, observing that the model achieves optimal performance w1 1.5, highlighting the importance of enforcing orthogonality between shared and specific components. Subsequently. with w1 = 1.5, we tune uy and find that the best results obtained at wa 5.25, confirming the benefit of encouraging representational synergy and complementarity. These observations validate that a contribution from both ROL and KDL is crucial for maximizing retrieval accuracy.
5 Conclusion
In this paper, we propose MDRelD, an image-level any-to-any object re-identification framework that supports retrieval across arbitrary query-gallery modality combinations. Our approach leverages Learning (MDL) to explicitly disentangle modality-sharge -shrred and modality-specifie features, while Modality-aware Metrie Learning (MML) refines the feature space enhance discrim- ination. Extensive evaluations on the RGBNT201, RGBNT100, and MSVR310 datasets validate MDRelD's superior performance both modality-matchie and modality-mismatoon scenarios, demonstrating its practical potential for any-to-an RelD tasks.
Acknowledgements. This research was supported in part by the China Postdoctoral Science Foun- dation under Grant Number 2025M77158 and 0025T180433 and the National Natural Science Foundation of China under Grant 625007198.
A comprehensive analysis of the model's computational complexity (Sec. A.1), along with additional visualization results (Sec. A.4), is provided in the supplementary material.
10

---

References
[1] Shuting He, Hao Luo, Pichao Wang, Fan Wang, Hao Li, and Wei Jiang. Transreid: Transformer- based object re-identification. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 15013-15022, October 2021.
[2] Wen Li, Cheng Zou, Meng Wang, Furong Xu, Jianan Zhao, Ruobing Zheng, Yuan Cheng, and Wei Chu. De-former: Diverse and compact transformer for person re-identification. In Proceedings of the AAAI Conference Artificial Intelligence, volume 37, pages 1423, 2023.
[3] Guiwei Zhang, Yongfei Zhang, Tianyu Zhang, Bo Li, and Shiliang Pu. Pha: Patch-wise high- frequency augmentation for transformer-basee person re-identification. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 14133-14142, 2023.
[4] Siyuan Li, Li Sun, and Qingli Li. Clip-reid: exploiting lision-language model for image re- identification without concrete text labels. In Proceedings of the AAAI conference on artificial intelligence, volume 37, pages 1405-1413, 2023.
[5] Guan an Wang, Xiaowen Huang, Jitao Sang, et al. Denoiserep: Denoising model for repre- sentation learning. Advances in Neural Information Processing Systems, 37:40032-40053 2024.
[6] Lei Tan, Pingyang Dai, Rongrong Ji, and Yongjian Wu. Dynamic prototype mask for occluded person re-identification. In Proceedings of the 30th ACM international conference on multimedia, pages 531-540, 2022.
[7] Lei Tan, Pingyang Dai, Jie Chen, Liujuan Cao, Yongjian Wu, and Rongrong Ji. Partformer: Awakening latent diverse representation from vision transformer for object re-identification. arXiv preprint arXiv:2408. /6684, 2024.
[8] Hongchao Li, Chenglong Li, Xianpeng Zhu, Aihua Zheng, and Bin Luo. Multi-spectral vehicle re-identificatioo:: A challenge. Proceedings of the AAAI Conference on Artificial Intelligence, 34407):11345-11333 Apr. 2020.
[9] Zi Wang, Chenglong Li, Aihua Zheng, Ran He, and Jin Tang. Interact, embed, and enlarge: Boosting modality-specific representations for multi-modal person re- re-identification. Proceed- ings of the AAAI Conference on Artificial Intelligence, 36(3):2633 266,-26,,,,, Jun. 2022.
[10] Zi Wang. Huaibo Huang, Aihua Zheng, and Ran He. Heterogeneous test-time training for multi- modal person re-identification. Proceedings of the AAAI Conference on Artificial Intelligence, 5856-5850-58888 Mar. 2024.
[11] Pingping Zhang, Yuhao Wang, Yang Liu, Zhengzheng Tu, and Huchuan Lu. Magic tokens: Select diverse tokens for multi-modal object re-identification. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 17117-17126. June 2024.
[12] Yuhao Wang, Xuehu Liu, Pingping Zhang, Hu Lu, Zhengzheng Tu, and Huchuan Lu. Top- reid: Multi-spectral object re- ce-identification with token permutation. Proceedings of the AAAI Conference on Artificial Intelligence, 38(6):5758 5766, Mar. 2024.
[13] Yingying Feng, Jie Li, Chi Xie, Lei Tan, and Jiayi Ji. Multi-modal object re-identification via sparse mixture-of-experts. In Ferty-second International Conference on Machine Learning.
[14] Lei Tan, Yukang Zhang, Keke Han, Pingyang Dai, Yan Zhang, Yongjian Wu, and Rongrong Ji. Rle: A unified perspective of data augmentation for spectrll re-identification. Advances Neural Information Processing Systems, 37: 1699771 2024.
[15] Yulin Li, Jianfeng He, Tianzhu Zhang, Xiang Liu, Yongdong Zhang, and Feng Wu. Diverse part discovery: Occluded person re- i--intiffication with part-aware transformer. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2898-2907, 2021.
[16] Mang Ye, Jianbing Shen. Gaojie Lin, Tao Xiang, Ling Shao, and Steven C. H. Hoi. Deep learning for person re-identificatioon: A survey and outlook. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(6):2872-2893 2022.
[17] Chuanming Wang, Yuxin Yang, Mengshi Qi, Huanhuan Zhang, and Huadong Ma. Towards effi- cient object re-identification with novel cloud-edge collaborative framework. Proceedings of the AAAI Conference Artificial Intelligence, volume 39, pages 7600-7608, 2025.
11

---

[18] Lei Tan, Jiaer Xia, Wenfeng Liu, Pingyang Dai, Yongjian Wu, and Liujuan Cao. Occluded person re-identification via saliency-guided patch transfer. In Proceedings of the AAAI conference on artificial intelligence, volume 38, pages 5070-5078, 5078, 2024.
[19] Yizhen Jia, Rong Quan, Yue Feng, Haiyan Chen, and Jie Qin. Doubly contrastive learning for source-free domain adaptive person search. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 3949-3957, 2025.
[20] Ziniu Yin, Yanglin Feng, Ming Yan, Xiaomin Song, Dezhong Peng, and Xu Wang. Roda: Robust domain alignment for cross-domain retrieval against label noise. In Proceedings of the
AAAI Conference on Artificial Intelligence, volume 39, pages 9535-9543, 2025.
[21]
Haoran Liu, Ying Ma, Ming Yan, Yingke Chen, Dezhong Peng, and Xu Wang. Dida: Disam- biguated domain alignment for cross-domain retrieval with partial labels. In Proceedings of the AAAI conference on artificial intelligence, volume 38, pages 3612-3620, 2024.
[22] Chao Su, Huiming Zheng, Dezhong Peng, and Xu Wang. Dica: Disambiguated contrastive alignment for cross-modal retrieval with partial labels. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 20610-20618, 2025.
[23] Yukang Zhang and Hanzi Wang. Diverse embedding expansion network and low-light cross- modality benchmark for visible-infrared person re- re-identification. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2153-2162, 2023.
[24] Aihua Zheng, Zi Wang, Zihan Chen, Chenglong Li, and Jin Tang. Robust multi modality person re-identification. Proceedings of the AAAI Conference on Artificial Intelligence, 35(4):3529-
3537, May 2021.
[25] Jinbo Guo, Xiaojing Zhang, Zhengyi Liu, and Yuan Wang. Generative and attentive fusion for multi-spectral vehicle re- identification. In 2022 7th International Conference on Intelligent Computing and Signal Processing (ICSP). pages 1565-1572, 2022.
[26] Aihua Zheng, Xianpeng Zhu, Zhiqi Ma, Chenglong Li, Jin Tang, and Jixin Ma. Multi-spectral ve- hicle re-identification with cross-directional consistency network and a high-quality benchmark, 2023.
[27] Aihua Zheng, Ziling He, Zi Wang, Chenglong Li, and Jin Tang. Dynamic enhancement network for partial multi-modality person re-identificattinn, 2023.
[28] Qiaolin Hc, Zefeng Lu, Zihan Wang, and Haifeng Hu. Graph-based progressive fusion network for multi-modality vehicle re-identification IEEE Transactions Intelligent Transportation 2023.
Systems, 24(1I):
[29] Mang Ye, Shuoyi Chen, Chenyue Li, Wei-Shi Zheng. David Crandall, and Bo Du, Transformer for object re-identification:: A survey. International Journal of Computer Vision, pages 1-31, 2024.
[30] Yingquan Wang, Pingping Zhang. Dong Wang, and Huchuan Lu, Other tokens matter: Exploring global and local features of vision transformers for object re-identification. Computer Vision and Image Understanding, 244:104030, 2024.
[31] Zhi Yu, Zhiyong Huang, Mingyang Hou, Jiaming Pei, Yan Yan, Yushi Liu, and Daming Sun. Representation selective coupling via token sparsification for multi-spectral object re- identification. IEEE Transactions on Circuits and Systems for Video Technology. 2024.
[32] Yanping Li, Yizhang Liu, Hongyun Zhang, Cairong Zhao, Zhihua Wei, and Duogian Miao. Occlusion-aware transformer with second-order attention for person re-identification. IEEE Transactions on Image Processing, 2024.
[33] Kuan Zhu, Haiyun Guo, Shiliang Zhang, Yaowei Wang, Jing Liu, Jinqiao Wang, and Ming Tang. Aaformer: Auto-aligned transformer for person re-identification. IEÉE Transactions on Neural Networks and Learning Systems, 2023.
[34] Wenjie Pan, Hanxiao Wu, Jianqing Zhu, Huanqiang Zeng, and Xiaobin Zhu. H-vit: Hybrid vision transformer for multi- vehicle re-identification. In Lu Fang, Daniel Povey, Guangtao Zhai, Tao Mei, and Ruiping Wang, editors, Artificial Intelligence, pages 255-267, Cham, 2022. Springer Nature Switzerland.
[35] Wenjie Pan, Linhan Huang, Jianbao Liang, Lan Hong, and Jianqing Zhu. Progressively hybrid transformer for multi-mode vehicle re-identification. Sensors, 23(9), 2023,
12

---

[36] Jennifer Crawford, Haoli Yin, Luke McDermott, and Daniel Cummings. Unicat: Crafting a stronger fusion baseline for multimodal re-identification, 2023.
[37] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 8748-8763, PMLR, 18-24 Jul 2021.
[38] Zhun Zhong, Liang Zheng, Guoliang Kang, Shaozi Li, and Yi Yang. Random erasing data augmentation. Proceedings of the AAAI Conference Artificial Intelligence, 3407)) 34(07):13001- 1 13008, Apr. 2020.
[39] Xuelin Qian, Yanwei Fu, Yu-Gang Jiang, Tao Xiang, and Xiangyang Xue. Multi-scale deep learning architectures for person re-identification. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), Oct 2017.
[40] Guangyi Chen, Tianren Zhang, Jiwen Lu, and Jie Zhou. Deep meta metric learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2019.
[41] Wei Li, Xiatian Zhu, and Shaogang Gong, Harmonious attention network for person re- identification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recog- nition (CVPR), June 2018.
[42] Hao Luo, Youzhi Gu, Xingyu Liao, Shenqi Lai, and Wei Jiang. Bag of tricks and strong baseline for deep person re-identification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2019.
[43] Xiaobin Chang, Timothy M. Hospedales, and Tao Xiang. Multi-level factorisation net for person re-identification.
Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition (CVPR), June 2018.
[44] Yifan Sun, Changmao Cheng, Yuhan Zhang, Chi Zhang, Liang Zheng, Zhongdao Wang, and Yichen Wei. Circle loss: A unified perspective of pair similarity optimization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
[45] Yifan Sun, Liang Zheng, Yi Yang, Qi Tian, and Shengjin Wang. Beyond part models: Person retrieval with refined part pooling (and strong convolutional baseline). In Proceedings of the European Conference on Computer Vision (ECCV), September 2018.
[46] Jiajian Zhao, Yifan Zhao, Jia Li, Ke Yan, and Yonghong Tian. Heterogeneous relational complement for vehicle re-identification. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 205-214, October 2021.
[47] Kaiyang Zhou, Yongxin Yang, Andrea Cavallaro, and Tao Xiang. Omni-scale feature learning for person re-identification. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2019.
[48] Yongming Rao, Guangyi Chen, Jiwen Lu, and Jie Zhou. Counterfactual attention learning for fine-grained visual categorization and re-identification. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 1025-1034, October 2021.
[49] Haoli Yin, Jiayao Li, Eva Schiller, Luke McDermott, and Daniel Cummings. Graft: Gradual fusion transformer for multimodal re-identification, 2023.
[50] Yukang Zhang and Hanzi Wang. Diverse embedding expansion network and low-light cross- modality benchmark for visible-infrared person re-identification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2153- 2162, June 2023.
13

---

A Supplemental Material
A.1 Analysis of model computational complexity
Table 5 compares the computational cost of our method with recent state-of-the-arr approaches. MDRelD achieves the most efficient design, requiring only 57.2M parameters and 22.3G FLOPs, which is significantly lower than HTT (85.6M/ 33.1G), EDITOR (117.5M / 38.6G), and TOP-Rell (OP-RelI (278.2M/ 2 34.5G). Despite efficient architecture, MDReID consistently outperforms these heavier models in accuracy, demonstrating superior trade-off between performance and efficiency. This highlights the effectiveness of our MDL and MML designs, which provide robust alignment in all kinds of query-gallery combinations without incurring large computational overhead. Furthermore, we evaluate the total feature generation and retrieval times of different models on the RGBNT201 test set under the RNT-to-RNT, RT-to-NT, and R-to-N scenarios. Theoretically, method reduces retrieval cost compared to Top-ReID. For example, the RT-to-NT scenario, we only perform T-to-T (specific features) and R-to-T (shared features) matching, whereas Top-RelI) first generates -asiiitt features before conducting a full RNT-to-RNT matching. Experimental results confirm this, showing that method incurs additional time and generates features faster.
Table 5: Comparison computational cost with recent methods. We show the best result in bold.
RNI-to-RNT
RT-o-NT(ss)
R-to-N(s)
Methods
Params(M)
Glops(G)
Extract
Retrieval
Extract
Retrieval
Extract
Retrieval
HTT [10]
85.6
33.1
8.32
0.12
EDITOR [1]
117.5
38.6
31.12
0.12
30.99
0.12
30.98
0.12
TOP-ReID (12)
278.2
34.5
71.78
0.12
73.32
0.11
73.71
0.12
MDRelD (Ours)
$7.2
22.3
3.27
0.14
2.90
0.11
3.14
0.12
A.2 Analysis of specific and shared features
In MDReID, both shared and modality-specifft features are essential. For the RT-to-NT task, using only shared features overlooks the unique characteristics of the T modality, while using only modality- specific features prevents valid R-to-N matching. To quantify this, we evaluate RT-to-NT performance under three settings: shared features only, modality-specific features only, and both combined. As shown in Table 6, combining shared and modality-specific features yields the best performance.
Table 6: Performance analysis under different features. We show the best result in bold.
Feature
mAP
R-1
Specific feature
52.0
50.5
Shared feature
52.7
51.2
Specific and shared feature
53.1
51.7
A.3 Comparison with cross-modality method
We compare our method with the cross-modality object re- re-identification model DEEN [50] on the RGBNT201 dataset under the R-to-N scenario. Table shows that even when DEEN is specially trained for R-to-N, its mean precision is 3.5% lower than ours. Notably, our model demonstrates stronger generalization across diverse application scenarios.
Table 7: Comparison with cross-modality method, We show the best result in bold.
Feature
mAP
R-1
Average
DEEN [50]
11.5
9.2
10.35
Ours
16.6
11.1
13.85
A.4 Visualization
To better illustrate the impact different components on feature disentanglement, we visualize the learned mopelity-specific and modality-shared features across RGB, NIR, and TIR using t-SNE, as shown in Figure 3. Notably, the baseline method does not use MDI., so does not separate specific and shared features and only outputs fused features, hence corresponding visualization.
14

---

DEXOLD DEIL HOOL
DELL
Figure 3: The visualization of the ablation study. Triangles, circles, and X symbols correspond to the RGB, NIR, and TIR modalities, respectively. Black borders highlight shared features for clarity. Without ROL, distinguishing between shared and modality-specifie features is challenging, but its introduction effectively clusters the shared features.
Specifically, in (a), with only MDL. applied, the features exhibit noticeable overlap, and the separation between modality-speciffe and shared components remains unclear. In (c), the addition KDL slightly improves the clustering and separation of shared and specific features. In contrast, (b) shows that introducing ROL significantly enhances feature orthogonality, forming clearer boundaries between shared and sppeeic clusters. Finally, (d) demonstrates that the combination of both ROL and KDL yields the most structured and disentangled feature space, with shared features tightly clustered and rell-separated from their modality-specifie counterparts. These results confirm the complementary roles of ROL and KDL in refining the representation space.
A.5 Limitations and Broader Impact
Although our work is the first explore object re-identification under the uncertain query-gallery modality setting and achieves significant performance improvements, the current accuracy is still insufficient for deployment real-world applications. Furthermore, existing datasets are limited in both scale and modality diversity, making unclear how well MDReID would generalize to larger and more complex datasets involving broader range modality types. Nevertheless, MDRelD represents the first effort to address the any-to-any object re-identification task, marking significant step toward this emerging direction. We believe our work will inspire new perspectives and challenges in the re-identification community.
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction of this paper accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We have discussed the limitations of this work in Section A.5.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
16

---

Justification:
We have introduced all the details in our theoretical result.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: This paper has provided detailed information for reproducing the experimental results.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
17

---

Answer: [Yes]
Justification:
The code is available at: https//giinubb.oom//ooocc//22221
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (uttps://nips.cm cc/
mublic/guides Stwiidde//odeeummissiisss.. ceaminssssooooooo
onioliccyy oolicylyyyy)oo for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https:
onPolicy) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: This paper has specified all the training and test details, such as dataset and hyperparameters.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: This paper does not provide error bars due to the limited resources. Also, previous methods do not provide error bars either.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
18

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: This paper has provided sufficient information on the computer resources.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: //te://nneuiseeeeeee ctt.c//c........ccccooooooo sec/public/Etiiee csGuidel de ines?
Answer: [Yes]
Justification:
This paper meets the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: This paper has societal impact to the best of our knowledge.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
19

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: This paper poses such risks since it does not release data models that have risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: This paper has properly cited the original paper that produced the code package or dataset.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URI
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in package should be provided. For popular datasets, paperswi twwode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
20

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
This paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moddl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Bthies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
21

---

16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: This paper only uses LLMs for writing and editing.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to L.LM policy (https: / http//uurrs.cccee// http:/pss..///oeeeeee//////////e /LLM) for what should or should not be described.
22


Reviews:
[{"review_id": "0p7pCXRdko", "reviewer": "Reviewer_Zmjf", "summary": "This paper proposes MDReID framework that addresses any-to-any multi-modal object ReID task, enabling robust re-identification even when query and gallery images come from different modalities. The proposed framework is built on the core idea of decoupling image features into two components: modality-shared features that are consistent across different modalities, and modality-specific features that preserve unique characteristics. This is achieved through two main contributions: Modality-Decoupled Learning (MDL), which uses dedicated tokens in a Vision Transformer to disentangle the features, and Modality-aware Metric Learning (MML), which applies specialized losses to refine the separation. Specifically, MDL extracts modality-shared and -specific features and then compute the robust similarity based on extracted features and availability masks. MML promotes further disentangled & discriminative features through a representation orthogonality loss (ROL) and a knowledge discrepancy loss (KDL). Experiments on three benchmarks show the proposed method approach outperforms state-of-the-art methods, particularly in modality-mismatched scenarios.", "questions": "Please address my concerns, especially weaknesses 1, 2, and 3.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\n1. The tackled problem, any-to-any multi-modal object ReID, is interesting considering the practicality for real-world applications.  \n2. Overall, the key idea of the paper is reasonable and the proposed method to achieve this is logical and well-designed (especially ROL & KDL).\n\n**Weaknesses**\n\n1. The comparison to state-of-the-art methods seems potentially unfair due to the use of a \"modality availability mask\" during inference. This mask provides explicit information about the modality scenario for both the query and gallery, an advantage not available to the baseline methods. This calls into question whether the reported performance gain stems from the novelty of the modality-decoupling approach itself, or simply from leveraging this additional information. To provide a fairer assessment, the paper should compare MDReID against an alternative strategy that also utilizes this mask. For instance, one could train an ensemble of separate \"expert\" models, each specialized for a specific modality combination (e.g., an R-to-N model, an RT-to-NT model). At inference time, the availability mask could then be used to select and deploy the appropriate expert model for the given scenario. A comparison against such a baseline would better isolate the true contribution of the unified MDReID framework over a simpler, scenario-specific approach.\n2. While the paper compares computational costs in terms of parameters and FLOPs, it omits the retrieval cost (i.e., search time), which is a critical metric for ReID systems. The proposed similarity calculation is more complex than a standard feature dot product. Therefore, a comparison of the actual search time against baselines is necessary to fully evaluate the practical efficiency of the proposed method in large-scale gallery settings.\n3. To provide stronger intuition for the proposed feature decoupling, a more detailed ablation study is needed. The paper should analyze the individual contributions of the modality-shared and modality-specific features. For instance, what is the performance when using only the shared features or only the specific features during inference? This analysis should be conducted for both modality-matched and mismatched scenarios to clearly demonstrate the respective roles of the decoupled components. Also, the ablation studies of these individual features with different loss functions would be also helpful.\n4. (minor) Notations throughout the method is confusing and lacks rigor.", "was_revised": true, "final_justification": "Most of my concerns have been addressed during the rebuttal period. Also, I like the potential practical value of the paper. After reading the rebuttal, I'd like to raise my score to 4.", "rebuttal": "Thanks for your constructive feedback. Below, we respond to your key concerns point by point.\n> **Q1:** The comparison to state-of-the-art methods seems potentially unfair due to the use of a \"modality availability mask\" during inference. This mask provides explicit information about the modality scenario for both the query and gallery, an advantage not available to the baseline methods. This calls into question whether the reported performance gain stems from the novelty of the modality-decoupling approach itself, or simply from leveraging this additional information. To provide a fairer assessment, the paper should compare MDReID against an alternative strategy that also utilizes this mask. For instance, one could train an ensemble of separate \"expert\" models, each specialized for a specific modality combination (e.g., an R-to-N model, an RT-to-NT model). At inference time, the availability mask could then be used to select and deploy the appropriate expert model for the given scenario. A comparison against such a baseline would better isolate the true contribution of the unified MDReID framework over a simpler, scenario-specific approach.\n> \n\n**A1:** Thank you for your feedback. The use of the modality availability mask is justified, since explicit information about the modality scenario is defaulted as available information and used by all methods. For example, the Complementary Reconstruction Module (CRM) in Top-ReID introduces dense token-level reconstruction constraints to reduce the distribution gap across different image spectra. As a result, it predicts unavailable modality information by leveraging available modalities, representing the current state-of-the-art approach. To this end, TOP-ReID needs to know the explicit information about the modality scenario, and then it can apply the specific generator for reconstruction (e.g., R->T). Meanwhile, following your suggestion, we train four expert models for the RT-to-NT, RT-to-N, R-to-N, and R-to-NT scenarios. Table A shows that our method outperforms these experts by 9.2% on average in mAP and 8.9% in R-1. Furthermore, our single-model approach adapts to all four scenarios, offering greater flexibility. Although Top-ReID also handles multiple scenarios, our method surpasses it by 3.4% in average mAP and 1.1% in average R-1.\n\nTable A: Performance analysis of expert model, Top-ReID, and Ours.\n\n| Scenarios | RT-to-NT |  | RT-to-N |  | R-to-N |  | R-to-NT |  | Average |  |\n| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |\n|  | mAP | R-1 | mAP | R-1 | mAP | R-1 | mAP | R-1 | mAP | R-1 |\n| Expert model | 37.8 | 36.4 | 4.1 | 2.5 | 3.8 | 1.7 | 4.1 | 1.8 | 12.4 | 10.2 |\n| Top-ReID | 43.0 | 44.6 | 14.4 | 13.3 | 15.4 | **14.0** | 11.9 | 8.7 | 18.2 | 18.0 |\n| Ours | **53.1** | **51.7**  | **16.7** | **13.8** | **16.6** | 11.1 | **15.1** | **10.9** | **21.6** | **19.1** |\n\n> **Q2:** While the paper compares computational costs in terms of parameters and FLOPs, it omits the retrieval cost (i.e., search time), which is a critical metric for ReID systems. The proposed similarity calculation is more complex than a standard feature dot product. Therefore, a comparison of the actual search time against baselines is necessary to fully evaluate the practical efficiency of the proposed method in large-scale gallery settings.\n> \n\n**A2:**  Following your suggestion, we further evaluate the total feature generation and retrieval times of different models on the RGBNT201 test set under the RNT-to-RNT, RT-to-NT, and R-to-N scenarios. Theoretically, our method reduces retrieval cost compared to Top-ReID. For instance, in the RT-to-NT scenario, our approach only matches T-to-T (specific features) and R-to-T (shared features), whereas Top-ReID first generates the missing modality features before performing a full RNT-to-RNT matching. Table B confirms this, showing that our method incurs no additional time and generates features more rapidly.\n\nTable B: Performance analysis under different methods.\n\n| Methods  | Params(M) | Flops(G) | RNT-to-RNT (s) |  | RT-to-NT (s) |  | R-to-N (s) |  |\n| --- | --- | --- | --- | --- | --- | --- | --- | --- |\n|  |  |  | extract | retrieval | extract | retrieval | extract | retrieval |\n| HTT | 85.6  | 33.1 | 8.32 | **0.12** | - | - | - | - |\n| EDITOR | 117.5 | 38.6 | 31.12 | **0.12** | 30.99 | 0.12 | 30.98 | **0.12** |\n| TOP-ReID | 278.2  | 34.5 | 71.78 | **0.12** | 73.32 | **0.11** | 73.71 | **0.12** |\n| Ours  | **57.2**  | **22.3** | **3.27** | 0.14 | **2.90** | **0.11** | **3.14** | **0.12** |\n\n> **Q3:** To provide stronger intuition for the proposed feature decoupling, a more detailed ablation study is needed. The paper should analyze the individual contributions of the modality-shared and modality-specific features. For instance, what is the performance when using only the shared features or only the specific features during inference? This analysis should be conducted for both modality-matched and mismatched scenarios to clearly demonstrate the respective roles of the decoupled components. Also, the ablation studies of these individual features with different loss functions would be also helpful.\n> \n\n**A3:**  Thank you for your suggestion. In our model, both shared and modality-specific features are essential. In the RT-to-NT task, using only shared features overlooks the unique characteristics of the T modality, while using only modality-specific features prevents valid R-to-N matching. To quantify this, we evaluate RT-to-NT performance under three settings: shared features only, modality-specific features only, and both combined. As shown in Table C, combining shared and modality-specific features yields the best performance.\n\nTable C: Performance analysis under different features.\n\n| RT→NT | mAP | R-1 |\n| --- | --- | --- |\n| specific feature | 52.0 | 50.5 |\n| shared feature | 52.7 | 51.2 |\n| specific and shared feature | **53.1** | **51.7** |\n\n> **Q4:** (minor) Notations throughout the method is confusing and lacks rigor.\n> \n\n**A4:** Thank you for the suggestion. We have revised the notations throughout the manuscript to improve readability in recent days. If you have any recommended parts, please provide them in the discussion. We are willing to further clarify them."}, {"review_id": "OS2OT4rM0Y", "reviewer": "Reviewer_FcJV", "summary": "MDReID is a novel framework for multi-modal object re-identification that supports any-to-any retrieval across both modality-matched and modality-mismatched scenarios. To tackle the challenges of modality discrepancy, MDReID introduces a Modality-Decoupled Learning (MDL) strategy to explicitly separate modality-shared and modality-specific representations. In addition, a Modality-aware Metric Learning (MML) strategy is proposed, which enhances this disentanglement through a representation orthogonality loss and a knowledge discrepancy loss, encouraging the two components to capture distinct yet complementary information. The framework is comprehensively evaluated under both modality-aligned and modality-misaligned settings to demonstrate its effectiveness.", "questions": "1.\tThe any-to-any setting is highly related to both modality-matched object re-identification and cross-modality object re-identification. However, this work only considers the modality-matched side. Therefore, comparison and discussion with cross-modality methods will also be useful.\n2.\tDuring the retrieval phase. From the paper, the author considers the shared representations equally, whether the shared feature from the same modality. From the visualization results, the shared feature.\n3.\tFigure 3 lacks the visualization result of the baseline method. It will make audience hard to get the improvement of MDL. The same issue applies to Table 5.\n4.\tIt would be helpful to clarify how the 4 testing scenarios under modality-mismatched conditions were selected.", "limitations": "Yes", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Pros:\n-\tThis paper introduces the novel task of any-to-any multi-modal object re-identification, which supports flexible retrieval under arbitrary query-gallery combinations across three different spectral modalities.\n-\tCompared to the previous works, the proposed method achieves a significant improvement in all the testing protocols.\nCons:\n-\tThe any-to-any setting is also highly related to the cross-modality re-identification task. However, this paper may ignore this part.  \n-\tSome details should be further explained. Some important details require further clarification. For example, what is the computational cost of the baseline method? Additionally, how the 4 testing scenarios under modality-mismatched conditions were selected?", "was_revised": true, "final_justification": "The authors have addressed all my concerns with quantitative evidence and clear explanations.", "rebuttal": "> **Q1:** The any-to-any setting is highly related to both modality-matched object re-identification and cross-modality object re-identification. However, this work only considers the modality-matched side. Therefore, comparison and discussion with cross-modality methods will also be useful.\n> \n\n**A1:**  Thank you for your feedback, and we incorporated additional discussions on cross-modality object re-identification. In this experiment, we evaluate the DEEN [1] algorithm on the RGBNT201 dataset under the R-to-N scenario. As shown in Table A, our results demonstrate that even with specialized training for R-to-N, DEEN achieves an average precision 3.5% lower than our method. Notably, our model exhibits stronger generalization capability across diverse application scenarios.\n\nTable A: Comparison of DEEN and Our Method in R-to-N Scenario.\n\n| R→N | mAP | R-1 | Avg |\n| --- | --- | --- | --- |\n| DEEN  | 11.5 | 9.2 | 10.35 |\n| Ours | **16.6** | **11.1** | 13.85 |\n\n[1] Diverse embedding expansion network and low-light cross-modality benchmark for visible-infrared person re-identification. CVPR 2023.\n\n> **Q2:** During the retrieval phase. From the paper, the author considers the shared representations equally, whether the shared feature from the same modality. From the visualization results, the shared feature.\n> \n\n**A2:** Thanks for your valuable suggestion. We have evaluated the performance under different weight combinations of specific features (w_sp) and shared features (w_sh), and showed the result below. The experimental results indicate that adjusting the weights does indeed lead to a slight improvement. Specifically, assigning a higher weight to specific features benefits retrieval in modality-matched scenarios, while a higher weight to shared features helps in modality-mismatched retrieval. We will include a detailed discussion on parameter settings in a future version to update the relevant experimental sections.\n\nTable B: Analysis of different weights.\n\n| (w_sh,w_sp) | RNT→RNT |  | RT→NT |  | RT→N |  | Avg |  |\n| --- | --- | --- | --- | --- | --- | --- | --- | --- |\n|  | mAP | R-1 | mAP | R-1 | mAP | R-1 | mAP | R-1 |\n| ours | 82.1 | 85.2 | 53.1 | 51.7 | 16.7 | 13.8 | 50.63 | 50.23 |\n| 1.0,0.0 | 79.6 | 83.5 | 51.9 | 51.0 | 16.6 | 11.1 | 49.37 | 48.53 |\n| 0.9,0.1 | 80.1 | 83.7 | 52.1 | 51.0 | 17.5 | 12.9 | 49.90 | 49.20 |\n| 0.8,0.2 | 80.6 | 84.1 | 52.4 | 51.0 | 18.2 | 14.5 | 50.40 | 49.87 |\n| 0.7,0.3 | 81.1 | 84.2 | 52.7 | 50.8 | **18.5** | **15.9** | 50.77 | 50.30 |\n| 0.6,0.4 | 81.6 | 84.9 | 52.9 | 51.2 | 18.0 | 14.8 | **50.83** | **50.30** |\n| 0.5,05 | 82.1 | 85.2 | 53.1 | 51.7 | 16.7 | 13.8 | 50.63 | 50.23 |\n| 0.4,0.6 | 82.6 | 85.0 | 53.3 | 51.4 | 14.7 | 11.1 | 50.20 | 49.17 |\n| 0.3,0.7 | 82.9 | 85.9 | 53.3 | 51.2 | 12.5 | 9.2 | 49.57 | 48.77 |\n| 0.2,0.8 | 83.2 | 86.0 | **53.3** | **52.2** | 10.9 | 7.4 | 49.13 | 48.53 |\n| 0.1,0.9 | **83.4** | **86.6** | 53.2 | 51.7 | 9.9 | 6.7 | 48.83 | 48.33 |\n| 0.0,1.0 | 83.2 | 86.4 | 53.0 | 51.4 | 9.2 | 5.3 | 48.47 | 47.70 |\n\n> **Q3:** Figure 3 lacks the visualization result of the baseline method. It will make audience hard to get the improvement of MDL. The same issue applies to Table 5.\n> \n\n**A3:** Thank you for the suggestion. The baseline method does not employ MDL, so it does not separate specific and shared features and only produces fused features. Therefore, we omit this step from the visualization. Due to rebuttal interface constraints, we cannot upload images here, but we are willing to include this visualization in the revised manuscript.\n\n> **Q4:** It would be helpful to clarify how the 4 testing scenarios under modality-mismatched conditions were selected.\n> \n\n**A4:** NIR is the second most widely used modality in surveillance systems after RGB, and RGB-to-NIR research continues to attract significant interest. Therefore, we evaluate our method in four R-to-N-based application scenarios."}, {"review_id": "VvFoFzJhYs", "reviewer": "Reviewer_oBaa", "summary": "This paper introduces MDReID, a general-purpose object Re-ID framework designed for modality-mismatched and modality-matched scenarios. It addresses the practical limitation of existing methods that assume consistent modalities. MDReID consists of two key modules: MDL decomposes features into modality-shared and modality-specific components. MML enhances feature discrimination via orthogonality and complementarity. The method is validated on three multi-modality Re-ID benchmarks and shows significant improvements over baselines in both modality-matched and mismatched settings.", "questions": "See Weakness.", "limitations": "Yes", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Strengths:\nThis paper clearly identifies the modality inconsistency issue in real-world ReID systems and proposes a targeted solution.\nThis paper Introduces a dual mechanism of modality decoupling (MDL) and metric learning (MML), with a well-structured design.\nThe performance improvements are validated across multiple benchmark datasets, supported by strong empirical evidence.\nWeaknesses:\n1. Lack of a Clear Classification Framework in Related Work: The related work lacks a clear technical taxonomy. The current version lists existing methods but does not organize them by core challenges (e.g., Conventional multi-modal methods,ViT-based, missing-modality methods). It also lacks analysis of each method's key ideas and limitations.\n2. Redundant or Delayed Definition of Technical Terms (e.g., ViT): Several technical terms (e.g., ViT) are either repeated or introduced only after they have already been used multiple times.\n3. Binary Availability Mask Explanation is Overly Verbose: The explanation of the binary availability mask spans over 0.5 pages. The logic of zero-padding can be simplified or moved to the supplementary material. The current version distracts from the main technical innovation.\n4. Knowledge Discrepancy Loss Definition (Line 224) is Unclear: The structure of the knowledge discrepancy loss is hard to interpret.\nAnyway, this is a good paper with a new task and reasonable solution, although there are some minor flaws in the details.", "was_revised": true, "final_justification": "The response addresses my concerns.", "rebuttal": "> **Q1:** Lack of a Clear Classification Framework in Related Work: The related work lacks a clear technical taxonomy. The current version lists existing methods but does not organize them by core challenges (e.g., Conventional multi-modal methods,ViT-based, missing-modality methods). It also lacks analysis of each method's key ideas and limitations.\n> \n\n**A1:** Thank you for the valuable feedback. We agree that the introduction and related work section would benefit from a clearer technical taxonomy and deeper analysis. We have revised these sections following your suggestions to categorize existing methods by core challenges and include a discussion of their key ideas and limitations.\n\n> **Q2:** Redundant or Delayed Definition of Technical Terms (e.g., ViT): Several technical terms (e.g., ViT) are either repeated or introduced only after they have already been used multiple times.\n> \n\n**A2:** Thank you for the comments. We have revised the manuscript to ensure that all technical terms are clearly defined and used. \n\n> **Q3:** Binary Availability Mask Explanation is Overly Verbose: The explanation of the binary availability mask spans over 0.5 pages. The logic of zero-padding can be simplified or moved to the supplementary material. The current version distracts from the main technical innovation..\n> \n\n**A3:**  Thank you for the suggestion. We have accordingly reduced the content of the related section following your suggestion.\n\n> **Q4:** Knowledge Discrepancy Loss Definition (Line 224) is Unclear: The structure of the knowledge discrepancy loss is hard to interpret. Anyway, this is a good paper with a new task and reasonable solution, although there are some minor flaws in the details.\n> \n\n**A4:**  Thank you for pointing out this issue. We design the KDL objective to enforce a knowledge gap between specific and shared features. Without KDL, the model may degrade into relying only on shared features. We therefore ensure that combining specific and shared features yields higher retrieval performance than using either feature alone. To achieve this, we apply the formulation in lines 222–225, which imposes distance constraints in the feature space based on the mentioned relation. Table 4 validates our approach: as shown in Table A, adding KDL improves mAP by 2.0% and Rank-1 by 1.5% across multiple scenarios. We have also revised this section accordingly to make it clearer.\n\nTable A: Ablation study.\n\n| Methods | mAP | R-1 |\n| --- | --- | --- |\n| Without KDL | 41.2 | 40.8 |\n| KDL | 43.2 | 42.3 |"}, {"review_id": "znpxygnmi2", "reviewer": "Reviewer_dspH", "summary": "This paper proposes MDReID, a framework for object re-identification across that allows any query and gallery combinations from RGB, NIR, and TIR images. Unlike prior work that focuses on fixed modality pairs, MDReID handles arbitrary modality combinations through two key components: Modality-Decoupled Learning (MDL), which separates features into modality-shared and modality-specific parts, and Modality-aware Metric Learning (MML), which ensures these parts are complementary via orthogonality and discrepancy constraints. This design enables robust matching under varying or missing modalities. Extensive experiments on RGBNT201, RGBNT100, and MSVR310 show that MDReID achieves state-of-the-art performance with significantly fewer parameters and lower computation cost.", "questions": "1. This paper appears to be the only one among all compared methods that adopts CLIP as the backbone. It would be helpful to clarify whether this choice contributes additional performance gains.\n\n2. The discussion on KDL may be somewhat insufficient. While the authors have dedicated adequate space to describing how KDL is implemented, they have not clearly articulated how it contributes to the model’s performance or why it is designed in its current form. I encourage the authors to elaborate on the underlying design rationale of KDL in their rebuttal. \n\n3. I believe that using a classifier to explicitly distinguish between different modalities is a simple yet effective strategy. However, it appears that the authors did not adopt this approach in the current work. I would suggest including a comparison to assess whether incorporating such a strategy could lead to further performance improvements.\n\n4. In Table 2, the performance of TOP-ReID appears to be mistakenly highlighted in bold.", "limitations": "Yes, this work has addressed their limitations in ‘Supplemental Material’.", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "The strengths of this paper are quite clear. The proposed any-to-any object re-identification task is both novel and compelling. Existing works typically split modality-aligned and cross-modal retrieval into two separate problems. I do not believe this is a desirable direction, as it fragments the solution space and limits generalization. In contrast, this paper is the first to support arbitrary query-gallery modality combinations within a unified framework, which I consider a significant contribution.\n\nOn the downside, the overall methodology appears relatively simple. While this simplicity may be an advantage from an application standpoint, the idea of feature decoupling itself is not particularly novel within the ReID literature. \n\nNevertheless, I believe the paper’s contribution outweighs this concern. Its exploration of the any-to-any setting is timely and has the potential to make a notable impact on the re-identification community.", "was_revised": true, "final_justification": "This paper proposes MDReID, a novel framework for object re-identification. It handles arbitrary modality combinations through two key components: Modality-Decoupled Learning (MDL) and Modality-aware Metric Learning (MML). This design enables robust matching under varying or missing modalities.", "rebuttal": "> **Q1:** This paper appears to be the only one among all compared methods that adopts CLIP as the backbone. It would be helpful to clarify whether this choice contributes additional performance gains.\n> \n\n**A1:**  Thank you for your suggestion. To validate that CLIP contributes additional performance gains, we conduct experiments on our server to compare Top-ReID with our method, both using a CLIP-based ViT backbone. As shown in Table A, our method outperforms Top-ReID by approximately 11.1 % in mAP and 11.3 % in R-1. These results confirm the superior performance of our approach under identical settings and with the same backbone.\n\nTable A: Performance comparison on CLIP-based ViT.\n\n| Methods | mAP | R-1 |\n| --- | --- | --- |\n| TOP-ReID | 71.0 | 73.9 |\n| Ours | **82.1** | **85.2** |\n\n> **Q2:** The discussion on KDL may be somewhat insufficient. While the authors have dedicated adequate space to describing how KDL is implemented, they have not clearly articulated how it contributes to the model’s performance or why it is designed in its current form. I encourage the authors to elaborate on the underlying design rationale of KDL in their rebuttal.\n> \n\n**A2:**  Thank you for pointing out this issue. We design the KDL objective to enforce a knowledge gap between specific and shared features. Without KDL, the model may collapse into relying only on shared features. We therefore ensure that combining specific and shared features yields higher retrieval performance than using either feature alone. To achieve this, we apply the formulation in lines 222–225, which imposes distance constraints in the feature space. Table 4 validates our approach: as shown in Table A, adding KDL improves mAP by 2.0% and Rank-1 by 1.5% across multiple scenarios. In subsequent versions, we provide more detailed explanations. We have also revised this section accordingly to make it clearer.\n\nTable B: Ablation study.\n\n| Methods | mAP | R-1 |\n| --- | --- | --- |\n| Without KDL | 41.2 | 40.8 |\n| KDL | **43.2** | **42.3** |\n\n> **Q3:** I believe that using a classifier to explicitly distinguish between different modalities is a simple yet effective strategy. However, it appears that the authors did not adopt this approach in the current work. I would suggest including a comparison to assess whether incorporating such a strategy could lead to further performance improvements.\n> \n\n**A3:**  Thank you for your suggestion. Actually, in our ablation study, we perform relevant experiments, but our inadequate description of the baseline may lead to misunderstanding. Specifically, in Table 4(a), configuration “1” corresponds to using a single classifier for all modalities, while configuration “3” corresponds to assigning a separate classifier to each modality. The ablation results show that modality-specific classifiers effectively improve performance, boosting mean mAP by 13.4% and mean R-1 by 13.7%.\n\nTable C: Comparison of different classifiers.\n\n| Methods | mAP | R-1 |\n| --- | --- | --- |\n| One classifier  | 27.8 | 27.1 |\n| different modalities classifier | **41.2** | **40.8** |\n\n> **Q4:** In Table 2, the performance of TOP-ReID appears to be mistakenly highlighted in bold.\n> \n\n**A4:**  Thank you for your suggestion. We will make the corresponding changes in future versions."}]

Author rebuttal:

Meta-review: This paper proposes a novel framework, MDReID, for object re-identification across that allows any query and gallery combinations from RGB, NIR, and TIR images, in which the any-to-any object re-identification task is both novel and compelling. The authors make a significant contribution by being the first to accommodate arbitrary query-gallery modality combinations within a unified framework. All reviewers appreciated this paper’s innovation and practicality for real-world applications. The main concerns raised by reviewers were insufficient discussion about KDL, a lack of comparisons of the actual search time against baselines and an inadequately fair assessment. The authors’ rebuttal addressed most of these concerns, resulting in unanimous acceptance by the conclusion of the discussion. Therefore, AC recommends acceptance.
Decision: Accept (spotlight)