RLE: A Unified Perspective of Data Augmentation for Cross-Spectral Re-Identification
Authors: Lei Tan
Yukang Zhang
Keke Han
Pingyang Dai
Yan Zhang
YONGJIAN WU
Rongrong Ji
Venue: NeurIPS 2024 poster (2024)
Keywords: Cross-Spectral Retrieval
Data Augmentation
Re-Identification
Primary area: machine_vision

Abstract: This paper makes a step towards modeling the modality discrepancy in the cross-spectral re-identification task. Based on the Lambertain model, we observe that the non-linear modality discrepancy mainly comes from diverse linear transformations acting on the surface of different materials. From this view, we unify all data augmentation strategies for cross-spectral re-identification as mimicking such local linear transformations and categorize them into moderate transformation and radical transformation. By extending the observation, we propose a Random Linear Enhancement (RLE) strategy which includes Moderate Random Linear Enhancement (MRLE)  and Radical Random Linear Enhancement (RRLE)  to push the boundaries of both types of transformation. Moderate Random Linear Enhancement is designed to provide diverse image transformations that satisfy the original linear correlations under constrained conditions, whereas Radical Random Linear Enhancement seeks to generate local linear transformations directly without relying on external information. The experimental results not only demonstrate the superiority and effectiveness of RLE but also confirm its great potential as a general-purpose data augmentation for cross-spectral re-identification.

RLE: A Unified Perspective of Data Augmentation for Cross-Spectral Re-identification
Lei, Tan', Yukang Zhang', Keke Han', Pingyang Dai'; Yan Zhang's Yongjian Wu', Rongrong Ji
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China.
Tencent Youtu Lab, China.
tanlei@stu. xmu edu. cn, zhangyk@stu, xmu. edu. cn, hankeke3030stu. xmu xmu. edu. cn, pydai @xmu edu. cn, bzhy986@xmu edu. en, Littlelkenm@ttencent com, rr rrji@xmu, edu.
Abstract
This paper makes step towards modeling the modality discrepancy in the cross- spectral re-identification task. Based on the Lambertain model, we observe that the non-linear modality discrepancy mainly comes from diverse linear transformations acting on the surface different materials. From this view, we unify all data augmentation strategies for cross-spectral re-identification by mimicking such local linear transformations and categorizing them into moderate transformation and radical transformation. By extending the observation, we propose Random Linear Enhancement (RLE) strategy which includes Moderate Random Linear Enhancement (MRLE) and Radical Random Linear Enhancement (RRLE) push the boundaries of both types of transformation. Moderate Random Linear Enhancement is designed provide diverse image transformations that satisfy the original linear correlations under constrained conditions, whereas Radical Random Linear Enhancement seeks to generate local linear transformations directly without relying on external information. The experimental results not only demonstrate the superiority and effectiveness of RLE but also confirm its great potential as a general-purpose data augmentation for cross-spectral re-identification. The code is available https: //gi thub. com/ /tt.//12//n9613/
Introduction
Identity recognition has attracted intensive attention the last few years due to its wide applications in surveillance systems [1, 2, 3, 4]. Since silicon-based digital cameras naturally sensitive near- infrared (NIR), most cameras provide infrared (IR) images instead of visible (VIS) images for better visual quality under poor illumination conditions. In practice, this puts the re-identification (Re-ID) problem in a cross-spectral setting and requires the approaches to properly handle both the intra-class variance and the more significant modality discrepancies between cross-spectral images [5, 6, 7]. Encouraged by the great success of mingle-modaliy re-identification, substantial research efforts in cross-spectral re-identification attempt transform the cross-spectral re-identification challenge into a single-modalin learning task. To achieve this goal, previous efforts utilize DNN-based image processing such as Generative Adversarial Networks (GANs) [8], to construct the translation from one spectrum to another. These methods [9, 10] generally provide good visual effects and adjustability. However, the limited visual quality of the generated images and the lack of large-scal databases providing cross-spectral image pairs make GAN training challenging, thus limiting the performance of these methods. Another mainstream strategy focuses on the channel difference between infrared
"Corresponding Author.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

images [11, 12]. Methods such as grayscale transformation and random channel selection attempt to use image transformation strategies to mimic the transformation between cross-spectras spectral images. thereby pushing the network to adapt to such a transformation. While these methods make sense and decrease the modality discrepancy, lacking the modeling of cross-spectral transformation, they usually tend to pursue the similarity in human visual perception rather than real cross-spectral transformation.
In this paper, we attempt to explore the possibility of modeling the multi- spectral transformation to provide more interpretability, and thus further push the boundary of cross- cross-spectral Re-ID approaches. Based on the Lam- bertian reflection model [14, 15, 16], we find that the illuminations of the same region in VIS and NIR photos should be able to be described us- ing a simple linear model, as long as the region is composed of one con- sistent material (details are discussed in Sec. 3). This is illustrated in Fig- ure 1. Here, we use paired VIS-NIR images from the dataset in [13]. For the red and yellow regions in the mid- dle of the image, with simple linear model, we can accurately predict the pixel values of the NIR image based on the VIS image, as long as the re- gion only has one material, Although the linear transformation exists at the pixel level of cross-spectral image pairs, the material's 's reflection function determines the linear factor. It means that the linear factor is inconsistent across different surfaces, resulting an image-level non-linear transformation. In Section 3, we analyze and visualize the result to confirm whether the different linear factors on different surfaces are the main culprit that induces the modality discrepancy in the cross-spectral images. It is interesting to find that the modality discrepancy occurs when using variable linear factors among different patches the image.
Chan
0.08
0.2
NRR
Clanni
0.00
0.02
0.75
0.35
045
MR
Irage
G Channel Image
G-NIR Scatter Piot
NER Image
Figure I: Illustration of the cross-spectral transformation. G refers to the green channel of the visible image. Un- der the same illumination, the cross-spectral transformation could be described as linear transformation in material- similar surface. Still, in the whole image level, the transfor- mation is nonlinear due to the diversity of materials. Since the Re-ID image pairs are not well aligned, we select the cross- srrsstrpeettral image pairs from [13].
The above observation provides us with a fresh perspective on the cross-spectral Re-ID task. Em- pirically, adopting observation in image generation seems to be the most intuitive way. As long as we are able to identify regions' materials with their visible or infrared input and calculate the linear coefficients transform the input image from one spectrum to another. the modality discrepancy would be easy to bridge. Unfortunately, the correlation between visible or infrared input and regional materials is quite limited, which also confines generative strategies in this task to a clear upper bound. Besides exposing the bottleneck of the generative strategy, the observation also provides a unified perspective to rethink the augmentation strategies within this topic. From this perspective, we dis- cover that data augmentation for cross-spectral re-identification is formed to achieve non-linear transformations with different distinet local linear factors, thus encouraging the network to be robust to such transformation. Therefore, under this view, we can easily categorize all the data augmentation strategies designed for cross-spectral re-identification into moderate transformation and radical transformation based on the extent of changes to images. We assign moderate transformation as a strategy that can still keep the original linear correlation after the transformation. Generally, achieving moderate transformation may require precise material labels each pixel. However, with benefits from the diversity of different channels in visible images, we can obtain a moderate transformation by linear calculation based on the original image channels. Methods like channel exchange and grayscale transformation are both special cases under moderate transformation. Within the unified formulation of moderate transformation, in this paper, we further provide a more general moderate transformation as Moderate Random Linear Enhancement (MRLE), which aims to use an unfixed mixing of different channels to provide more diverse augmentation results. In contrast to moderate transformations, radical transformations attempt to apply linear transformations to randomly selected local areas. Compared moderate transformations, which have limited transformation space and are only effective on multi- chatti- visible images, radical transformations can produce a more diverse range of results even on single-channel infrared images. However, due the lack of
2

---

constraints, these transformations often introduce additional noise into the original image. Methods such as random erasing [17] and channel random erasing [11] can be considered special cases of radical transformation where the linear factor is set to O. Similarly, based on the above perspective, we also provide a Radical Random Linear Enhancement strategy, that yields competitive augmentation results by directly applying linear transformations to randomly selected local areas.
In summary, our contributions are threefold:
As an effort to model the transformation behind the modality discrepancy the cross- spectral Re-ID) task, we discover that the cross-spectral modality discrepancy mainly comes from different local linear transformations caused by the diversity of materials. Based on this observation, we further categorize the cross-spectral data augmentation strategies into moderate and radical transformations under a unified perspective.
By extending the observation, propose Random Linear Enhancement (RLE) strategy, which includes Moderate Random Linear Enhancement (MRLE) and Radical Random Linear Enhancement (RRLE). The RLE effectively takes advantage of the aforementioned unified perspective and embeds it in a controllable linear transformation.
Extensive experiments on cross-spectral re-identification datasets demonstrate the effective- ness and superior ability of the proposed RLE, which can boost performance under various scenarios.
2 Related Works
Cross-spectral re-identification is a challenging task due to the significant modality discrepancy. Two typical frameworks have been proposed to solve such challenging task. The first one is feature-level learning [18, 19, 20, 21, 22], which aims to bridge the modality gap through well-designed loss funetions and end-to-end training. Such strategy works well in both supervised, supi-vuper,,sed, and unsupervised [23] cross-spectral re- re-identification tasks due to the great power deep learning. However, these approaches usually do not use any real physics models, making it not uncommon for them to make strange mistakes. To make things worse, due to the high complexity and lack of interpretability, the models hard to adjust improve. The other mainstream method to solve cross-spectral re-identification is the image-level strategy, which aims to construct an efficient transformation between different spectrums. Under this condition, the cross-modality discrepancy is considered an individual problem alongside the Re-ID problem. D²RL [5] makes the first attempt by using variational autoencoders (VAE) for style disentanglement and generates synthesis images from one spectrum to another. AlignGan improves this framework by proposing unified GAN framework with efficient constraints. Although playing a min-max game between the complex generator and discriminator offers visually impressive results, the generated images are still far from photorealistic, and in turn, limit the final performance. Therefore, these methods were subsequently superseded by eighter-wight modality generate strategies. This improvement suggests that cross- spectral transformations may not be as complicated as previously envisaged. X- X-modality [24] designs lightweight network to learn an intermediate mediator from visible images, while MMN [25] improves by extending an infrared side. Recently, CAJ [11] and CAJ+ [12] directly removed the extra generator and utilized several types of grayscale images as assistant for training which also achieves satisfying performance. Although recent methods have made some progress this topic, due to the lack of analysis and modeling for cross-spectral transformation, the methods tend to pursue the similarity of transformation in human visual perception rather than real cross-spectra transformation.
3 Reflection Prior for Cross-Spectral Images
VIS-NIR matching is a longstanding computer vision problem that has been explored for decades [26, 27]. One of the main challenges is to formulate and thus alleviate the modality discrepancy. Using the Lambertian model analyze the digital image from multi-sensor cameras is widely applied in some pioneer works [15, 28). With light source emitting photons across different wavelengths X, the response of each pixel (x. y) in the camera sensors can be formulated as:
(5 v) = ol. { E, E,,,, v) x. V)
(I)
3

---

NIR
R-NIR
G-NIR
B-NIR
Figure 2: Example images from the VIS-NIR dataset [13]. After divide the visible image into the red, green, and blue channels and form chromaticity band ratios from these three spectra and the NIR image, is clear that the ratio for pixels from the surface with high material-similart is nearly constant.
W1.5
XO5
x1.
X05
(5)
(b)
(c
Figure 3: A example about how modality discrepancy occurs. Feature space visualization of 100 randomly selected images with (dot) and without (fork) the local linear transformation the original image. (a)~(b): The same linear factor takes effect on the whole image bringing limited modality discrepancy. (c): Variable linear factors take effect on different parts showing huge modality discrepancy. The "cross' and 'dot' marks indicate the samples from the original one and the generated one respectively.
where À is the wavelength, well F(X) and (()) denote the spectral power distribution (SPD) of incident light and surface spectral reflectance. Q(A) is the spectral sensitivity of the camera sensor. (R, G, B, N) indicates the channel (spectrum). (2, y) is the Lambertian reflection term which is constant factor and can be calculated by the dot product of the surface normal with the illumination direction.
Following Eq. (1), we leverage a mild assumption to derive a representation between the SPD of the light source and incident light. Generally, could describe the SPD of the light source by relative spectral power distribution F(A, x, 3) together with variable w that reflects the illumination intensity. We assume that the SPD of incident light in the whole image keeps the same relative spectral power distribution as the light source. Then we could formulate the E A, x,y) y) as:
E,,A, (, F2(),
where 3 is parameter to reflect the ratio of intensity between incident light and the light source. Then from Eq. (1) and Eq. (2), if we consider the images under different spectra, such G channel images and NIR images, is clear that the transformation of G-NIR could be described as:
PN( v) wNBN(a, v) JAy Fn(A)S(A, x, 2))x PG(I,)) woBola, y) Sha Fo(A)S(A, 2,)
Under an ideal condition, 3 is supposed to be a high-order term determined by the distance between light source and the surface [29]. Now, we could utilize this approximation and regard as constant under the same light source get a simplified expression:
(N(x, wN wx. y, N) PG(x,y)
(4)
Wo (2, G)

---

Randonn: Patch Randors- inser Parmneeer
Random-- Lis--Liie inner
Parameter
Fenture Extractor
Invariable
Inscrriable
Feature Extractor
Invariable
Extractor
Q
B
(e) Definite ineer Inaeforaaion on Definite image potch.
(b) Dandom lineer transfortation uz Definits image patch.
(cl Random Lineer trans/ettaionn on Randem image paich.
The motivation RLE. Herein, we construct an ideal person with only two different surfaces and ignore the background. (a): As demonstrated above, to obtain representation, the network should be robust such a transformation that takes effect upon definite surfaces by definite linear factors. (b): An ideal data augmentation strategy that takes effect upon definite surfaces by random linear factors. However, this method needs a mard-achieved extra material- aware network for segmentation. (c): The idea of RRLE. By taking effect upon random surfaces by random linear factors, the RRLE encourages the network to be robust to linear transformation anywhere in the image. Under this condition, the cross-spectral transformation can be considered as an easy state of RRLE space.
Figure 4:
ennctraliinvaian feature
Since F(A), Q()), and S(A) three inner functions depending the SPD of the light source, the sensitivity of the camera sensor, and the reflection function of the surface material, we replace the Riemann integral with function (x, y.j). In addition, wx could be considered factor in two determined spectra. From this representation, transformation a linear transformation in those regions of the same material and under the same illumination condition, as shown in Figure 1. If we further extend it to the entire image, the factor is only influenced by S(A, x, y) which is determined by the material.
constant could observe that the cross-spectral
we
To verify whether the above equation could be used in various real-world scenarios, used the paired VIS-NIR scene image dataset introduced by [13]. Figure 2, we form chromaticity band ratios between three VIS spectra and NIR spectrum at each pixel and use the color to reflect the ratio. We discovered that the ratio is nearly constant within a region with a consistent material, which holds across R, G, B. and NIR spectra.
After observing the above linear transformation, we further explore whether the variable linear factor in different surfaces is the main culprit that induced the modality gap in such an application. Due to the lack of material labels that are available to guide sample generation, we uniformly segmented 100 randomly selected images into six parts from the top to the bottom and multiplied each part by linear factor. Then, we send the new images and original images into an ImageNet [30] pre-trained Resnet-50 [26]. Although not well-aligned, benefiting from the body structure prior from head to toe, we still find that the modality discrepancy occurs when suffering from variable linear factors.
4 Random Linear Enhancement
From the above observation, we can unify all data augmentation strategies for cross-spectral re- identification as mimicking such local linear transformation, thereby encouraging the network to be robust to transformation, as shown in Figure 4 (a). Based on this perspective, by considering the influence on the original image, the data augmentation strategies can easily be categorized into two types: moderate transformation and radical transformation. By extending the observation, this paper pushes the boundary of both types by proposing a Random Linear Enhancement strategy (RLE).
4.1 Moderate Random Linear Enhancement
As mentioned before, we assign moderate transformation as strategy that maintains the original linear correlation after the transformation. Benefiting from the difference between different channels (R. G. B) in visible images, can obtain moderate transformation without precise material labels.

---

From this view, the processing of moderate transformation can be unified as:
1mt = A,1, + 2o + ApTo. 8.1. d - >g Ao 1,
(5)
where the Lit indicates the transformed image, as well as Lr, Lg. and La refer to the red, green, and blue channels of the visible image, respectively. Here, År. Ag. and X are hyper-parameters the mixing percentage. It is evident from Eq. (5) that random channel selection corresponds to the specific cases where the parameters Xy. Ag. and d are specified as [1, 0), [0, 1, 0j. or [0, O, 1]. Also, the grayscale transformation is the specific cases where the parameters Xy, Ag. and A are specified as (0.299, 0.587, 0.114].
control
is apparent that previous strategies exhibited significant limitations their parameter settings, leading to highly restricted augmentation results. Therefore, attempt relax the settings of X,. Ag. and A and propose a Moderate Random Linear Enhancement (MRLE). Generally, sampling from uniform distribution to determine the values of three hyper-parannn simplest and most efficient approach. However, while uniform distribution uniformly covers the entire feasible transformation domain, in practice, those samples the boundaries always contribute more to learning decision boundaries. Consequently, we employ a U-shaped beta distribution instead of uniform distribution for hyper-panne parameter sampling. This not only maintains the feasible transformation domain but also enhances the sampling probability of boundary samples. In general, the formulation of MRLE can be given as follows:
is identified as the
Iite Xp + dgga + Aodo.
with Arx Nn: AB ~ Beta Bm))
(6)
s.l. X + Xg + As 1.
Herein, Sm is the hyper-parameter to control the shape of the beta distribution.
4.2 Radical Random Linear Enhancement
Although MRLE can provide diverse transformation results that obey the original linear correlation in the image, it can only take effect on the multi-channel visible image and shows quite limited transformation space. The ideal data augmentation appears to be using random linear factors different surfaces as shown in Figure 4 (b). However, this approach heavily relies on pixel-level material labels, which hard obtain. Therefore, achieving such local linear transformation without adequate material labels may inevitably involve some risk-taking. To achieve this goal, as shown in Figure 4 (c), we propose the Radical Random Linear Enhancement (RRLE) that randomly selects several image patches and multiplies them with variable linear parameter to directly mimic the local linear transformation. Under the RRLE, the cross-spectral transformation could be considered as sub-state of the whole state space.
Coneretely, for input image I, the RRLE randomly selects rectangle region Lsetect following the same setting of random erasing [17] and multiplies with a linear factor a. In case it may exceed the upper bound, calculate the maximum feasible linear factor in Tyte Omaaz The linear factor calculated by multiplying Oraaz and a random factor fg between O to 1. Typically, small linear factors may not be sufficient to effectively provide enough variation in the original image". Therefore, U-shaped Beta distribution is utilized for fy obtain and provide high-quality training samples. In general, the formulation of RRLE can be given as follows:
Lrt algelect:
with a amaafg.
and fg ~ Beta( 8r).
Herein, I,, indicates the transformed selected region. 8, refers to the hyper- papee--rrrmeter to control the shape of the beta distribution.
Furthermore, following the setting of random erasing, we set Smin and Smar to control the area of the selected region, while setting Timn and Fmar adjust the aspect ratio. Unlike most data augmentation strategies that take effect on the image only once, the RRLE will be repeated several
VVisualization results provided in the appendix.

---

times to obtain a higher modality discrepancy. Since the repeat will bring extra noise, we set memory matrix M store the cumulative changes at each pixel. We set Emin to terminate the RRLE when min(M) < Imin- To better explain the processing, we provide detailed procedure of RRLE in the appendix.
Experiments
5.1 Datasets and Implementation details
We conduct experiments two publicly available visible-infrared person re-identification datasets SYSU-MM01 [31] and RegDB [32].
SYSU-MM01 is large-scale dataset captured by four visible cameras and two infrared cameras in both indoor and outdoor environments. The training contains 395 identities with 22, 258 visible images and 11, 909 infrared images, while the testing set includes 96 identities with 3, 803 infrared images as the query. This dataset contains two different search modes, the all-search mode and the indoor-seareh mode. In the all-search mode, the gallery images are from all the visible cameras. For the andoor-search mode, the source of the gallery set excludes two outdoor cameras.
RegDB dataset collected by two aligned cameras, one for visible and the other for far-infrared (thermal). It contains 412 identities, each with 10 visible images and 10 infrared images. Following the evaluation protocol of previous works [33, 34], we choose half of the identities at random for training and the other half for testing. The results are the average of 10 repeating.
We follow the evaluation settings in existing VI-ReID methods [11, 1, 12] and adopt the Cumulative Matching Characteristic (CMC), mean Average Precision (mAP) and mean Inverse Negative Penalty (mINP) as evaluation metrics.
5.2 Implementation details
We Pytorch implement our method and finish all the experiments on single RTX 3090 GPU, The mini-batch size is set to 48. For each mini-batch, we randomly select 4 identities, each with visible images and 6 infrared images. We select the ResNet-50-basse branch as the baseline, which is a widely used fine-grid part feature learning framework in both Re-ID and isible-infrarre Re-ID [25, 18]. We also divide the first convolutional layer to tackle the two modalities' input as usual [36, 37J. We resize all of the images to 384 x 192 and use random flipping as basic data augmentation. The initial learning rate is set 0.1, and decayed by 0.1 and 0.01 at 20, and 50 epochs. Following previous works[38, 11, 39], we apply warm-up strategy in the first 10 epochs. To better verify the ability of the proposed data augmentation strategy, we just use the basic softmax cross-entropy loss and triplet loss during the training without adding any extra constraints to solve the modality discrepancy.
PCB [35] with the global
5.3 Ablation Study
In this section, we conduct empirical experiments to show the performance under different data aug- mentation strategies. Since we have categorized all data augmentation strategies for cross-spectrum re-identification into moderate transformations and radical transformations, have conducted relevant discussions on these two aspects and mixed transformations.
Moderate transformation. Here, we evaluate the influence of the performance with different moderate transformation strategies and show the quantitative results Table 1. In particular, we compare the proposed MRLE with the widely used grayscale transformation (" ('Gray") and random channel selection ('RC'). In previous works [1 I, 12], the 'Gray" and 'RC" are usually used together to obtain more diverse results. Therefore, also give the result under both 'Gray" and 'RC'.
Compared the baseline, every moderate transformation yielded positive gains showing the effective- ness of moderate transformation. However, although methods such as 'RC" and 'Gray' do simulate cross-spectrum transformations to degree, their restricted transformation spaces result in smaller performance enhancements compared to MRLE. By fully exploring the feasible transformation space, MRLE managed to surpass the limits of earlier moderate transformation strategies, achieving a significant increase in performance in all metrics.
7

---

Table 1: Ablation study of different data augmentation strategies on the cross-spectin re- ce-identification task. "Gray' denotes the grayscale transformation, 'RC' refers to the random channel selection.
"MRLE' indicates the moderate random linear enhancement. 'RE' refers the random erasing, and
'RRLE" means the radical random linear enhancement.
All Search
Indoor Search
Setting
R-1
R-5
R-
10
R-20
MAP
mlNP
R-1
R-5
R-10
R-20
MAP
mLNP
Moderate Transformation
Baseline
64.5
88.1
94.2
98.1
62.9
50.4
70.0
91.7
96.6
99.1
75.1
71.1
Baseline+Gray
66.7
89.4
95.2
98.7
64.2
50.8
72.0
93.9
97.8
99.6
77.1
73.0
Baseline+RC RC
68.3
90.6
95.6
98.6
65.3
51.9
72.7
93.9
97.6
99.7
77.7
73.7
Baselinc+RC-Hee RC+Gray
68.6
90.7
96,0
98.8
64.9
52.3
74.3
94.1
98.1
99.6
78.8
75.1
Musclinc+MRLE
70.2
91.6
96.5
99.0
67.0
53.5
75.5
95.2
98.2
99.7
79.7
75.9
Radical Transformation
Baseline+RE RE
71.0
91.4
96.3
99.1
69.5
57.4
78.5
95.8
98.7
99.8
82.1
78.4
Baseline+RDe RRRE
72.0
92.4
97.2
99.4
69.1
56.3
77.0
96.4
99.1
99.9
81.4
77.7
Busclinc+RERRRLE
74.2
93.0
97.4
99.5
71.8
60.4
81.7
96.7
99.1
99.9
84.5
81.2
Mixed Transformation
Baseline+CAI [11]
73.5
92.9
97.4
99.4
69.4
55.4
80.7
96.1
98.6
99.8
83,5
79.8
Baseline+LLE+Be
75.4
93.5
97.7
99.6
72.4
60.9
84.7
97.9
99.3
99.9
87.0
83.7
Radical transformation. Besides the moderate transformation, we also provide a detailed empirical study of the radical transformation including random erasing ('RE") and RRLE in Table 1. We can observe that due to a more flexible transformation space, radical transformations reach an even better performance than the best moderate transformation MRLE.
Although 'RE' can be considered a special case of RRLE with a linear factor of O, RRLE encourages images undergo more transformations while preventing the loss of information. Therefore, RRLE and RE tend towards different valuation perspectives and can be used together. As shown Table 1, the peak performance under radical transformation is reached when combining both 'RE' and RRLE.
Mixed transformation. Given that moderate and radical transformations do not conflict formally, they can be combined during the training. Accordingly, we present the performances under mixed transformation in Table 1. The results indicate that moderate transformations and radical transformations can be used simultaneously and lead to significant performance improvements. Also, for better comparison, we add the recently proposed mixed transformation CAJ which combines the grayscale transformation and random crasing strategy. It is shown that the proposed RLE also works better than the CAJ in cross-spectral re-identification task.
5.4 Comparison with State-of-the-Arts
In Table 2, we combine the 'RLE+RE' with a basic framework and evaluate it against the previously reported state-of-the-an methods on the SYSU-MM01 and RegDB. Compared to previous works, is worth noticing that the basic network doesn't 't have any extra modules or constraints to cope with the modality discrepancy in cross-spectral re-id. Just combining the basic network with the 'RLE+RE' can achieve comparable performance with state-of of-the-art methods, which indicates the great adaptability of RLE in the cross-spectral re-id task.
5.5 Discussion
Seyper-parametr settings of RLE. RLE contains several hyper-parameters ensure its effective- ness, such as the 3m in Eq. (6), as well as the 3, and tmi in Eq. (7). Therefore, this part evaluates the performance under different hyyee--parameee settings and shows the results in Table 3.
Compared a uniform distribution, sampling from U-shape beta distribution performs better in MRLE, The optimal reach when 3m is set to 0.3. For RRLE, radical transformation, the boundary more sensitive. Over-transfoommttion may easily destroy the original image. In this framework, peak performance is achieved when 3, = 0.4 and Emin 0.1.
Applicability of RLE for other methods. Besides the basic framework employed above, also explored the integration of the proposed RLE strategy with the current state-of-the-art method to evaluate its extensive adaptability in cross-spectral re-identification tasks.
8

---

Table 2: Comparisons between the proposed method and some state-of coute-of-the-eee the-art methods on the SYSU- MM01 and RegDB datasets.
SYSU-MMOI
RegDB
Methods
All Search
Indoor Search
VIS to IR
IR to VIS
R-1
R-10
mAP
R-1
R-10
mAP
R-1
R-10
mAP
R-1
R-10
mAP
BDTR[40]
17.0
55.4
19.7
33.6
58.6
32.8
32.9
58.5
32.0
D`RL(S)
28.9
70.6
29.2
43.4
66.1
44.1
H--CMD[41]
34.9
77.6
35.9
70.9
86.4
66.0
Al
42.4
85.0
40.7
45.9
87.6
54.3
57.9
53.6
56.3
53.4
DDAG
54,8
90.4
53.0
61.0
94.1
68.0
69.3
86.2
63.5
68.1
85.2
61.8
LbA[42]
55.4
54.1
58.5
66.3
74.2
67.6
67.5
72.4
NFS[43]
91.3
96.5
69.8
80.5
91.6
72.1
78.0
CM-NAS[44]
60.8
92.1
58.9
68.0
94.8
52.4
82.8
95.1
79.3
81.7
94.1
77.6
[CLNe(37]
65.4
93.3
62.0
72.6
97.0
76.6
80.3
92.7
73.1
75.9
90.9
69.5
FMCNe1(45]
74.1
89.1
84.4
88.4
83.9
SMCL[46]
67 4
92.9
61.8
68.8
96.6
75.6
83.9
79.8
83.1
78.6
22RT]20]
68.7
96.4
66.3
72.5
97.8
78.2
83.6
75.7
82.0
73.8
CAJ[I]]
69.9
95.7
76.3
97.9
80.4
85.0
95.5
79.1
84.8
77.8
MPANet(47]
70.6
96.2
68.2
76.7
98.2
81.0
82.8
80.7
83.7
80.9
MMN [25]
70.6
96.2
66.9
76.2
97.2
79.6
91.6
97.7
84.1
87.5
96.0
80.5
MAUM [48]
87.9
87.0
84.3
CAJ+ [12]
71.5
96.2
68.2
78.4
98.4
82.0
85.7
95.5
79.7
84,9
95.9
78.6
DEEN [39]
74.7
97.6
71.8
80.3
99.0
83.3
91.1
97.8
85.1
89.5
96.8
83.4
Ours
75.4
97.7
99.3
87.0
92.8
97.9
88.6
91.0
86.6
Table 3: Hyper-parameter settings of RLE, The optimal performance reaches when Bms Br, and Livin is set to 0.3, 0.4, and 0.1 respectively.
(a) Performance under different 8m Compared to the uniform dis- tribution, shapee beta distribu- tion works better in MRLE.
(b) Performance under different G... Compared to the uniform dis- tribution, U-shaped beta distribu- tion works better in MRLE,
(c) Performance under different Emiis Using too small bmin will introduce excessive noise and lead performance degradation.
Bm
R-1
mAP
mINP
Br
R-1
mAP
mINP
Emin
R-1
mAP
mLNP
1.0
67.9
65.3
52.2
0.5
72.5
70.7
59.3
0.3
72.9
70.9
58.2
0.5
67.8
65.1
51.8
0.4
74.2
71.8
60.4
0.2
73.8
71.3
59.0
0.4
68.3
65.6
52.3
0.3
73.2
71.3
60.2
74.2
71.8
60.4
0.3
70.2
67.0
53.5
0.2
73.7
71.7
60.2
0.01
73.8
71.7
59.8
0.2
67.9
66.0
52.6
0.001
73.6
71.3
59.5
Herein, we add the RLE in the open-sourced method DEEN [39] and show the result in Ta- ble 4. Specifically, since the DEEN already con- tains the random grayscale and random erasing for data augmentation, we remove the random grayscale and add the RLE. Although the DEEN already contains strong augmentations, adding RLE can also bring a performance gain. Beyond the CNN models, we also investigated whether RLE could be applied to a ViT-based structure. Since there is no open-source ViT model for cross-spectral re-id, random erasing augmentation as the basic framework in this part. From Table 4, we can observe that RLE can still work well in ViT structure. To ensure the generalization ability of RLE, when applying to other methods, we keep the same hyperparameters setting RLE with the previous experiments. Therefore, better performance may be achieved on specific methods by fine-tuning the hyperparameters.
Table 4: Applicability of our opposed RLE to other methods on the SYSU-MMOJ dataset.
All Search
Indoor Search
Setting
R-1
mAP
R-1
mAP
DEEN [39]
74.7
71.8
80.3
83.3
+Ours
76.2 (41.5)
73.0 (41.2)
83.2 (42.9) 85,3 (42.0)
VIT-B [49]
66.0
63.1
69.9
75.1
+Ours
70.2(+42)
66.7(-36)
71.9
76.4
the vanilla VIT-B with
Visualization results of RLE, To gain a deeper understanding of RLE processing, we visualize the RLE augmented images from both the visible and infrared sides in Figure 5. It can be seen that MRLE provides an efficient way to provide diverse transformations from multi-spectral images to

---

Visible
MRLE
RRLE
RLE
Infrared
MRLEE
RRLE
RLE
Visualization results RLE. Since the MRLE can not take effect on the infrared images, we use "rv' instead. Meanwhile, Both MRLE and RRLE used with certain probability. Therefore, all of the augmentation images above are potential results.
Figure 5:
single-spectral images, while the RRLE gets rid of the dependence on the multiple spectral images and makes such linear transformation directly the local part. In general, adding such a random linear transformation in the local area of the images largely breaks the color information of the image while preserving the semantic.
Limitations and Broader Impact
Based on the specific observation in the cross-spectral re-identification, the proposed RLE may not be general as data augmentation strategy like random flipping. Whether breaking the modality- similarity between the image pairs could make sense in other computer vision tasks still needs to be evaluated. Meanwhile, under extremely bad weather, such as heavy rain, fog, or limited illumination, the Lambertain model may not work well. So, whether RLE can still perform well in these complex weather is ambiguous. On the other hand, the RegDB and SYSU-MM01 datasets are limited in scale and environment. Although the proposed RLE shows strong ability to boost the methods in both two datasets, the performance of RLE in an open- world scenario has not yet been verified. Nevertheless, we still believe that the proposed RLE can boost the research of image generation and data augmentation more general cross-spectral scenarios.
Conclusion
This paper provides a unified perspective on data augmentation strategies for cross-spectral re- identification. We observe the non-linear modality discrepancy mainly comes from the diverse linear transformation taking effect on different material surfaces; all data augmentation strategies for cross- spectral re- re-identification aim to simulate this kind of transformation. By extending the observation, we introduce a more general augmentation Random Linear Enhancement (RLF), further pushing the boundary of moderate transformation by Moderate Random Linear Enhancement (MRLE) and radical transformation by Radical Random Linear Enhancement (RRLE). Experimental results show that RLE is effective and applicable in cross-spectral re-identification tasks.
Acknowledgements. This work was supported by the National Key R&D Program of China (No.2022ZD011 1EN0D00111222))) 1200), the National Science Fund for Distinguished Young Scholars (No.62025603). the National Natural Science Foundation China (No.U21B2037, No. U22B2051, No. 62176222. No. 62176223,No. 62176226, No. 62072386, No. 62072387, No. 62072389, No. 62002305. and No. 62272401), and the Natural Science Foundation of Fujian Province China (No. 102000000 No. No.2002106001).
10

---

References
Mang Ye, Jianhing Shen, Gaojie Lin, Tao Xiang, Ling Shao, and Steven CH Hoi. Deep learning for person re-identification: A survey and outlook. IEEE transactions on pattern analysis and machine intelligence. 44(6):2872-2893, 2021.
[2] Lei Tan, Pingyang Dai, Rongrong Ji, and Yongjian Wu. Dynamic prototype mask for occluded person re-identification.. In Proceedings of the 30th ACM international conference multinedia, pages 531-5411 131-500, 2022.
[3] Yunpeng Gong, Liqing Huang, and Lifei Chen. Person re-identification method based on color attack and joint defence. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pages 131--4322, 2022.
[4] Lei Tan, Pingyang Dai, Jie Chen, Liujuan Cao, Yongjian Wu, and Rongrong Ji. Partformer: Awak- ening latent diverse representation from vision transformer for object re-identification. arXiv preprint arXiv:2408. 16684, 2024.
[5] Zhixiang Wang, Zheng Wang, Yinqiang Zheng, Yung-Yi Yu Chuang, and Shin Shin'ichi Satoh. Learning to reduce dual-level discrepancy for infrared-visihle person re-identification. In Proceedings of the CVPR, pages 618-626, 2019.
Mouxing Yang. Zhenyu Huang, and Xi Peng. Robust object re-identiification with coupled noisy labels. International Journal of Computer Vision, pages 1-19, 2024.
[7] Yunpeng Gong et Cross-modality perturbation synergy attack for person re-identification. arXiv preprint arXiv:2401. 10090, 2024.
[8] Ian Goodfellow. Jean Pouget-Abadic, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Proceedings of the NeurlPS, pages 2682-2680, 2014.
[9] Guanshuo Wang, Yufeng Yuan, Xiong Chen, Jiwei Li, and Xi Zhou. Learning discriminative features with multiple granularities for person re-identification. In Proceedings of the ACM MM, pages 274-282, 2018.
[10] Guan' an Wang, Tianzhu Zhang, Jian Cheng, Si Liu, Yang Yang. and Zengguang Hou. Rgb-infrared cross-modality person re-identification via joint pixel and feature alignment. In Proceedings of the ICCV, pages 3623-3623633 23322, 2019.
[11] Mang Ye, Weijian Ruan, Bo Du, and Mike Zheng Shou. Channel augmented joint learning visible- infrared recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 13567-13576, 2021.
[12] Mang Ye, 7esen Wa, Cuiqun Chen, and Bo Du. Channel augmentation for visible-infrared re-identification. IEFE Transactions on Pattern Analysis and Machine Intelligence, 2023.
[13] Matthew Brown and Sabine Süsstrunk. Multi-spectral sift for scene category recognition. In Proceedings of the CVPR, pages 177-184, IEEE, 2011.
[14] Berthold Horn, Berthold Klaus, and Paul Horn. Robot vision. MIT press, 1986.
[15] Graham D Finlayson, Steven D Hordley, Cheng Lu, and Mark S Drew. On the removal of shadows from images. IEEE Transactions on Putlem Analysis and Machine Intelligence, 28(1):59-68, 2005.
[16] Lei Tan, Yukang Zhang, Shengmei Shen, Yan Wang, Pingyang Dai, Xianming Lin, Yongjian Wu, and Rongrong Ji. Exploring invariant representation for visible-infrared person re-identification. arXiv preprint 0XX1v22302.08884, 2023.
[17] Zhun Zhong, Liang Zheng, Guoliang Kang, Shaozi Li, and Yi Yang. Random erasing data augmentation. In Proceedings of the AAAI, 2020.
[18] Yukang Zhang, Yan Yan, Jie Li, and Hanzi Wang. Mren: a novel modality restitution and compensation network for visible-infrared person re-identification. Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 3498-3506, 2023.
[19] Lei Tan, Pingyang Dai, Qixiang Ye. Mingliang Xu, Yongjian Wu, and Rongrong Ji. Spectral aware softmax for visible-infrare person re- identification. arXiv preprint MxXiv:2302.01512 2023.
11

---

[20] Mouxing Yang, Zhenyu Huang, Peng Hu, Taihao Li. Jiancheng Lv, and Xi Peng. Learning with twin noisy labels for visible-infrared person re-identification. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 14308-14317, 2022.
[21] Jiangming Shi. Yachao Zhang. Xiangbo Yin. Yuan Xie, Zhizhong Zhang, Jianping Fan, Zhongchao Shi, and Yanyun Qu Dual pseudo-labels interactive self-training for semi- seme-super visible-infrared person re-identification. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pages 11218-1 1228,222222 2023.
[22] Jiangming Shi, Xiangbo Yin, Yeyun Chen, Yachao Zhang, Zhizhong Zhang, Yuan Xie, and Yanyun Qu Multi-memory matching for unsupervised visible-infrared person re-identification. In Earopean Conference on Computer Vision, pages 456-474. Springer, 2024.
[23] Jiangming Shi, Xiangbo Yin, Yaoxing Wang, Xiaofeng Liu, Yuan Xie, and Yanyun Qu Progressive contrastive learning with multi- pulti-prolotype for unsupervised visible-infrared person re-identification arXiv preprint arXi::2402. 19026, 2024.
[24] Diangang Li, Xing Wei, Xiaopeng Hong, and Yihong Gong. Infrared-visihle cross-modal person re- identification with an x modality. In Proceedings of the AAAI, pages 4610-4617, 2020.
[25] Yukang Zhang. Yan Yan, Yang Lu, and Hanzi Wang. Towards unified middle modality learning for visible-infrared person re-identification. In Proceedings of the 29th ACM International Conference on Multimedia, pages 788-796. 2021.
[26] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the CVPR, pages 770-777 778, 2016.
[27] Chunlei Peng. Xinbo Gao, Nannan Wang, and Jie Li. Graphical representation for heterogeneous face recognition. IEEE Transactions on Puttern Analysis and Machine Intelligence, 39(2):301-3) 2016.
[28] Jie Chen, Dong Yi, Jimei Yang. Guoying Zhao, Stan Z Li, and Matti Pictikainen. Learning mappings for face synthesis from near infrared to visual light images. In Proceedings of the CVPR, pages 156-163, 2009.
[29] Bui Tuong Phong. Illumination for computer generated pictures. Communications of the ACM, 18(6)311- 317, 1975.
[30] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai L., and Li Fei-Fei. Imagenct: large-scale hicrarchical image datahase. Proceedings of the CVPR, pages 248-255, 2009.
[31] Ancong Wo, Wei-Shi Zheng, Hong Xing Yu, Shaogang Gong, and Jianhuang Lai. Rgb-infrared cross- modality person re-identification. In Proceedings of the ICCV, pages 5380-5389, 2017.
[32] Dat Tien Nguyen, Hyung Gil Hong, Ki Wan Kim, and Kang Ryoung Park. Person recognition system based on a combination of body images from visible light and thermal cameras. Sensors, 17(3):605, 2017.
[33] Mang Ye, Xiangyuan Lan, Jiawei Li, and Pong C Yuen. Hierarchical discriminative learning for visible thermal person re-identification. Proceedings of the AAAI, 2018.
[34] Mang Ye, Xiangyuan Lan, Qingming Leng, and Jianbing Shen. Cross-modality person re-identification via modality-aware collaborative ensemble learning. IEEE Transactions on Image Processing, 2020.
[35] Yifan Sun, Liang Zheng, Yi Yang, Qi Tian, and Shengjin Wang. Beyond part models: Person retrieval with refined part pooling (and strong convolutional baseline). In Proceedings of the ECCV, pages 480-496, 2018.
[36] Mang Ye, Jianhing Shen, David J Crandall, Ling Shao, and Jiebo Luo. Dynamic dual-attentive aggregation learning for visible-infrared person re- re-idenifiicatinn. In Proceedings of the ECCV, 2020.
[37] Xin Hao, Sanyuan Zhao, Mang Ye, and Jianbing Shen. Cross-modality person re-identification via modality confusion and center aggregation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 16403 16412, 2021.
[38] Hao Luo, Youzhi Gu, Xingyu Liao, Shenqi Lai, and Wei Jiang. Bag of tricks and a strong baseline for deep person re- re-ideniification. In Proceedings of the CVPR Workshops, June 2019.
[39] Yukang Zhang and Hanzi Wang. Diverse embedding expansion network and low- lightight cross cross- modality benchmark for visible-in inffe-inrrreed person re- re-ideniffiaatinn In Proceedings of the IEEE/CVF conference on computer vision and pattem recognition, pages 2153-2162, 2023.
12

---

[40] Mang Ye, Zheng Wang, Xiangyuan Lan, and Pong C Yuen. Visible thermal person re-identification via dual-constrained top-ranking. In Proceedings the IJCAI, pages 1092-1099, 2018.
[41] Seokeon Choi, Sumin Lee, Youngeun Kim, Tackyung Kim, and Changick Kim. Hi-emd: Hierarchical cross-modality disentanglement for visible-infrared person re-identification. In Proceedings of the CVPR, pages 10257-10266, 2020.
[42] Hyunjong Park, Sanghoon Lee, Junghyup Lee, and Bumsub Ham. Learning by aligning: Visible-infrared person re-identification using cross-mode modal correspondences. In Proceedings of the IEEE/CVF international conference on computer vision, pages 12046 12055, 2021.
[43] Yehansen Chen, Lin Wan, Zhihang Li. Qianyan Jing, and Zongyuan Sun. Neural feature search for rgb-infrared person re-identification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 587-597, 2021.
[44] Chaoyou Fu, Yibo Hu, Xiang Wu, Hailin Shi, Tao Mei, and Ran He. Cm-nas: Cross modallmy neural archi- tecture search for visible-infrared person re- e--ieetifficaionn In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1823-118322 2021.
[45] Qiang Zhang, Changzhou Lai, Jianan Liu, Nianchang Huang, and Jungong Han. Fmenet: Feature-leve modality compensation for visible-infrared person re-identification. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7349 7358, 2022.
[46] Ziyu Wei, Xi Yang, Nannan Wang, and Xinbo Gao. Syneretic modality collaborative learning for visible infrared person re-identification. Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 225-34, 2021.
[47] Qiong Wu, Pingyang Dai, Chen, Chia- Wen Lin, Yongjian Wu, Feiyue Huang, Bineng Zhong, and Rongrong Ji. Discover cross-modality nuances visihle-infrared person re-identification. Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 4330-4339, 2021.
[48] Jialun Liu, Yifan Sun, Feng Zhu, Hongbin Pei, Yi Yang, and Wenhui Li. Learning memory- aemmer-augmmented unidirectional metrics for cross-modality person re-identification. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 19366-19375, 2022.
[49] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition scale. arXiv preprint arXiv:2010. 11929, 2020.
13

---

A Appendix supplemental material
A.1 Additional Explanation of Radical Random Linear Enhancement
Algorithm 1: Radical Random Linear Enhancement
Input: I: Input image; C, H and W: Image channel and size; p: Probability of the LTG; Smin and Smaz: Area of the selected region; Ymin and Tmani Aspect of the selected region; bmin: Terminate the LTG;
Output: Enhanced image 1
Initialization: P1 < (0nd(0, 1):
if 2 then 1 1
return r";
else
M OOee(C, W);
while True do
S, -RRannd Smiir Smaz)
x W H;
< ---addandda Tmaz) )
H, + VS, Tri W, f- Se:
24 ROn W); yr Rand H))
if Ør + Wr < W and + Hr S H then
Lselect < (C, Tr, Mr, Tr + Wr, yr H,);
Mselect t (C, Iy, Br 2t + Wr, Ur + H,);
for to C do
Omaz "ox(enelect)"
a t- Omax x fg: I(I.) t lei M(Mo) + a x Mci
end end
if min < Emin then
I$ t I:
Return |"
end
end end
In Radical Random Linear Enhancement (RRLE), we use a U-shape beta distribution instead of the uniform distribution to generate the linear factor. Here, show example of modality discrepancy under different linear factors. Following the ahove setting, uniformly segmented 100 randomly selected images into six parts from the top to bottom and multiplied each part by a linear factor. Then, we send the new images and original images into an ImageNet [30] pre-trained Resnet-50 [26] and visualization of the feature space. As shown in Figure. 6, when using small linear factor may not be enough to bring a significant modality gap. U-shaped beta distribution to drive more dramatic changes in the linear factors.
Thus,
Meanwhile. in this section, provide detailed presentation of the RRLE, including detailed procedure of the RRLE in Alg.
14

---

1.5
1.5
0.5
0.9
1.1
x 1.1
0.9
Figure 6: A example of modality discrepancy. The dot and forks refer to the sample with and without linear transformation. Clearly, Small linear factors may not be so efficient in generating images with significant modality gap in the training stage.
NeurIPS Paper Checklist
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction of this paper accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings.
is fine to include aspirational goals motivation long it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We have discussed the limitations of this work in Section
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings. model well- sell-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated practice what implications would
The authors should reflect the scope of the claims made, e.g., if the approach only tested on a few datasets or with few runs. general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-tex, text system might not be used reliably to provide closed captions for online lectures because fails to handle technical jargon.
factors that influence performance the approach.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
15

---

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification:
We have introduced all the details in our thcorctical result.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross-referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of paper should be complemented by
formal proof' provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification:
This paper has provided detailed information for reproducing the experimental results.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example. if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to hosted model (e.g., in the case of a large language model), releasing of model checkpoint, or other means that appropriate to the research performed.
While NeurIPS does not require relcasing code, the conference does require all submissions provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution a new model (c.g.. a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (c.g.. with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, which case authors are welcome to describe the particular way they provide for reprodueibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g.. registered users), but it should be possible for other researchers to have some path to reproducing verifying the results.
16

---

5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
httpa:// http:://gthib..c///attnng5222222 wtt..///toonen/////.....
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https: :/tpps:////pp..//////peees http://iiippp.pppp.oooee cc/public/ SridessCodeSubmiee httsSubbbsssooooocoyyy. for more details.
While we encourage the release of code and data, we understand that this might not be possible,
"No" is acceptable answer. Papers cannot be rejected simply for not including code, unless this is central the contribution (c.g., for
open-source benchmark).
The instructions should contain the exact command and environment needed the results. See the NeurlPS code and data submission guidelines (https: //tps/ipps..../// ctlappp.ccpppiiee cc/public/ guides/Cor Leles/oooeeubaiiii ssi ssissssoooooooooooo oosiionoolieieeoo icy) for more details.
reproduce
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended the paper) recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g.. data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
This paper has specified all the training and test details, such as dataset and hyperparam- eters.
Justification:
Guidelines:
The answer NA that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary appreciate the results and make sense of them.
The full details be provided either with the code, in appendix, supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate informa- tion about the statistical significance of the experiments?
Answer: [No]
Justification; This paper does not provide error bars to limited resources. Also, previous methods do not provide error hars either.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, statistical significance tests, least for the experiments that support the main claims the paper.
The factors variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the enror bars should be explained (closed form formula, call to library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
17

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report 2-sigma enror bar than state that they have 96% CI, if the hypothesis of Normality of errors not verified.
For asymmetric distributions, the authors should be careful symmetric error hars would yield results that are out of range (c.g. negative error rates). error bars are reported tables or plots, The authors should explain the text how they were calculated and reference the corresponding ligures or tables in the text.
show in tables or figures
8. Experiments Compute Resources
Question: For each experiment, docs the paper provide sufficient information on the computer resources (type of compute workers, memory, time of exccution) needed reproducc the experiments?
Answer: [Yes]
Justification:
This paper has provided sufficient information the computer resources.
Guidelines:
answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code Ethics pestps:/neurriee cttpppp...pppppeee httpppbbicc//hhiccucilll.l...a..
Answer: [Yes]
Justification:
This paper meets the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure preserve anonymity (e.g., if there is special consideration due to laws regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts the work performed?
Answer: [NA]
Justification: This paper has no societal impact to the best of our knowledge.
Guidelines:
The answer NA means that there is no socictal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deploy- of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used generate deepfakes for disinformation. On the other hand, it not needed point out that generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
18

---

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitor- ing misuse, mechanisms to monitor how a system learns from feedback over time, improving efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g. pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
Justification: This paper poses no such risks since does not release data or models that risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines restrictions access the model implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
This paper has properly cited the original paper that produced the code package or dataset.
Guidelines:
answer means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, possible, include URL.
The name of the license (e.g.. CC-BY 4.0) should be included for each asset. For scraped data from particular source (c.g., website), the copyright and terms of service of that source should provided.
If assets released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license a dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are introduced in the paper well documented and is the documentation provided alongside assets?
Answer: [NA]
Justification: This paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/modde as part of their sub- missions via structured templates. This includes details about training. license. limitations.
The paper should discuss whether and how consent obtained from people whose asset used.
At submission time, remember anonymize your assets (if applicable). You either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
19

---

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, well as details about compensation (if any)?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in supplemental material is fine, but main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on requirements your country or institution) were obtained?
Question:
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
answer NA means that paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, approval (or equivalent) may be required for any human subjects research. If you ohtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applica- ble), such the institution conducting the review.
20


Reviews:
[{"review_id": "WrSapbJmDQ", "reviewer": "Reviewer_SzQ4", "summary": "This paper presents a novel approach to addressing the challenges in cross-spectral Re-ID, particularly the modality discrepancy between visible (VIS) and near-infrared (NIR) images. The authors propose a unified perspective based on the Lambertian reflection model to understand and categorize data augmentation strategies. This model helps in analyzing how local linear transformations can bridge the gap between different spectral images.", "questions": "(1) Why is [$\\lambda_r$, $\\lambda_g$, $\\lambda_b$] set to [0.299, 0.587, 0.114]? The authors should perform as many ablation experiments as possible.\n\n(2) How sensitive is the performance of the RLE strategy to different parameter settings? Have the authors explored the impact of varying these parameters systematically?\n\n(3) How does the RLE strategy compare with other advanced data augmentation techniques in the context of cross-spectral re-identification?\n\n(4) How applicable is the RLE strategy in scenarios where the Lambertian reflection model does not hold? Have the authors tested the method in such conditions?\n\n(5) Can the authors provide a more detailed explanation of how the RLE strategy works?\n\n(6) Figure 2 is very similar to PCB, can the authors do a comparative analysis accordingly?", "limitations": "The length of the related work is obviously insufficient. Meanwhile, the data enhancement has been explored in the Re-ID field for many years, the novelty is not very impressive.", "rating": 3, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "**Originality:** The paper introduces a novel perspective on cross-spectral Re-ID by leveraging the Lambertian reflection model. This approach provides a unique and insightful framework for understanding the modality discrepancy between visible (VIS) and near-infrared (NIR) images.\n\n**Clarity:** The authors provide a thorough background on cross-spectral Re-ID and the Lambertian reflection model, ensuring that readers can grasp the context and significance of their contributions.\n\n**Significance:**  By offering a unified perspective on data augmentation and introducing the RLE strategy, the authors address a critical challenge in surveillance and other applications involving spectral images.", "weaknesses": "(1) The paper lacks a detailed analysis of the sensitivity of the RLE strategy to its parameters involved in RLE, MRLE, and RRLE. Understanding how different parameter settings impact performance is crucial for practical deployment.\n\n(2) While the paper demonstrates performance improvements, there is limited discussion of the real-world applicability and potential deployment challenges of RLE strategies.\n\n(3) The paper focuses on the proposed RLE strategy but does not provide a detailed comparative analysis with other advanced data augmentation techniques beyond traditional methods.\n\n(4) The reliance on the Lambertian reflection model may limit the generalizability of the proposed methods to scenarios where this model does not hold.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your constructive feedback. Below, we respond to your key concerns point by point.\n\n>**Q1: Why is [lambda_r, lambda_g, lambda_g] set to [0.299, 0.587, 0.114]?**\n\n**R1:**  In fact, you may have misunderstood MRLE. As mentioned in Line 180~183, the [0.299, 0.587, 0.114] are not adopted for MRLE but come from the 'transforms.RandomGrayscale' function in the torchvision library. This is a classic grayscale method that previous works widely used. However, such transformation shows limited transformation results. Therefore, as described in Lines 184-194 and Eq. (6), we proposed MRLE, where the $\\lambda_r$, $\\lambda_g$, and $\\lambda_b$ are randomly sampled from a Beta distribution in every transformation. It provides more diverse augmentation during training, enhancing the model's capability. The related experiments in Tbl. 1 demonstrate the effectiveness of the proposed MRLE. We hope this response clarifies our approach and the rationale behind it.\n\n>**Q2: How sensitive is the performance of the RLE strategy to different parameter settings? Have the authors explored the impact of varying these parameters systematically?**\n\n**R2:**  Yes, we have explored different parameters and shown results in Table 3 of the paper. The results indicate that RLE is sensitive to parameter selection. For instance, $\\beta_m$ can lead to a 2.4% increase in the R-1. It is important to note that $\\beta_m$ and $\\beta_r$ control the shape of the sampling function, while $t_{min}$ controls the stopping time. We selected the most suitable parameters based on ablation experiments.\n\n>**Q3: How does the RLE strategy compare with other advanced data augmentation techniques?**\n\n**R3:**  Following your suggestion, we have included a comparison of our proposed method with the recently released CAJ [1] on the SYSU-MM01 dataset. The results, shown below, prove the effectiveness of our approach. We will add these experiments to the paper to make our conclusions more robust (More detailed experimental results are provided in Table 1 of the PDF rebuttal).\n| Method | R-1 | mAP | mINP |\n|:------|:-------:|------:|-----:|\n| CAJ | 73.5 | 69.4 | 55.4 |\n| **Ours** | **75.4** | **72.4** | **60.9** |\n\n>**Q4: How applicable is the RLE in scenarios where the reflection model does not hold?**\n\n**R4:** Thank you for the interesting question. Generally, the Lambertian model is suitable for most real-world applications, and major datasets, which makes RLE work well in both datasets. But the problem you mentioned is still a good suggestion. Since the Lambertian model may not work well when facing strong specular reflection or complex weather environments. Under these conditions, combining RLE with advanced image deraining/dehazing or light enhancement methods may be necessary. Your suggestion indeed points us toward future research directions, and we will include this discussion in the limitation section. Thank you again for your constructive feedback.\n\n>**Q5: Can the authors provide a more detailed explanation of how the RLE works?**\n\n**R5:** Thank you for your question. As mentioned in Sec. 3, the cross-spectral modality discrepancy mainly comes from the local linear transformation on different surfaces. Therefore, to overcome the modality discrepancy caused by this phenomenon, RLE aims to mimic such a linear transformation to provide a more diverse training set, thus encouraging the network to be robust to such a transformation. Meanwhile, the detailed processing of RRLE is also provided in Appendix A.1. \n\n>**Q6: Fig. 3 is very similar to PCB, can the authors do a comparative analysis accordingly?**\n\n**R6:**  The similarity you noted might be due to both PCB and Fig. 3 employing a horizontal segmentation strategy. However, Fig. 3 is not related to PCB. Fig. 3 presents a validation visualization experiment aimed at showing that the modality gap results from diverse local linear transformations on the image. As mentioned in Lines 160~165, we uniformly segmented 100 randomly selected images into six parts from top to bottom and multiplied each part by a linear factor. These images were then fed into ResNet-50 to observe the differences in feature space before and after the transformation. This transformation is directly applied to the image. On the other hand, PCB is a model structure that segments the features extracted from ResNet to obtain fine-grained details. \n\n>**Q7: The length of the related work is obviously insufficient.**  \n\n**R7:**  Thank you for pointing out this issue. Following your suggestion, we have included additional classic and recent related works in Sec. 2 to make it more comprehensive. If you have any recommended works, please provide them in the discussion. We are willing to include them.\n\n>**Q8: The data enhancement has been explored for many years, but the novelty is not very impressive.**\n\n**R8:**  Our core contribution lies in providing a unified perspective to explain the discrepancy caused by the cross-spectral transformation, rather than purely focusing on data augmentation. **_This is the first work in this area to provide new insight that will guide us to understand which kind of data augmentation is needed and how to design strategies to overcome the cross-spectral modality discrepancy_**. It even explains why complex GAN-based models [2] may not be as efficient as basic data augmentations in this context [1]. We believe RLE is just one attempt at using this perspective to design a data augmentation strategy. It may inspire more image generation or data augmentation strategies and even network designs.\n\n[1] Channel augmentation for visible-infrared re-identification. TPAMI 2023.\n\n[2] RGB-infrared cross-modality person re-identification via joint pixel and feature alignment. ICCV 2019.\n\n**Finally**, we hope our response has adequately addressed your concerns. If you have any further suggestions, please feel free to discuss them with us. We will provide the corresponding results as soon as possible.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ISvmmujSHI", "reviewer": "Reviewer_muwA", "summary": "This paper presents a unified perspective that reconsiders data augmentation strategies in cross-spectral re-identification. The authors identify that the main source of cross-spectral modality discrepancies stems from various local linear transformations due to material diversity. To address this, the authors propose a novel Random Linear Enhancement (RLE) strategy, effectively leveraging this unified perspective. Experimental results demonstrate that the proposed method significantly improves the visible-infrared re-identification performance.", "questions": "Please see the weaknesses :)", "limitations": "The authors point out the potential limitations of the proposed method.", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.\tThe novel approach of considering cross-spectral data augmentation from the perspective of the reflection model is well-motivated and innovative.\n\t2.\tThe paper is technically sound, with an elegant motivation and formulation of the proposed method.\n\t3.\tExtensive testing on various benchmark datasets, including SYSU-MM01 and RegDB, showcases the method’s effectiveness and robustness compared to state-of-the-art methods.", "weaknesses": "(1) Some details are simplified in the paper. For instance, visualization results of the proposed RLE are not shown, and the baseline structure could be included in the appendix.\n(2) In MRLE, the sum of lambda_r, lambda_g, and lambda_b is constrained to 1, which may seem unnecessary from the basic formulation perspective. Alternative values like [0.2, 0.2, 0.5] might also satisfy the proposed perspective.\n(3) This paper could do a better work by citing, comparing, and building some connections with more recently-published literatures in the ReID community such as “Robust Object Re-identification with Coupled Noisy Labels”.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your positive and constructive feedback which inspired us a lot. Below, we respond to your key concerns point by point.\n\n>**Q1:  Some details are simplified in the paper. For instance, visualization results of the proposed RLE are not shown, and the baseline structure could be included in the appendix.**\n\n**R1:**  Thanks for your suggestions. We have added some visualization results of RLE in the paper. Meanwhile, we also provide several visualization results in Figure 1 of the pdf-version rebuttal. Please kindly refer to it. Through the visualization results, we can clearly observe the differences between MRLE and RRLE. Specifically, MRLE aims to provide diverse linear augmentation results that strictly adhere to the initial reflection correlation prior, while RRLE aims to provide diverse linear augmentation results under limited risk-taking.\n\n>**Q2: In MRLE, the sum of $\\lambda_r$, $\\lambda_g$, and $\\lambda_b$ is constrained to 1, which may seem unnecessary from the basic formulation perspective. Alternative values like [0.2, 0.2, 0.5] might also satisfy the proposed perspective.**\n\n**R2:** As shown in Fig. 3, the linear transformation brings limited modality discrepancy. So, the transformation under [0.2, 0.2, 0.5] can also be considered as a linear transformation after the transformation under [0.22, 0.22, 0.56], which will bring limited difference. Therefore, we may only need to consider the mix percentage of different channels. We also provide the related experimental results on the SYSU all-search mode below to show the influence of the sum constraint (More detailed experimental results are provided in Table 2 of the PDF version rebuttal). \n\n| Sum Constraint| R-1 | mAP | mINP |\n|:------:|:-------:|------:|-----:|\n| | 66.3 | 63.8 | 50.3 |\n| &#10004; | **70.2** | **67.0** | **63.5** |\n\nSince removing the sum constraint may lead to some bad cases (e.g. all parameters are very small), the performance is worse than the strategy with the sum constraint.\n\n>**Q3: This paper could do a better work by citing, comparing, and building some connections with more recently-published literatures in the ReID community such as “Robust Object Re-identification with Coupled Noisy Labels”.**\n\n**R3:** Thanks for your suggestion. We have included several recent related works in Sec. 2 including this paper to make our work more comprehensive. If you have any other recommended works that are related to this paper, you can provide them in the discussion. We are willing to add them in.\n\n**Finally,**  we hope our response has adequately addressed your concerns. If you have any further suggestions, please feel free to discuss them with us. We will provide the corresponding results as soon as possible.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PAGZBtOy3W", "reviewer": "Reviewer_gwge", "summary": "This paper explores data augmentation strategies for cross-spectral re-identification. The authors find that non-linear modal differences arise mainly from different linear transformations occurring on various material surfaces; all data enhancement strategies for cross-spectral re-identification aim to simulate such transformations. Based on these observations, they introduce a Random Linear Enhancement (RLE) augmentation method and further extend the boundaries of the moderate and radical transformations by Moderate Random Linear Enhancement (MRLE) and Radical Random Linear Enhancement (RRLE).", "questions": "See the weakness.", "limitations": "There is no analysis of limitation in the paper. As mentioned in weakness, the correlation between the observations based on the physical model in this paper and the subsequent methods in the article is not strong, and the results of the article did not reach a SOTA level, and the generalisability of the data augmentation methods proposed in the paper deserves further validation.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "In this paper, by introducing the Lambertian model, the analysis finds that non-linear modal differences arise mainly from different linear transformations occurring on various material surfaces, which is an intuitive conclusion.\n\nThe RLE method proposed by the authors together with the existing random erasing achieved good experimental results.", "weaknesses": "Innovation is limited, and although the authors' introduction of the physical model is striking, the conclusions obtained can not actually guide the authors well in designing their methodology.\n\nThe experimental results in this paper do not reach SOTA, and the article lacks a comparison with the SOTA method [1] [2].\n\nThe ablation experiments in the article are not comprehensive, lacking experiments on smaller $t_{min}$ than 0.1 as well as experiments on single terms in mixed transformations.\n\nThe paper lacks an analysis of the performance improvement due to the joint use of random erasing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your constructive feedback which helps us a lot. Below, we respond to your key concerns point by point.\n\n>**Q1: Innovation is limited, and although the authors' introduction of the physical model is striking, the conclusions obtained can not actually guide the authors well in designing their methodology.**\n\n**R1:** Thank you for your comments and your high recognition of the physical model we introduced. The core contribution of our work is that it provides a new perspective of data augmentation for cross-spectral re-identification based on the reflection model. Compared to empirical data augmentation methods that pursue visual similarity, **_this is the first work that has modeled the cross-spectral transformation to guide the design of data augmentation._** It helps us to understand where the problem lies in this task and guides us to find what is needed for this task. In fact, the proposed RLE was totally designed under the guidance of the reflection prior, which has strong directive significance. \n\nAs we mentioned in Section 3, the cross-spectral modality discrepancy mainly comes from the diverse local linear transformation on different material surfaces. Therefore, to overcome the modality discrepancy caused by the cross-spectral transformation, RLE aims to mimic such diverse linear transformations, thus encouraging the network to be robust to these transformations. Specifically, MRLE aims to provide diverse linear augmentation results that strictly adhere to the initial reflection correlation prior, while RRLE aims to provide diverse linear augmentation results under limited risk-taking.\n\nThe results in Table 1 and Table 4 of this paper show that the proposed RLE brings significant performance improvements, proving that the theoretical guidance of this paper withstands scrutiny. This has also been highly recognized by Reviewer muwA, who noted, \"The paper is technically sound, with an elegant motivation and formulation of the proposed method.\"\n\n>**Q2: The experimental results in this paper do not reach SOTA, and the article lacks a comparison with the SOTA method [1] [2].**\n\n**R2:** Thank you for your comments. However, it is regrettable that you did not provide the references for you mentioned method. In fact, as a data augmentation strategy, our proposed RLE is a plug-and-play component that can be applied to any state-of-the-art method to achieve improvements. In Table 2 and Line 280\\~285, we demonstrate the RLE can boost a vanilla baseline to an impressive performance. Meanwhile, in Line 294\\~304 and Table 4, we embedded RLE into the latest open-sourced method DEEN [1] and achieved significant performance enhancements, reaching new SOTA results.\n\nSo, if you are interested and notice any new SOTA that we have missed, please provide a reference for related works (_preferably with open-source code due to time constraints_). We are willing to provide the performance by combining the SOTAs you mentioned with RLE.\n\n[1] Zhang, Yukang, and Hanzi Wang. \"Diverse embedding expansion network and low-light cross-modality benchmark for visible-infrared person re-identification.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2023.\n\n>**Q3: The ablation experiments in the article are not comprehensive, lacking experiments on smaller $t_{min}$ than 0.1 as well as experiments on single terms in mixed transformations.**\n\n**R3:** Following your suggestion, we have added the experiments under more $t_{min}$ settings in Table 3(c). Meanwhile, we also added experiments on each single term in the ablation.\nHerein, we show the results under SYSU-MM01 all-search mode as below (More detailed experimental results are provided in Table 3 and Table 4 of the PDF version rebuttal). \n\n$t_{min}$:\n| $t_{min}$ | R-1 | mAP | mINP |\n|:------|:-------:|------:|-----:|\n| 0.3 | 72.9 | 70.5 | 58.2 |\n| 0.2 | 73.8 | 71.3 | 59.0 |\n| **0.1** | **74.2** | **71.8** | **60.4** |\n| 0.01 | 73.8 | 71.7 | 59.8 |\n| 0.001 | 73.6 | 71.3 | 59.5 |\n\n**single terms**:\n| Method | R-1 | mAP | mINP |\n|:------|:-------:|------:|-----:|\n| R only | 65.1 | 62.8 | 49.7 |\n| G only | 65.2 | 63.0 | 50.1 |\n| B only | 63.4 | 61.5 | 48.4 |\n| **MRLE** | **70.2** | **67.0** | **53.5** |\n\n>**Q4: The paper lacks an analysis of the performance improvement due to the joint use of random erasing.**\n\n**R4:** Thank you for your comment. As explained in Lines 271-274, we have clarified why RLE can be used in conjunction with random erasing. However, we are willing to strengthen this part based on your suggestion. Specifically, although 'RE' can be considered a special case of RRLE with a linear factor of 0, RRLE encourages images to undergo more times of transformations while preventing the loss of information. Therefore, RRLE and RE have different valuation perspectives and can be effectively used together.\n\n>**Q5: There is no analysis of limitation in the paper.**\n\n**R5:** Thanks for your suggestion. In fact, we have discussed the limitation of the RLE in Sec. 6 ‘Limitations and Broader Impact’. But we are willing to strengthen related discussion following your suggestion. In general, the core limitation of RLE is that it is founded based on a Lambertain model under cross-spectral conditions. Therefore, it may limit its adaptability to other scenarios. Under extremely bad weather such as heavy rain, fog, or limited illumination, it may show limited improvement since the Lambertain model may not work well. Under these conditions, it may be necessary to combine the RLE with advanced image deraining/dehazing or illumination enhancement strategies. However, these limitations are not the main challenge we discussed in this paper but may push forward future works to explore those complex circumstances.\n\n**Finally,** we hope our response has adequately addressed your concerns. If you have any further suggestions, please feel free to discuss them with us. We will provide the corresponding results as soon as possible.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gjPDd720IJ", "reviewer": "Reviewer_DZyN", "summary": "This paper provides a unified perspective to consider the data augmentation strategies in cross-spectral re-identification. Based on the Lambertain model, the author finds that the cross-spectral discrepancy is induced by multiple local linear transformations. Furthermore, the authors propose a robust linear enhancement (RLE) to imitate such a transformation moderately and radically. Visualization and experimental results in related datasets show the proposed RLE's effectiveness in cross-spectral circumstances.", "questions": "Please check the weakness part for more details.", "limitations": "The authors have discussed the limitations of the proposed method in Section 5.", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The motivation of this paper seems to be valid and intriguing, and the proposed unified perspective makes sense in this topic.\n2. The paper is easy to follow and well-written.\n3. The proposed method shows superior performance on both RegDB and SYSU-MM01 datasets.", "weaknesses": "1. As a data augmentation strategy, it will be better if the author can provide several visualization examples.\n2. In Table 4, I wonder about the detailed augmentation strategy that contains in the mentioned 'ours'. In DEEN, it already contains grayscale transformation and random erasing. So, how do you insert the RLE in such a structure?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your constructive and positive feedback which inspired us a lot. Below, we respond to your key concerns point by point.\n\n>**Q1: As a data augmentation strategy, it will be better if the author can provide several visualization examples.**\n\n**R1:** Thanks for your suggestion. We have provided some visualization results in Figure 1 of the attached PDF version rebuttal following your suggestion and added this part in the final-version paper. Therefore, please kindly refer to the attached PDF. Through the visualization results, we can clearly observe the differences between MRLE and RRLE. Specifically, MRLE aims to provide diverse linear augmentation results that strictly adhere to the initial reflection correlation prior, while RRLE aims to provide diverse linear augmentation results under limited risk-taking.\n\n>**Q2: In Table 4, I wonder about the detailed augmentation strategy that contains in the mentioned 'ours'. In DEEN, it already contains grayscale transformation and random erasing. So, how do you insert the RLE in such a structure?**\n\n**R2:** We feel sorry for the inconvenience caused. Since the DEEN [1] already contains the random grayscale and random erasing for data augmentation, we remove the random grayscale and add the RLE just like what we applied for the baseline method in Tbl. 1. Following your suggestion, we have strengthened this part to make it more clear.\n\n[1] Diverse embedding expansion network and low-light cross-modality benchmark for visible-infrared person re-identification. CVPR 2023.\n\n**Finally**, we hope our response has adequately addressed your concerns. If you have any further suggestions, please feel free to discuss them with us. We will provide the corresponding results as soon as possible.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Dear Reviewers,

We would like to thank all reviewers for providing constructive feedback that helped us improve the paper. We are encouraged that reviews think our paper:

* "provides a unique and insightful framework for understanding the modality discrepancy between visible (VIS) and near-infrared (NIR) images" (Reviewer SzQ4).

* "an elegant motivation and formulation of the proposed method" (Reviewer muwA).

* "motivation of this paper seems to be valid and intriguing, and the proposed unified perspective makes sense in this topic" (Reviewer DZyN).

We have been working diligently on improving the paper on several fronts these days to address your critique. Specific feedbacks for every reviewer are provided below your review. **Herein, we have provided an additional  PDF version rebuttal including the visualization results and detailed experimental results you mentioned in your reviews.**

Best regards,

The Authors

Meta-review: The paper received one strong accept, one accept, one borderline accept, and one reject after rebuttal. While most reviewers agreed with the contribution of this work particularly for the VI-ReID domain, one reviewer carefully reviewed the paper from a more general perspective for the paper to be suited for NeurIPS. AC agrees with most points from both sides, and consider that for the particular VI-ReID domain, the paper's strengths overweigh its weaknesses, and therefore suggest acceptance of the paper. It will be of interests to researchers working on the VI-ReID domain. Authors are required to address the review comments and include the rebuttal materials to the camera-ready version. Especially, a number of Reviewer SzQ4's concerns are valid and include those discussions will definitely strengthen the current paper.
Decision: Accept (poster)