Feature-Level Adversarial Attacks and Ranking Disruption for Visible-Infrared Person Re-identification
Authors: Xi Yang
Huanling Liu
De Cheng
Nannan Wang
Xinbo Gao
Venue: NeurIPS 2024 poster (2024)
Keywords: Visible-Infrared Person Re-identification
Universal Adversarial Perturbation
Frequency-Spatial Attention Module
Auxiliary Quadruple Adversarial loss
Primary area: machine_vision

Abstract: Visible-infrared person re-identification (VIReID) is widely used in fields such as video surveillance and intelligent transportation, imposing higher demands on model security. In practice, the adversarial attacks based on VIReID aim to disrupt output ranking and quantify the security risks of models. Although numerous studies have been emerged on adversarial attacks and defenses in fields such as face recognition, person re-identification, and pedestrian detection, there is currently a lack of research on the security of VIReID systems. To this end, we propose to explore the vulnerabilities of VIReID systems and prevent potential serious losses due to insecurity. Compared to research on single-modality ReID, adversarial feature alignment and modality differences need to be particularly emphasized. Thus, we advocate for feature-level adversarial attacks to disrupt the output rankings of VIReID systems. To obtain adversarial features, we introduce \textit{Universal Adversarial Perturbations} (UAP) to simulate common disturbances in real-world environments. Additionally, we employ a \textit{Frequency-Spatial Attention Module} (FSAM), integrating frequency information extraction and spatial focusing mechanisms, and further emphasize important regional features from different domains on the shared features. This ensures that adversarial features maintain consistency within the feature space. Finally, we employ an \textit{Auxiliary Quadruple Adversarial Loss} to amplify the differences between modalities, thereby improving the distinction and recognition of features between visible and infrared images, which causes the system to output incorrect rankings. Extensive experiments on two VIReID benchmarks (i.e., SYSU-MM01, RegDB) and different systems validate the effectiveness of our method.

Feature-Level Adversarial Attacks and Ranking Disruption for Visible-Infrared Person Re-identification
Xi Yang', Huanling liu', De Cheng", Nannan Wang', Xinbo Gao²
Xidian University, "Chongqing University Posts and Telecommunications yangxexidian edu cn hu.nulliiggluusee xidixiiiin edu, en, acheng@xidian. edu. cn nnvang@xidian edu, cn, gaoxbecqupt. edu, cn
Abstract
Visible-infrared person re-identification (VIReID) is widely used in fields such video surveillance and intelligent transportation, imposing higher demands model security. In practice, the adversarial attacks based on VIReID aim to disrupt output ranking and quantify the security risks of models. Although numerous studies have been emerged on adversarial attacks and defenses in fields such face recognition, person re-identification, and pedestrian detection. there currently lack of research on the security of VIReID systems. To this end, we propose explore the vulnerabilities of VIReID systems and prevent potential serious losses due to insecurity. Compared to research single-modalig ReID, adversarial feature alignment and modality differences need to be particularly emphasized. Thus, we advocate for feature-level adversarial attacks to disrupt the output rankings VIReID systems. To obtain adversarial features, we introduce Universal Adversarial Perturbations (UAP) to simulate common disturbances in real-world environments. Additionally, we employ a Frequency- Spattil Attention Module (FSAM), integrating frequency information extraction and spatial focusing mechanisms, and further emphasize important regional features from different domains the shared features. This ensures that adversarial features maintain consistency within the feature space. Finally, we employ an Auxiliary Quadruple Adversarial Loss amplify the differences between modalities, thereby improving the distinction and recognition of features between visible and infrared images, which cause the system output incorrect rankings. Extensive experiments VIRelD) benchmarks (i.e., SYSU-MM01, RegDB) and different systems validate the effectiveness of our method.
Introduction
VIReID is widely applied in key tasks such as security monitoring [333. Suppose the law enforcement agency of a city uses ReID system to monitor public places for tracking criminal suspects. Internal personnel may attempt deceive the system by modifying the images O4 criminal suspects due to improper behavior or other reasons, in order to protect specific individuals. According to Figurq]] infrared adversarial samples will erroneously match visible samples, while visible adversarial samples will also erroneously match infrared samples. The credibility and stability of VIReID are crucial in such special application scenarios. However, there is currently insufficient theoretical research on the security of VIReID. Therefore, this paper explores how to obtain better adversarial features and how address the task characteristics VIReID modality differences and re-ranking.
Corresponning author.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Match
Match
Infrured Camera
Visible
Camera
Mismatch
(b)
Mismatch
Figure 1: Security risks of VIReID in the physical world. Images with added noise are referred to as adversarial samples. Red indicates that adversarial samples will incorrectly match pedestrians. Green indicates that clean samples will correctly match pedestrians.
The current research adversarial attacks in the fields of face recognition and person re- re-identification mainly focuses on digital attacks 77.), which refer to the manipulation, distortion, or tampering of digital images to deceive, disrupt, mislead systems, leading to erroneous identification results reduced recognition accuracy| [TO]. Adversarial attacks based on RelD focus on crafting adversarial samples suitable for visible images and disrupting internal rankings. In contrast, VIReID requires more consideration regarding the generalizability of attack methods under different imaging mecha- nisms and how align adversarial features between infrared and visible images. Moreover, dealing with two modalities necessitates comparing distances between modalities and within modalities. To address these challenges, we propose feature-level adversarial attacks and ranking disruption for VIReID.
First, adopt the method of universal adversarial perturbation (UAP) to generate adversarial samples, which seeks a set of universal perturbations independent of the image and can generalize well in deep neural networks. At the same time, it significantly lowers the threshold for implementing adversarial attacks and adapts to different systems. Secondly, to make visible and infrared images more consistent in the feature space, we propose frequency-spatial attention module, achieving adversarial feature alignment by unifying frequency and spatial features. Visible and infrared images generated under different imaging conditions; the former provides rich texture information, while the latter contains significant pixel amplitude information. This module uses fast Fourier transform to decompose features into amplitude and phase, corresponding to the texture details and spatial position information of images, respectively. Since these features are closely related to the spatial domain, spatial attention module is chosen for these two components to further emphasize or suppress different regions in the feature map. Additionally, weighted spatial attention module is applied to the shared features to maintain consistency in the feature space. This method not only focuses on the frequency domain features of visible and infrared images from different imaging conditions but also emphasizes their spatial features for significant pedestrian poses, thereby achieving adversarial feature alignment. To disrupt the ranking of identification results, we propose auxiliary quadruple adversarial loss function. The visible and infrared pedestrian features extracted by the first-stage model are used auxiliary features in the calculation process of the features loss function extracted in the second stage. By pulling the distance between the same modality and the same person closer and pushing the distance between different modalities and the that the intra-class distance smaller than the inter-class distance, the differences between modalities expanded and the ranking under different modalities is disrupted. By utilizing generated features containing multiple types of information, the network's 's ability to explore features at different levels is enhanced. That is, with four types of features, ensure double guarantee to achieve the goal of disrupting the ranking. The main contributions of this paper can be summarized as follows:
person farther, while also ensuring
We are the first to propose exploring the security of VIReID, considering the alignment of adversarial features across modalities in VIReID,
We propose frequency-spatin attention module that integrates frequency-domain features with spatial features to enhance the consistency and representation ability of adversarial features.

---

We design an auxiliary quadruple adversarial loss function, which utilizes auxiliary features to amplify the differences within and between modalities, thereby disrupting the ranking results.
2
Related Work
Infrare Person Re-identification. VIReID [12 Irfffrs to the technique of identifying and matching pedestrians from one modality to another using visible infrared images. Moreover, VIReID finds wide application in the field of security, enhancing the intelligence level of security systems. The significant differences between different modalities make VIReID challenging. To alleviate the modality discrepancy at the feature level, some methods adopt single-stream, dual-stream, or multi-stream networks [I6-19,, extracting shared features from different modalities by designing various attention mechanisms and loss functions 230) Ye et al. [T6] proposed the concept of metric learning, jointly optimizing codallty-speccffe and modality
matrices. Subsequently, Zhu et al. [18] proposed hetero-center loss, which for the first time shortens the distance between feature centers of the same identity, bridging the gap between features of the same pedestrian across different modalities. In addition, Ling et 24. devised a multi--cnsstaint similarity learning approach to comprehensively explore the relationships between cross-modal information. Meanwhile, as the posture and shape of pedestrians provide important information in the recognition process, these methods mainly focus on spatially enhancing feature representation. However, there are also differences in frequency information between visible and infrared images. Li et al. [25] proposed novel frequency-domain modality-invarian feature learning framework to reduce modality differences from frequency-domain perspective.
Ad Adversarial Attacks. In the fields of computer graphics and pattern recognition, adversarial attacks 2ttacks[26|29] are an important area of research. Many studies have revealed the vulnerability of deep models carefully crafted small perturbation, resulting significant errors with high confidence in predictions. Adversarial attacks aim to explore adversarial noise that causes deep learning models to behave abnormally. PG) belongs to the single-step attack algorithm, which optimizes the loss by quickly determining the direction of perturbation for input samples and calcu- lates the adversarial perturbation through backpropagation. The concept of iterative thinking was subsequently incorporated into FGSM, thereby leading to the development of Projected Gradient Descent (3T] (PGD). Subsequently, adversarial attacks on ReID were also studied, with the core idea of generating well-crafted adversarial examples disturbing ranking results. LTA used local grayscale iteration to generate adversarial examples, mainly focusing on disturbing the color of the original image. A mis-ranking formula was proposed by DMR (33 to increase the distance between images of the same pedestrian while decreasing the distance between images of different pedestrians, effectively disrupting the ranking results.
Cross-modality Attacks. Adversarial instances are widely present across various domains of visible images. In recent years, exploration of adversarial instances in the field of infrared pedestrian detection has begun. Osahor et al. (34] discussed perturbation by altering pixel values within infrared images. Subsequently, Zhu et [1.33 attempted for the first time to alter the infrared radiation distribution of the human body by simulating additional heat sources using set of small bulbs, generating physical adversarial examples. To be more casily implemented in the physical world, Wei et al. [36] 37) considered the different imaging mechanisms of visible and infrared sensors, proposing unified adversarial patch to execute cross-modality physical attacks. Meanwhile, we chose to perform security evaluation on visible-infrared person re-identification models proposed in recent years in the digital world.
3 Proposed Method
As shown in Figure 2 the overall structure of the proposed method adopts dual-stream ResNet network as the backbone. Firstly, Universal Adversarial Perturbation (UAP) are added separately to visible and infrared images. In the first stage, the Frequency-Spone Spattial Attention Module (FSAM) is embedded to extract frequency-domain spatial correlated features as auxiliary features for images of two different modalities, visible and infrared. Subsequently, through a shared module, further focus is applied to the temporal domain features. This completes the focus on spatial features in both frequency and normal domains, making the learned information more diversified, thereby completing
3

---

Xat
Shared block
Xy2
La
OOOOOO
SAM Module
BN
Visible
X11
X12
Intrared
Prequency Spattal Attention Modsle(SAMI
Ausiliary Quadruple Adversarial Loos(
X
X
ID2
FFT
IFFT
Sernow the mudhiliy DND
pull
Ap
push
Spatial Alleation SANI
Figure 2: Overview of the proposed method. FSAM consists FFT and IFFT along with spatial focusing module, focusing on the frequency-domal spatial characteristics of the image. We propose a auxiliary quadruple adversarial loss function and provide simple illustration of its operation.
the learning of features in the second stage. During the training phase, the features from the first stage used as auxiliary features and combined with the features from the second stage, all of which inputted into the Auxiliary Quadruple Adversarial Loss for optimizing the entire module.
3.1 Universal Adversarial Perturbation
Universal Adversarial Perturbation (UAP) aims to generate a single perturbation that can be added to any image from the same distribution, resulting networks are highly susceptible to this type of UAP, yet they remain imperceptible to the human eye. Defining a set images that satisfies distribution μ,, where g()) fits the output function, and after perturbation 5, the labels are not equal. That is:
cissciassiiiittion when added. Deep neural
61+
When 5 is constrained by two conditions simultaneously, the optimization problem of finding universal perturbation can be described follows:
s.t.1001z S Pi~p(e(T +))+ 2 c,
(2)
where P(-) represents probability, || lpp denotes the p-norm, indicates the magnitude of the perturbation to ensure that the adversarial perturbation is visually imperceptible, represents the (0,1) denotes the success rate of deception ensure the attack's 's success rate. The goal to find an adversarial perturbation 5 that can be added to all sample points and will result in misclassification of adversarial samples with probability require solving optimization problems or gradients of the model and is applied in scenarios where large number of adversarial samples need to be quickly generated.
data distribution,
1- e. The UAP algorithm does not
So we choose universal perturbation for addition, which can used on both visible and infrared images and can also adapt to scenarios with a large number of pedestrian samples in real-world environments. The introduction of UAP can make the model more independent specific data distributions and training sets, thereby reducing the model's dependence specific data and making the system more flexible and adaptable to challenges in different environments and scenarios. Visible and infrared images generate their respective adversarial samples follows:
Q.i Qui + dwis
Q Qir + dar-
For the input Q and Qir. adding perturbation((..:.6 related to the data distribution to each, the generated adversarial sample Qui. Qur deceive the system by exploiting visual similarity attacks.

---

3.2 Frequency-Spatial Attention Module
3.2.1 Fast Fourier Transform
The Fast Fourier Transform (FFT) is widely utilized the field of image processing to convert images into the frequency domain, enabling the analysis of the frequency components of the image. This aids understanding the overall structure, texture, and edge information of the image. Therefore, we combine the Fast Fourier Transform with spatial attention module to focus on the unique features in the frequency domain of the image, which aids in enhancing feature representation capabilities, First, we provide brief introduction to the basic concepts of the FFT. Given the feature æ E C output by network, its FFT can expressed as follows:
H- -- ΣΣ h=0 w==
F(x)(u, VHW
2th, w)e 2jm(u 2j #
(5)
where j represents the imaginary unit, u and are the horizontal and vertical coordinates of the æ, and F-) denotes the Fourier transform, C, H and W denote the number of the channel, height and
width of features. The frequency-domaii feature F(x) is represented
F(x) R(x) + (I().,
where Rxx) and [(x) represent the real and imaginary part of F(x). These real and imaginary parts
can be converted to amplitude and phase spectrums, which can be formulated as follows:
ul u) [R²(x)) I (ax)
(6)
(u,) R R(x)(u,
P(z) v) = arctan
(7)
As shown in Figure in the task of VI-ReID, the amplitude component captures the overall bright- ness and contrast of pedestrian images, reflect- ing the luminance and color information of the image, while the phase component captures the structural information and details of the pedes- trians, including their shape and outline, to help distinguish between different pedestrians' details and features. The combination of these compo- nents allows for the effective extraction of global and local features of pedestrians. By focusing fur- ther on spatial information characteristics in the phase component, attention to spatial information in the frequency domain is increased, enhancing the ability to express distinguishing features. the context of the features extracted by the net- work, we represent the amplitude component of the FFT as aa, and the phase component as p-- Given that the dual-branch network separately extracts visible and infrared pedestrian features,
(b)
Figure 3: Decomposition and reconstruction of visible and infrared image in the frequency do- main. (a) denote visible and infrared images of pedestrian; (b)present the reconstructed images with amplitude information only: (c) are the re- constructed images with phase information only.
the amplitude component of the visible pedestrian features after FFT is represented as T and the phase component Tep- while the amplitude component of the infrared pedestrian features after FFT is represented as and the phase component as Tip.
3.2.2 Spatial Attention Module
The spatial attention module generates spatial attention maps using the spatial relationships within the features. It focuses on the distribution of information, locating the position and shape of the subject while reducing background information interference. We apply max pooling and average pooling operations along the channel axis. These two pooling operations respectively extract the maximum and average value information from the feature map and concatenate them form richer feature descriptor. Pooling is performed along the channel axis, which helps highlight information-rich
5

---

regions in the feature map and better locate key information. In summary, the specific calculation process is as follows:
M Sigmoid( ([Maar/o(())))) Avg Pog
where (x) denotes the weight distribution generated by applying the convolution layer and Sigmoid denotes the sigmoid function. f represents a convolution operation with the filter size of MaxPool and AugPool(2 represent the features after passing through the max-pooling layer and average-pooling layer, respectively.
Given that the phase component involves more spatial information, we sequentially apply the spatial attention module to obtain the new phase component(2 up which can be expressed as follows:
up = My (2xpp) X Cop.
ip (x 83 Tip:
(10)
where @ denotes element-wise multiplication, My and (aap))) represent the attention weights generated by the feature phase components. The combination of the original amplitude component and the new phase component can reconstruct the original feature information through inverse fast
Fourier transform(IFFT). At this point, the network outputs features T. and Tyy, which will serve as auxiliary features. This process can be described as:
Tv1 = IFFT up: Too);
(11)
F3 (F Ftaa.
(12)
The features are then further input into shared module T in the model to complete feature ex- traction. At this point, choosing to pass through the spatial attention module allows for focusing on spatial information in the original domain. This two-stage process completes the focus on both frequency domain and original domain spatial information, enhancing the representation capability of distinguishing features. Thus, the final extracted visible and infrared features can be expressed as:
2 = (T((T( (zx1)) @ T(xa)).
(13)
322 = (T(aa) @ T(xa),
(14)
where M. and M, indicates the weight distribution generated by the second-stage SAM module, resulting in the final stage features, namely 222 and 222.
3.3 Auxiliary Quadruple Adversarial Loss
We propose an auxiliary quadruplet adversarial loss function to disrupt the system's 's output ranking. This method effectively adapts to RelD issues by attacking the predicted ranking results. Additionally, considering the modality differences involved in VIReID, we introduce auxiliary features to fully leverage the information disparities between modalities, erroneously outputting cross-modality ranking results. In this process, there are both modality differences and identity differences. Therefore, we approach it from two aspects: under the condition of the same modality, minimizing the distance between the same identities and maximizing the distance between different identities; under the condition of the same identity, minimizing the distance between the same modalities and maximizing the distance between different modalities. Let's start by controlling the same modality case:
L(x 2.1) [D(a/22 aaa)) D( xx22
(15)
where x represents the auxiliary feature, j and k denote different pedestrians, D(..) represents the Euclidean distance between two feature vectors and [x]_ max(x, 0). When the modalities are the same, the task is transformed into single-modalit ReID. This problem discusses the disruption of matching pairs across modalities, while still maintaining the original requirement for the same modality, namely, encouraging that the maximum distance between the most easily identifiable pairs of images across identities is still less than the minimum distance between the most easily identifiable pairs of images within an identity, ensuring that the sorting output within the same modality is normal.

---

This part ensures that the original order and matching relationships within the modality are not disrupted. When the identities are consistent across modalities:
L( Del, z2) = D(2) 2(11) - D(ar/2 o
(16)
we encourage reducing the distance between pedestrians in the same modality while increasing the distance between pedestrians in different modalities. This strategy aims to narrow the distance within the same modality while widening the distance between different modalities, thereby increasing the modalities differences. It disperses the features the feature space, making more challenging for the model to cluster features. This is intended to impact the matching and sorting results across modalities. The loss function augmented with visible features can be expressed as:
Li T22, 2vl) =
Σ 3,
D( o/1)) D(2 r/2)) - D( 252)) + a) {],
(17)
27
where N is the number of person ID in a mini-batch. Meanwhile, using more discriminative embedding centers(c.1, Cr2. Cı1 C22) for each class, we introduce margin term to balance the two terms. Thus, the loss function augmented with visible features and infrared features can be expressed follows:
N
[(c,2, C52, co1)=
Σ
D(c22 c.)) D(c C.1) De +
(18)
1,k=1
jØk
N
L(C;2 Co2, cn) =
Σ
Dice D(2,2 d(1 - D( c +
(19)
1-k=1 jik
Finally, the auxiliary quadruplet adversarial loss function (Caqal) is ultimately formulated as:
LAQAL [(c=2, C22, c=1) + L(c2, Cu2, c.1).
(20)
The LAQAL forces the distance between modalities increase, preventing them from easily clustering together, thereby disrupting the overall ranking results.
3.4 Objective Function
Besides the auxiliary quadruple adversarial loss LAQAL, also have the identity loss Lid- The training process of VIReID is considered an image classification problem, where each identity is distinct class. During the testing phase, the output from the pooling layer or embedding layer is used as the feature extractor. Given an input image with label yı, the probability of I; being recognized class y:is encoded using the softmax function and denoted p(,,(2;). The identity loss is then computed by the cross-entropy
Lu Σ 1og(p(ug(x4)),
(21)
where N represents the number of training samples within each batch.
4 Experiments
4.1
Implementation Details
The experiments are conducted on an NVIDIA GeForce 3090 GPU with Pytorch. We chose powerful baseline model AGW (38, which is ResNet-50 pretrained on ImageNet, as the backbone network. During training, we randomly sampled 16 identities, each with images, to form mini- batch of size 64. Pedestrian images are resized to 288 x 144, Data augmentation included random

---

Table 1: Comparison of CMC (%) and mAP (%) with the state-of-the-arr methods on SYSU-MM01 and RegDB datasets. Our results show the best results in terms Rank-1 accuracy and mAP.
SYSL-MM0I
RegDB
methods
All-search
Indoor scarch
Visible to Thermal
Themmal to Visible
Rank-1
Rank-10
mAP
mINP
Rank-1
mAP
mINP
Rank-1
mAP
mINP
Before Allack
47.50
81,39
47.65
35,30
51.17
91.11
62.97
59.23
70.05
66.37
70.19
65.90
FGSM [0]
36.02
59.22
31.80
19.72
33.25
69.17
46.31
30.36
45.87
44,39
36.59
46.15
46.12
43.24
PGD 31
26.60
46.65
25.67
16.90
36,89
75.19
43.50
27.20
29,37
26.87
17.83
29.05
29.12
17.14
21.72
39,80
21.31
20.39
25.77
51.21
30.91
29.96
19.85
17.37
9.49
16.57
18.23
10.81
CAP (1)
17.59
56.81
25.35
20.89
35,34
47.86
30.75
19.13
29.51
24.42
16.64
19.61
18.95
12,67
LTA (2
15,47
30.39
17.71
13.44
21.68
38.56
26.61
20.59
11.60
10.86
6.07
14.56
13.11
7.75
DMR [33
9.20
24.43
10.21
4.71
13.62
27.14
14.94
6.27
4.97
4.80
2.12
6.09
5.26
3.54
0.79
5.68
0.60
Table 2: Comparison of CMC (%) and mAP (%) of different VIReID systems before and after attack. Bold numbers indicate values after attack.
RegUii
muthods
All search
Indoce search
Visible to Thennal
Thermal to Visble
Rank
Kank: 10
MAP
Rank- 1
Rank 10
MAP
Rank
Rank 10
MAP
Kank-
Rank 10
MAP
HCLoss L83
$6
91 37V1037
54.95/911
59
9207718.85
64.917.17
86.02/2.7
96.3/10.75
74,002.22
87,201.02
97,0044.6
78.302.07
90799
76 15245
85:01.04
$4.9999
MMS 41
70601.39
96.24..55
66.93.81
76.34.36
07.3/27.11
79.68816
91,60449
97,70097
84.11.71
67,53,50
96,014.00
60.50072
74.701.71
11.1/318
07.4/12.88
96.616.
horizontal flipping and random erasing with probability of 0.5. We optimize using the stochastic gradient descent (SGD) optimizer, with a weight decay set 0.0005 and a momentum parameter set to 0.9. The initial learning rate for both datasets was set to 0.1, and it was decayed by factor of 0.1 at the 20th and 50th epochs, respectively. Finally, the margin in the auxiliary quadruplet adversarial loss function is set to 0.2. We adopt warm-up learning rate scheme, with total of 60 training epochs.
4.2 Datasets
SYSU-MMO1 is mross-modality pedestrian re- re-identification dataset proposed in 2017. There are 287,628 visible images and 15,792 infrared images total. Cameras 1 and 2 are installed in well-lit environments, cameras 3 and 6 operate under infrared conditions, and cameras 4 and 5 are placed in outdoor scenes. The dataset comprises two testing modes: all-search and indoor-search. The all-search mode is more challenging because the gallery includes images from all cameras.
The RegDB 43 dataset comprises 412 individuals, with each person having 20 images, including 10 visible images and 10 infrared images. Among the 412 individuals, there are 254 females and 158 males, with 156 individuals captured in frontal views and 256 individuals captured in rear views. During the testing phase, RegDB offers two modes: visible to infrared and infrared to visible. In the visible infrared mode, visible images serve as query images, while infrared images are used as gallery images. The infrared to visible mode operates in the opposite manner.
4.3 Comparison with State-of-the-Art Methods
We select six different methods to evaluate the security of the AGW model. Among them, FGSM and PGD are typical gradient-based methods for generating adversarial samples, while UAP generates universal perturbations that are independent of the image. Additionally, we chose SMA, DMR, and ITA, three pedestrian re-identification attack methods which primarily disrupt ranking and generate adversarial samples through local color iteration. The results, as shown in Table] indicate a sharp decline in all evaluation metrics after attacking AGW across two datasets and two different tests. In the RegDB dataset, under two testing modes, our mAP decreased dramatically from 66.37% and 65.90% to 0.85% and 0.89%, nearly approaching O. This demonstrates that our method is more suitable for testing robustness against VIReID tasks.
To demonstrate the universality of our method, we select three approaches to improve the recognition accuracy of VIReID systems: HCLoss enhances intra-class cross-modality similarity through a heterogeneous center loss function, CAJ improves recognition accuracy based data augmentation techniques, and MMN and DEEN utilize different network designs better extract effective shared features. These methods are commonly used to improve VIReID accuracy at the feature level. As
8

---

Table 3: Analysis about the influence of each component in terms of Rank- (%) and mAP (%).
SYSU-MM01
RegDB
Noise
FSAM
SAM
LAQAL
Rank-1
MAP
Rank-1
mAP
46.73
45.78
82.24
76.52
16.67
18.28
33.00
30.60
12.75
12.17
15.74
12.91
11.04
13.19
20.73
19.81
/
14.62
15.52
15.86
15.53
5.10
6.35
7.58
7.51
1.07
3.27
0.58
1.35
1.74
3.81
6.70
5.77
0.79
2.81
0,49
0.85
Table 4: Performance comparison of different feature extraction methods in terms of CMC (%) and mAP (%) on RegDB. (Setting: Baseline SFM LAQAL.)
SYSU-MM01
RegDB
Settings FSAM
Rank-1
Rank-10
mAP
mlNP
Rank-1
Rank-10
mAP
mINP
block0
1.05
10.60
3.15
2.04
0.49
4.17
1.07
1.27
block1
1.05
10.52
4.29
5.33
0.53
1.69
1.68
2.41
block2
1.05
10.07
3.56
2.25
0.54
2.94
1.23
1.11
block3
1.34
10.41
4.07
4.72
0.49
1.46
1.67
2.41
block4
1.01
10.12
2.91
1.68
0.49
4.71
0.93
0.67
block0-block4(ouss)
0.79
9.83
2.81
1.69
0.48
4.64
0.85
0.57
shown Table2 our experimental results show that after applying our attack method, the mAP of all systems dramatically decreased across different datasets and test modes (for example, the Rank-1 of DEEN drops from 74.70% to 1.71%, and mAP falls from 71.800 71.80% to 3.55% in all-search mode), indicating that these systems cannot withstand our attacks. This demonstrates that method can effectively test the robustness various VIReID systems.
4.4 Ablation Study
As shown in Table3] the effectiveness of different modules is validated on two datasets. Using the SYSU-MM01 dataset as an example, after adding noise to generate adversarial samples, the mAP decreases from 45. 75% to 18.28%. Subsequently, the three modules are individually tested on this basis, and all show decrease in accuracy, with the FSAM module being the most effective, reducing the mAP to 12. 17% Additionally, since both FSAM and SAM are designed to enhance adversarial feature representation capabilities, the combined effect of these two modules tested, further reducing the mAP to 6.35%. The experiments demonstrate that all three proposed modules are highly effective. Although targeting different aspects, their combined usage enhances the effectivvnees of method.
Determining which stage of ResNet-50 should have the FSAM module inserted. In this experiment, we use Re as the backbone, which has five stages: block 0 to block 4. We study the impact on model performance by inserting the FSAM module after different stages of ResNet-50. This experiment controls only the insertion position of FSAM as the variable, keeping all other factors constant. As shown in Table4] inserting FSAM after block and block 4 results lower accuracy, indicating better performance. This suggests that in the shallow layers of the network, image processing captures more effective information from the initial frequency domain features, while in the deeper layers of the network, the feature representation capability is further enhanced, and spatial information becomes more important. Based on these results, we integrate the proposed FSAM module after block 0 and block 4 of the ResNet-50 model.
As shown in Table 5 we compared our frequency domain attention module with channel attention mechanisms like SE-Net 44 and ECA-Net[45). The results reveal that while SE-Net and ECA- Net excel certain areas, our frequency domain attention module is highly competitive across various metrics. It achieves the best Rank-1 and mAP scores on both datasets. Channel attention mechanisms often focus on global features, which may overlook local details and spatial relationships in moos- VIReID tasks, leading to potential information loss and reduced performance.

---

Table 5: Performance of the FSAM module replacing different attention mechanisms on CMC(%)
and mAP(%).
SYSU-MMOI
RegDB
Attention
Rank-1
Rank-10
mAP
mINP
Rank-1
Rank-10 10
mAP
mINP
SENet
1.66
12.15
3.30
1.76
0.53
5.00
1.15
0.64
ECA-Net
160
11.65
3.24
1.65
0.68
3.64
1.20
0.68
Ours)
0.79
9.83
2.81
1.69
0,48
4.64
0.85
0.57
4.5 Visualization Analysis
We compare the t-SNE visualization results on the baseline and the proposed method. To ensure fairness, we randomly select several images of identities from all cameras. For each individual, 20 visible images and 20 infrared images are randomly chosen. As shown in Figure after the attack, the clustering of all pedestrians became more dispersed, with the visible and infrared modalities each forming their own clusters, and the relative distance between them increasing. The visualization experiments validated the effectiveness of our attack method, as the attack further exaggerated the gap between the two modalities, effectively suppressing the output performance. These results demonstrate that our method is highly effective for security testing in VIReID.
Baseline
Attacked
Figure 4: t-SNE visualization comparison before and after the attack. Different colors represent different identities. The 'asttrisks' and 'rectangles' denote the infrared person features and visible person features, respectively. After the attack, the features are dispersed, enlarging the distance between modalities.
5 Conclusion
In this paper, our objective to conduct security validation VIReID systems and propose a novel attack method suitable for cross-modality tasks. We introduce the frequency-spatial attention module, which are used at two stages of feature extraction, focusing on spatial information in both frequency and source domains enhance the representation capability of adversarial features and strengthen the effectiveness of adversarial samples. Additionally, we propose an auxiliary quadruple adversarial loss function considering the modalities differences involved in VIReID) tasks to interfere with the ranking of system outputs, completing the robustness test of the current VIReID system. Extensive experiments not only deepens the understanding of the security of eross-modality RelD systems but also provides a new direction for the development of VIReID and emphasizes the importance of ensuring their reliability and protection in practical applications.
Acknowledgement
This work was supported in part by the National Natural Science Foundation of China under Grant 62372348, Grant 62441601, Grant 62176195, Grant 62176198. Grant U22A2096. Grant U21A20514; in part by the Key Research and Development Program of Shaanxi under Grant 3024GX-ZDCYX 02-10; in part by Shaanxi Outstanding Youth Science Fund Project under Grant 2023-JC-JQ-53; in part by the Shaanxi Province Core Technology Research and Development Project under Grant part by the Fundamental Research Funds for the Central Universities under Grant QTZX24080 and Grant QTZX23042.
20240Y2-GJHX-I1
10

---

References
[1] Ancong Wu, Wei-Shi Zheng, Hlong-Xing Yu, Shaogang Gong, and Jianhuang Lai. Rgb-infrared cross- modality person re-identification. In ICCV, pages 5380-5389, 2017.
Qiong Wu, Pingyang Dai, Jie Chen, Chia-Wen Lin, Yongjian Wu, Feiyue Huang, Bineng Zhong, and Rongrong Ji. Discover cross-modality nuances for visible-infrared person re-identification. CVPR, pages 4330 4339, 2021.
[3] Jiangming Shi, Yachao Zhang. Xiangbo Yin, Yuan Xie, Zhizhong Zhang, Jianping Fan, Zhongchao Shi, and Yanyun Qu. Dual pseudo-labels interactive self-training for semi-supervised visible-infrared person re-identification. In ICCV, pages 11218-1 1218-11228, 2023.
[4] Yinpeng Dong, Hang Su, Baoyuan Wu, Zhifeng Ii, Wei Liu, Tong Zhang, and Jun Zhu. Efficient decision-based black-box adversarial attacks on face recognition. In CVPR, pages 7714-7722, 2019.
[5] Yaoyao Zhong and Weihong Deng. Towards transferable adversarial attack against deep face recognition. IEEE Transactions on Information Forensics and Security. 16: 1452 1466, 2020.
[6] Dong Yin, Raphael Gontijo I Lopes, Jon Shlens, Ekin Dogus Cubuk, and Justin Gilmer. A fourier perspective model robustness computer vision. In NeurlPS, 2019.
[7] Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, lan Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv: 1312.6199, 2013.
[8] Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In TCSP. pages 39-57, 2017.
[9] Andrew Ilyas, Shihani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. Adversarial examples are not hugs, they are features. In NeurlPS, 2019.
[10] Minghui Chen, Zhiqiang Wang, and Feng Zheng. Benchmarks for corruption invariant person re- identification. In NeurlPS, 2021.
[11] Seyed-Mohsen Moosavi-Dezfool: Alhussein Fawzi, Omar Fawzi, and Pascal Frossard. Universal adversar- ial perturbations. In CVPR, pages 1765-1773, 2017.
[12] Cuiqun Chen, Mang Ye, Meibin Qi, Jingjing Wu, Jianguo Jiang. and Chia-Wen Lin. Structure-aware positional transformer for visible-infrared person re-identification. JEEE Transactions on Image Processing, 31:2352-283 226422222222222 2022.
[13] Yunhao Du, Cheng Lei, Zhicheng Zhao, Yuan Dong, and Fei Su. Video-based visible-infrared person re-identification with auxiliary samples. IEEE Transactions Information Forensics and Security, 19:1313-1325, 2023.
[14] Hao Yu, Xu Cheng. Wei Peng, Weihao Liu, and Guoying Zhao. Modality unifying network for visible- infrared person re-identification. In ICCV, pages 11 11185-11195, 2023.
[15] Yukang Zhang, Yang Lu, Yan Yan, Hanzi Wang, and Xuelong Li. Frequency domain nuances mining for visible-infrared person re-identification. arXiv preprint arXiv:2401 022162, 2024.
[16] Mang Ye, Xiangyuan Lan, Jiawei Li, and Pong Yuen. Hicrarchical discriminative learning for visible thermal person re-identification. AAAI. 2018
[17] Mang Yc, Jianbing Shen, David Crandall, Ling Shao, and Jiebo Luo. Dynamic dual-attentive aggregation learning for inffare person re- re-identification. In ECCV, pages 229 22992247220 2020.
[18] Yuanxin Zhu, Zhao Yang, Wang. Sai Zhao, Xiao Hu, and Dapeng Tao. Hetero-center loss for cross- modality person re-identification. Neurocomputing, 386:97-109, 2020.
[19] Haijun Liu, Xiaoheng Tan, and Xichuan Zhou. Parameter sharing exploration and hetero-center triplet loss for visible-thermal person re-identification. IEEE Transactions on Multimedia, 13:451444255, 2020.
[20] Mang Ye, Zheng Wang, Xiangyuan Lan, and Pong C Yuen. Visible thermal person re-identification via dual-constrained top-ranking. In IJCAI, page 2, 2018.
[21] Mang Ye, Xiangyuan Lan, Zheng Wang, and Pong C Yuen. Bi-directional for visible thermal person re- r--ientiifiation. IEEE Transactions on Information Forensics and Security. 15:407-419 2019.
top lop-ranking
11

---

[22] Chaoyou Fu, Yibo Hu, Xiang Wu, Hailin Shi. Tao Mei, and Ran He. Cm-nas: Cross-modality neural architecture search for visible-infrared person re-identification. In ICCV, pages 11823-11832, 2021.
[23] Ychansen Chen, Lin Wan, Zhihang Li, Qianyan Jing, and Zongyuan Sun. Neural feature search for rgb-infrared person re-identification. CVPR, pages 587-597, 2021.
[24] Yongguo Ling, Zhun Zhong, Zhiming Luo, Paolo Rota, Shaozi Li, and Nicu Sebe. Class-aware modality mix and center-guided metric learning for visible-thermal person re-identification. In ACM MM, pages 889-897, 2020.
[25] Yulin Li, Tianzhu Zhang, and Yongdong Zhang. Frequency domain modality-invariinn feature learning for visible-infrared person re-identification. arXiv preprint 0XXi:24001018833 2024,
[26] Jie Li, Rongrong Ji, Hong Liu, Xiaopeng Hong, Yue Gao, and Qi Tian. Universal perturbation attack against image retrieval. In ICCV, pages 4899-4908, 2019.
[27] Mo Zhou, Zhenxing Niu, Le Wang, Qilin Zhang, and Gang Hua. Adversarial ranking attack and defense. In ECCV, pages 781-799, 2020.
[28] Zhedong Zheng, Liang Zheng, Yi Yang, and Fei Wu. Query attack via opposite-direction feature: Towards robust image retrieval. arXiv preprint arXiv: 1exXiv:800// 0889990668.. 2018.
[29] Fengxiang Yang. Zhun Zhong, Hong Liu, Zheng Wang, Zhiming Luo, Shaozi Li, Nicu Sehe, and Shin' ichi Satoh. Learning attack real-world models for person re-identification via virtual-guided meta-learning. In AAAI. pages 3128-3135, 2021.
[30] Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and hamesssing adversarial examples. arXiv preprint arXiv: 1412.5572 2014.
[31] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06640 06083, 2017.
[32] Yunpeng Gong. Liqing Huang, and Lifei Chen. Person re-identification method based on color attack and joint defence. In CVPR, pages 4313-4322, 2022.
[33] Hongjun Wang, Guangrun Wang, Ya Li, Dongyu Zhang, and Liang Lin. Transfcrable, controllable, and inconspicuous adversarial attacks on person re-identification with deep mis-ranking. In CVPR, pages 342-351, 2020.
[34] Uche M Osahor and Nasser M Nasrabadi. Deep adversarial attack on target detection systems. Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications, pages 620-628, 2019.
[35] Xiaopei Zhu, Xiao Li, Jianmin Li, Zheyao Wang, and Xiaolin Hu, Fooling thermal infrared pedestrian detectors in real world using small bulbs. In AAAI, pages 3616-3624, 2021.
[36] Xingxing Wei, Jie Yu, and Yao Huang. Physically adversarial infrared patches with learnable shapes and locations. CVPR, pages 12334-12342, 2023.
[37] Xingxing Wei, Yao Huang, Yitong Sun, and Jie Yu. Unified adversarial patch for cross-modal attacks in the physical world. In ICCV, pages 4445 4454, 2023.
[38] Mang Ye, Jianhing Shen, Gaojie Lin, Tao Xiang, Ling Shao, and Steven CH Hoi. Deep learning for person re-identification: A survey and outlook. IEEF Transactions on Pattern Analysis and Machine Intelligence, 44:2872-2893 2021.
[39] Quentin Bouniot, Romaric Audigier, and Angelique Loesch. Vulnerability of person re-identification models to metric adversarial attacks. In CVPR, pages 794-795, 2020.
[40] Mang Ye, Weijian Ruan, Bo Du, and Mike Zheng Shou. Channel augmented joint learning for visible- infrared recognition. In ICCV, pages 13567-13576, 2021.
[41] Yukang Zhang, Yan Yan, Yang Lu, and Hanzi Wang. Towards a unified middle modality learning for visible-infrared person re-identification. In ACM MM, pages 788-796, 2021.
[42] Yukang Zhang and Hanzi Wang. Diverse embedding expansion network and low-light cross- modalityy benchmark for visible-infrared person re-identification. In CVPR, pages 2153-2162, 2023.
[43] Dat Tien Nguyen, Hyung Gil Hong, Ki Wan Kim, and Kang Ryoung Park. Person recognition system based on combination of body images from visible light and thermal cameras. Sensors, 17:605, 2017.
[44] Jie Hu, Li Shen, and Gang Sun. Squeeze-and-excitaee
networks. In CVPR, pages 7132-7141, 2018.
[45] Qilong Wang, Banggu Wu, Pengfei Zhu, Peihua Li, Wangmeng Zuo, and Qinghua Hu. Eca-net: Efficient channel attention for deep convolutional neural networks. In CVPR, pages 11534 11542, 2020.
12

---

7 NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The abstract succinetly summarizes the core contributions and scope of the paper, while the introduction elaborates the research background, problem definition, proposed methods, and main results obtained.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [No]
Justification: Our approach was tested on only a few datasets and systems, demonstrating the feasibility of the theory without exploring its application in real-world scenarios.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full of assumptions and complete (and correct) proof?
13

---

Answer: [Yes]
Justification: Detailed theorems and formulas are provided Section 3, with the theorems relied upon being properly cited.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We contribute a novel architecture, and the entire process is detailed in Section 3 through images and formulas.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
14

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [No]
Justification:
We will consider making the code details publicly available in the future.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
//nips //ipssccc/pill wwtpcccpppppppppeeeeeeeeeeeeeeee http:///e/.ieee/eteSuuuiiiissss Coodeooomisssssseeeeeeee wwwssniiicciyyiicc for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification:
We provide a detailed explanation in Section
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: Our experimental results do not include error bars, confidence intervals, or tests for statistical significance.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for
example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
15

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We present the relevant computational information in Section 4.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics httpp:////eaerps.ccc///////////a
Answer: [Yes]
Justification: We have reviewed the relevant requirements and ensured compliance with ethical guidelines.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: While our method can applied for robustness testing models, it also implies that certain groups might use this method to attack systems, resulting in potentially destructive impacts. These are considerations that need to be taken into account in practical deployments.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
16

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
Justification:
We use publicly available datasets and do not involve scraping data from the
internet.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We ensure that all assets used are properly credited and adhere to the relevant licenses and terms of use.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
17

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
If assets are released, the license, copyright information, and terms use in the thcode. com/ com/dataseta has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: This paper does not introduce any assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [No]
Justification: The pedestrian dataset used in this paper publicly available, and it is stated by the publisher that each pedestrian captured in the dataset has signed privacy release allowing the use of the images for scientific research and display in research papers.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [No]
Justification: The pedestrian dataset used in this paper is publicly available, and it is stated by the publisher that each pedestrian captured the dataset has signed a privacy release allowing the use of the images for scientific research and display in research papers.
Guidelines:
18

---

The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
19


Reviews:
[{"review_id": "B8tUJkifl3", "reviewer": "Reviewer_Jhhd", "summary": "This paper addresses the security of visible-infrared person re-identification systems by introducing a method for feature-level adversarial attacks. The proposed approach integrates universal adversarial perturbations and a frequency-spatial attention module to disrupt the output ranking of VIReID systems. The auxiliary quadruple adversarial loss is designed to amplify modality differences, enhancing the distinction between visible and infrared features. Extensive experiments on the SYSU-MM01 and RegDB benchmarks validate the effectiveness of this method in compromising VIReID systems' rankings.", "questions": "Can the authors provide more insight into the computational complexity of the proposed method?", "limitations": "No limitations or negative impacts have been identified in this paper.", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. This paper is an early work in the field of the security of visible-infrared person re-identification systems.\n2. The authors have made some efforts in the experimental section to validate the effectiveness of the proposed method.", "weaknesses": "1. The paper could benefit from a more in-depth analysis of the failure cases and limitations of the proposed method. For instance, under what conditions does the attack fail, and why?\n2. The paper primarily focuses on the effectiveness of the attack in terms of ranking disruption. Additional metrics, such as computational efficiency and impact on overall system performance, could provide a more comprehensive evaluation.\n3. Minor issue: Equation 15 appears to have an error.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "w1: Limitations of the proposed method**\n\nAlthough the proposed method is effective in many scenarios, it has certain limitations and conditions under which it may fail. These include:\n\n1. **Extreme Imaging Conditions**: Our method relies on the alignment of adversarial features across different imaging modalities, such as visible and infrared images. However, significant variations in imaging conditions can impact the effectiveness of the feature alignment. For example, extreme lighting conditions or significant occlusions may degrade the performance of the adversarial attacks, leading to a failure in disrupting the identification process.\n\n2. **Feature Consistency**: The proposed frequency-spatial attention module is designed to enhance the consistency of features between visible and infrared images. Nevertheless, if the inherent feature differences between the two modalities are too large, the module may not effectively bridge the gap, resulting in reduced attack efficacy.\n\n3. **Generalizability**: Our approach has been tested primarily on specific datasets and systems, such as SYSU-MM01 and RegDB. While these datasets provide a controlled environment for evaluation, the method's performance in real-world scenarios with diverse and unseen data remains to be fully explored. Factors like variations in camera quality, resolution, and environmental conditions can influence the generalizability of the results.\n\nBy acknowledging these limitations, we aim to provide a comprehensive understanding of the scope and applicability of our proposed method, thereby enhancing its utility and reliability in practical deployments.\n\n**w2 & Q: Analysis of computational complexity**\n\nAccording to Table 1, we analyzed the computational complexity of several methods.\n\n**Flops (Floating Point Operations)**: This metric measures the number of floating point operations required for a single forward pass of the model, reflecting its computational complexity. The Flops values of all methods in the table are quite similar, ranging from approximately 331.45G to 331.88G, indicating that these methods have comparable computational complexity.\n\n**Params (Number of Parameters)**: This metric measures the total number of trainable parameters in the model, reflecting its scale and complexity. All methods have a parameter count of 23.55M, suggesting that these methods have the same parameter scale, likely due to using the same base model or network architecture.\n\n**Conclusion**: Despite the similar computational complexity (Flops) and parameter scale (Params) across all methods, there are significant differences in their performance metrics (Rank-1, mAP, mINP). Compared to other methods, our approach has similar computational complexity and parameter count, but performs significantly lower in performance metrics. We did not drastically increase the computational complexity; instead, we achieved superior performance by balancing metrics and computational complexity more effectively.\n\n| Methods       | SYSU-MM01  |          |          |          | RegDB              |          |          | Flops       | Params |\n| ------------- | ---------- | -------- | -------- | -------- | ------------------ | -------- | -------- | ----------- | ------ |\n|               | All-search |          |          |          | Visible to Thermal |          |          |             |        |\n|               | Rank-1     | Rank-10  | mAP      | mINP     | Rank-1             | Rank-10  | mAP      |             |        |\n| Before Attack | 47.50      | 84.39    | 47.65    | 35.3     | 70.05              | 66.37    | -        | **331.45G** | 23.55M |\n| FGSM          | 36.02      | 59.22    | 31.8     | 19.72    | 45.87              | 44.39    | 36.59    | 331.75G     | 23.55M |\n| UAP           | 17.59      | 56.81    | 25.35    | 20.89    | 29.51              | 24.42    | 16.64    | 331.60G     | 23.55M |\n| Ours          | **0.79**   | **9.83** | **2.81** | **1.69** | **0.49**           | **0.85** | **0.57** | 331.88G     | 23.55M |\n\nWe have carefully evaluated and optimized the computational complexity of our proposed method to ensure a balance between performance and efficiency. As indicated in Table 1, we have adopted efficient techniques to handle cross-modality data (visible and infrared), reducing the additional computational overhead typically associated with processing different data types. We utilized an optimized network architecture that balances depth and width, ensuring efficient computation without sacrificing performance.\n\n**w3: Minor issue: Equation 15 appears to have an error**\n\n The corrected formula is:\n$$\n\\[\n\\mathcal{L}(x_{v2}, x_{i2}, x_{v1}) = \\sum_{ \\substack{j,k=1 \\\\ j \\ne k} } \\left[ D(x_{v2}^{j}, x_{v1}^{j}) - D(x_{i2}^{j}, x_{v1}^{j}) - D(x_{v2}^{j}, x_{v2}^{k}) + \\alpha \\right]_{+}\n\\]\n$$", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "nNNfdC8cAX", "reviewer": "Reviewer_78wy", "summary": "This paper aims to explore the security of VIReID and introduces a Universal Adversarial Perturbations to simulate common disturbances in real-world environments. Additionally, a Frequency-Spatial Attention Module is proposed to integrate frequency information extraction and spatial focusing mechanisms. An Auxiliary Quadruple Adversarial Loss is proposed to amplify the differences between modalities, thereby improving the distinction and recognition of features between visible and infrared images. Extensive experiments on two VIReID benchmarks (i.e., SYSU-MM01, RegDB) and different systems validate the effectiveness of the proposed method.", "questions": "Please check the weakness.", "limitations": "The authors have adequately addressed the limitations of the proposed method.", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1) This paper aims to explore the security of VIReID and introduces a Universal Adversarial Perturbations to simulate common disturbances in real-world environments. \n\n2) Extensive experiments on two VIReID benchmarks (i.e., SYSU-MM01, RegDB) and different systems validate the effectiveness of the proposed method.\n\n3) The paper is well-written and easy to follow.", "weaknesses": "1) Since the author proposed a frequency domain attention module, the frequency domain and attention need to be introduced in the related work section.\n\n2) What is the motivation of the proposed frequency domain attention module for the VIReID task? \n\n3) The proposed frequency domain attention module uses a spatial attention module to generate spatial attention maps using the spatial relationships within the features. I'm quite curious about the performance if other attention mechanisms (such as SE-Net, Channel) are used. \nFurthermore, could the authors provide the performance of the proposed frequency domain attention module on the VIReID method, thereby proving its effectiveness?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**w1: The introduction of  frequency domain and attention mechanism should be added in the related work.**\n\n**Frequency domain**： In recent years, frequency domain information processing has gained significant attention in deep learning, proving effectiveness for tasks like face recognition and person re-identification. This method analyzes high and low-frequency components to extract useful information by enhancing or suppressing them. For example, PHA[1] boosts high-frequency components for better pedestrian representation. Amplitude and phase components in the frequency domain can also be used to focus on style and color versus spatial information. For instance, SFMNet[2] uses Fourier transforms for face super-resolution, capturing global image information, while Zhang et al.[3] extract modality-invariant features by leveraging amplitude and phase components.\nAttention Mechanism: Attention mechanisms allow convolutional neural networks to focus on important features while ignoring irrelevant ones. These mechanisms can be categorized into spatial, channel, and other domains. SENET[4] focuses on channel-specific features by compressing spatial dimensions and learning in the channel dimension. The Spatial Transformer Network (STN)[5] captures important regional features by transforming deformed data. CBAM[6] combines spatial and channel attention sequentially to refine image features. For visible-infrared person re-identification, we designed an attention mechanism tailored for the spatial domain.\n\n \n\n[1] PHA: Patch-wise High-frequency Augmentation for Transformer-based Person Re-identification. CVPR 2023\n\n[2] Spatial-Frequency Mutual Learning for Face Super-Resolution. CVPR 2023\n\n[3] Frequency Domain Nuances Mining for Visible-Infrared Person Re-identification VIReID. Arxiv 2024\n\n[4] Squeeze-and-Excitation Networks. CVPR 2018\n\n[5] Spatial Transformer Networks. NIPS 2015\n\n[6] Convolutional Block Attention Module. CVPR 2018\n\n**w2: The motivation of FSAM**\n\nTo generate more effective adversarial features, we propose the Frequency-Spatial Attention Module based on the following:\n\n**Frequency Domain Feature Decomposition**: Visible and infrared images differ significantly in the frequency domain—visible images offer rich texture information, while infrared images contain mostly pixel data. Both share spatial consistency. We use Fast Fourier Transform (FFT) to decompose features into amplitude and phase components, capturing texture and spatial information, respectively. This approach allows targeted processing of each component, optimizing feature extraction.\n\n**Application of Spatial Attention**: Attention is applied solely to the spatial component. Since the phase component is closely tied to spatial information, the spatial attention module can enhance or suppress specific regions of the feature map. This focus improves the model's ability to highlight important areas for better adversarial feature quality and performance.\n\n**Optimization of Cross-Modal Features**:  Emphasizing spatial information helps the model integrate features from different modalities, enhancing overall comprehension and generalization.\n\nIn summary, the Frequency-Spatial Attention Module improves adversarial feature quality and recognition accuracy by combining spatial attention with detailed frequency domain analysis.\n\n**w3: Replace it with channel attention**\n\nAs shown in Table 2, we compared our frequency domain attention module with channel attention mechanisms like SE-Net[4] and ECA-Net[7]. The results reveal that while SE-Net and ECA-Net excel in certain areas, our frequency domain attention module is highly competitive across various metrics. It achieves the best Rank-1 and mAP scores on both datasets. Channel attention mechanisms often focus on global features, which may overlook local details and spatial relationships in cross-modal VIReID tasks, leading to potential information loss and reduced performance.\n\n|           | SYSU-MM01 |          |          |      | RegDB    |          |          |          |\n| --------- | --------- | -------- | -------- | ---- | -------- | -------- | -------- | -------- |\n| Attention | Rank-1    | Rank-10  | mAP      | mINP | Rank-1   | Rank-10  | mAP      | mINP     |\n| SENet     | 1.66      | 12.15    | 3.30     | 1.76 | 0.53     | 5.00     | 1.15     | 0.64     |\n| ECA-Net   | 1.60      | 11.65    | 3.24     | 1.65 | 0.68     | **3.64** | 1.20     | 0.68     |\n| Ours      | **0.79**  | **9.83** | **2.81** | 1.69 | **0.49** | 4.64     | **0.85** | **0.57** |\n\n[7] ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks. CVPR 2020\n\n**w4: The effectiveness of FSAM on  VIReID** \n\nThe table3 compares the performance of three VIReID systems before and after the addition of FSAM (Frequency Domain Attention Module). All systems show significant improvements in Rank-1, Rank-10, and mAP after the addition of FSAM, indicating that FSAM plays a positive role in enhancing the performance of VIReID systems. Furthermore, the consistent effects across different datasets and search modes demonstrate its broad applicability and robustness.\n\n| Methods     | SYSU-MM01 |           |           | RegDB     |           |           |\n| ----------- | --------- | --------- | --------- | --------- | --------- | --------- |\n|             | Rank-1    | Rank-10   | mAP       | Rank-1    | Rank-10   | mAP       |\n| AGW         | 47.50     | 84.39     | 47.65     | 70.05     | 86.21     | 66.37     |\n| AGW + FSAM  | **49.41** | **87.64** | **49.39** | **75.05** | **89.17** | **68.28** |\n| CAJ         | 69.88     | 95.71     | 66.89     | 85.00     | 95.50     | 84.60     |\n| CAJ + FSAM  | **70.20** | **96.36** | **67.99** | **86.77** | **95.77** | **85.01** |\n| DEEN        | 74.70     | 97.60     | 71.80     | 91.10     | 97.80     | 85.10     |\n| DEEN + FSAM | **75.20** | **97.70** | **72.27** | **92.44** | **99.09** | **86.45** |", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ypAXYtIgS1", "reviewer": "Reviewer_ksiy", "summary": "There is currently a lack of research on the security of VIReID systems. This paper proposes to explore the vulnerabilities of VIReID systems and prevent potential serious losses due to insecurity. To obtain adversarial features, this paper introduces Universal Adversarial Perturbations (UAP) to simulate common disturbances in real-world environments. Additionally, the authors employ a Frequency-Spatial Attention Module (FSAM), integrating frequency information extraction and spatial focusing mechanisms, and further emphasize important regional features from different domains on the shared features. Extensive experiments on two VIReID benchmarks (i.e., SYSU-MM01, RegDB) and different systems validate the effectiveness of the proposed method.", "questions": "How could a real-world application employ this method for an attack in a visible-infrared surveillance system? The motivation for the methodology of this paper requires further elaboration", "limitations": "Authors need to consider more methods of attack than just adding noise.", "rating": 6, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "Experimental results on the VI-Person ReID task show the proposed method works well.", "weaknesses": "1. Adversarial attack in VI-ReID is a meaningful research topic. However, merely utilizing the noise-added image as an adversarial sample is singular and impractical. How could a real-world application employ this method for an attack in a visible-infrared surveillance system? Numerous other methods for generating adversarial samples are not discussed in the paper, thereby significantly diminishing its value.\n2. This paper does not present the image with the added noise. If the image loses its original information after noise is introduced, the recognition results will be poor even without using the method of this paper.\n3. FSAM is not a new thing, which has been proposed in many works [1] and is not innovative.\n[1] Li Y, Zhang T, Zhang Y. Frequency Domain Modality-invariant Feature Learning for Visible-infrared Person Re-Identification[J]. arXiv preprint arXiv:2401.01839, 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**w1 & limitation: Discussion of the definition of adversarial attacks and what they mean, and how they can be applied in the real world**\n\nAdversarial attacks induce misclassification in classifiers by introducing subtle perturbations to inputs. In 2014, Goodfellow et al. [1] demonstrated this using a panda image. Since then, research has focused on identifying model vulnerabilities and improving model robustness. Studies reveal that deep neural networks are susceptible to adversarial examples, where minor input perturbations cause incorrect predictions with high confidence. As deep learning applications expand, the security issues they expose garner more attention.\n\nTechnologies like face recognition and person re-identification (ReID) have significant potential in security fields such as criminal investigation, person tracking, and behavior analysis. These technologies support the safe and stable operation of public society. However, their security and reliability are questioned in adversarial environments, limiting their specialized applications. Adversarial attacks offer a new perspective on system security, showing how adversarial examples can evade recognition or impersonate others. For human observers, these examples are often indistinguishable from legitimate ones but cause deep models to err. Evaluating recognition systems' robustness with adversarial attacks identifies system vulnerabilities, encouraging improvements in machine learning model robustness.\n\nSimilarly, visible-infrared person re-identification (VIReID) is increasingly used in security systems, necessitating an exploration of its security. Existing attack methods focus on visible image features, neglecting other modalities and cross-modal data distribution variations, potentially reducing their effectiveness in cross-modal image retrieval. This study examines VIReID model security and proposes a universal perturbation attack designed for VIReID.\n\n[1] Explaining and harnessing adversarial examples. Arxiv  2014\n\n[2] Efficient Decision-based Black-box Adversarial Attacks on Face Recognition. CVPR 2019\n\n[3] Transferable, Controllable, and Inconspicuous Adversarial Attacks on Person Re-identification With Deep Mis-Ranking. CVPR 2020\n\n**w2: Visualization of antagonistic samples**\n\nAs shown in the figure1, we can observe the before-and-after comparison of the visualization with the addition of UAP (Universal Adversarial Perturbation). Adversarial attacks involve applying perturbations that are imperceptible to the human eye, causing the model to produce incorrect outputs. The visualization results show no visible difference between adversarial samples and original samples. According to Table 5, although the images appear identical to the human eye after adding noise, they lose some of the original information, leading to decreased recognition performance. \n\nHowever, our goal is to simulate real-world interference, hoping to enhance the representation capability of adversarial features to handle more complex and diverse scenarios. This approach aims to provide a stronger evaluation capability when assessing the security of VIReID systems, thereby enhancing the social value of our research.\n\n**w3: FSAM is not a new thing**\n\nWhile both our method and FDMNet explore feature extraction in the frequency domain, they differ in objectives and model design. FDMNet aims for modal-invariant feature learning via frequency domain decomposition, using the Instance Adaptive Amplitude Filtering (IAF) and Phase Preservation Normalization (PPNorm) modules to enhance modal-invariant components, as we as suppressing modality-specific ones. In contrast, our FSAM module integrates frequency and spatial features for adversarial feature alignment, using Universal Adversarial Perturbations (UAP) to generate adversarial samples. FSAM unifies features by combining the frequency and spatial domains, making visible and infrared image features more consistent.\n\nIn summary, FDMNet emphasizes the differences in amplitude components within the frequency domain, striving to enhance the consistency of modal features. Conversely, FSAM focuses on the commonality of phase components in the frequency domain, facilitating the alignment of adversarial features in both frequency and spatial domains.\n\n**Q:How could a real-world application employ this method for an attack in a visible-infrared surveillance system? The motivation for the methodology of this paper requires further elaboration.**\n\nThis study presents a digital attack on VIReID, offering a new perspective on ReID system security. However, it also raises ethical and security concerns about the potential misuse of adversarial attack techniques, which could threaten public safety.\n\nDespite these concerns, adversarial attack research has positive value. It uncovers vulnerabilities in existing systems, encouraging academia and industry to improve the robustness of machine learning models. This research assesses the robustness of VIReID systems and combines adversarial training with proposed attack methods, enhancing system security and benefiting society by promoting a safer technological environment.\n\nAdditionally, there is exploration of physical attacks in real-world applications. For example, AGNs[1] can create ordinary-looking glasses that cause facial recognition systems to misidentify individuals. AdvTexture[2] can cover clothing with arbitrary shapes, making individuals wearing such clothing undetectable by human detection systems. Wei et al.[3] propose using insulating materials to create physically feasible infrared patches with learnable shapes and positions, allowing pedestrians to evade infrared detectors.\n\n[1] Adversarial Generative Nets: Neural Network Attacks on State-of-the-Art Face Recognition. Arxiv 2018\n\n[2] Adversarial Texture for Fooling Person Detectors in the Physical World. CVPR 2022\n\n[3] Physically Adversarial Infrared Patches with Learnable Shapes and Locations. CVPR 2023", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GaCViITujA", "reviewer": "Reviewer_sAfC", "summary": "There is currently a lack of research focused on the security of VIReID systems. In light of this, the authors are the first to propose a method to disrupt the output ranking of VIReID systems by leveraging feature-level adversarial attacks, considering the specific characteristics of VIReID. This paper introduces Universal Adversarial Perturbations (UAP) and adopts a Frequency-Spatial Attention Module (FSAM) to integrate frequency and spatial information, ensuring the consistency of adversarial features. Additionally, the authors propose an Auxiliary Quadruple Adversarial loss to amplify modality differences, thereby disrupting the system's output ranking results.", "questions": "1. In Section 3.2.2, what are the meanings of features x_v1 and x_i1, and how are they generated?\n2. Regarding Table 2, the methods selected for attacking different VIReID systems are somewhat limited, which does not seem sufficient to validate the generalizability of the proposed approach. It is recommended to include more VIReID methods.\n3. According to Table 3, the authors explore the effectiveness of different modules, but there are no experimental investigations into the effectiveness of the two specific modules, which I think is insufficient.", "limitations": "The authors explain the limitations of their work, and there is no negative societal impact.", "rating": 7, "confidence": 5, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1.This study is the first to propose research on the security of VIReID systems, filling a gap in this field.\n2.The motivation behind this paper is strong. By introducing UAP, the method is able to generate universal adversarial samples adaptable to different modalities. It employs the FSAM module to enhance the consistency of adversarial features between modalities. L_AQA is able to effectively disrupt the system's output ranking by amplifying modality differences. This method aligns with the characteristics of digital attacks and caters to the task requirements of VIReID.\n3.The authors conducted extensive experiments on different datasets and with different backbone models, validating the effectiveness and generalizability of the method. The results indicate that the proposed method achieves state-of-the-art performance.\n4.The paper is well-structured, logically clear, and well-organized.", "weaknesses": "1.The paper points out the differences from single-modality ReID research but does not clearly explain the specific distinctions between them. Additionally, it does not clarify why adversarial feature alignment and modality differences need to be emphasized.\n2. What is the total loss function? What is the meaning of ‘L_id’ in Figure 2?\n3.The authors suggest that the phase components of the features after Fourier transformation are more closely related to spatial information, but there are no corresponding visualization experiments to support this claim.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**w1 :Comparison with ReID**\n\nAs illustrated in Figure 2, the differences and shared characteristics between visible and infrared pedestrian images can be understood. Compared to ReID, VIReID can extract features from two different types of images. While VIReID leverages the complementary information from both modalities, it also introduces additional modality-specific differences, such as the loss of crucial color information. However, the spatial information in both modalities remains unaffected and can mutually aid in capturing pedestrian features. By utilizing the FSAM, VIReID integrates global features from the frequency domain with detailed information from the spatial domain, thereby enhancing feature consistency and improving the representation capability of adversarial features.\n\n**w2: What is the total loss function? What is the meaning of ‘$\\cal{L_{id}}$’ in Figure 2?** \n\nThe total loss function can be formulated as follows:\n\n$$\n\\cal{L_{total}} = \\cal{L_{AQAL}} + \\cal{L_{id}}\n$$\n\n$\\cal{L_{id}}$ represents the identity loss. The training process of VIReID is considered an image classification problem, where each identity is a distinct class. During the testing phase, the output from the pooling layer or embedding layer is used as the feature extractor. Given an input image $x{i}$ with label $y{i} $, the probability of $x{i}$ being recognized as class $y{i}$is encoded using the softmax function and denoted as $p(y{i}|x_{i})$. The identity loss is then computed by the cross-entropy\n\n$$\n\\begin{equation}\n\t{\\cal{L_{id}}} = -\\frac{1}{N}\\sum_{i=1}^{N}log(p(y_{i}|x_{i})).\n\\end{equation}\n$$\n\nwhere N represents the number of training samples within each batch.\n\n**w3: Visual experiments supporting FSAM**\n\nAs shown in Figure 3, in the task of VI-ReID, the amplitude component captures the overall brightness and contrast of pedestrian images, reflecting the luminance and color information of the image, while the phase component captures the structural information and details of the pedestrians, including their shape and outline, to help distinguish between different details and features of pedestrians. The combination of these components allows for the effective extraction of global and local features of pedestrians. By focusing further on spatial information characteristics in the phase component, attention to spatial information in the frequency domain is increased, enhancing the ability to express distinguishing features.\n\n**Q1: the meanings of features $x_{v1}$ and $x_{i1}$**\n\nAfter applying the FSAM module to the spatial components of the original features, $x_{v1}$  and  $x_{i1}$ are combined with the amplitude components of the original features and then transformed back to the original domain using the inverse Fourier transform (IFFT). This process can be described as:\n$$\n\\begin{equation}\n\tx_{v1} = IFFT(x_{vp}^{'},x_{va});\n\\end{equation}\n$$\n\n$$\n\\begin{equation}\n\tx_{i1} = IFFT(x_{ip}^{'},x_{ia});\n\\end{equation}\n$$\n\n**Q2: Attacking more VIReID systems**\n\nWe understand the reviewer's concern regarding the breadth of attack method validation. In this study, we chose the DDAG[1] and MRCN[2] systems as targets for the attacks. These two methods are representative in the existing VIReID field, employing dual attention mechanisms and modal restoration and compensation mechanisms, respectively, to reduce modality differences, covering different technical approaches. According to Table 4, although their modules provided some resistance, they still achieved very good results. This further demonstrates the broad applicability and effectiveness of our method.\n\n| Methods | SYSU-MM01      |                 |                | RegDB          |                |                |\n| ------- | -------------- | --------------- | -------------- | -------------- | -------------- | -------------- |\n|         | Rank-1         | Rank-10         | mAP            | Rank-1         | Rank-10        | mAP            |\n| DDAG    | 54.75/**1.65** | 90.39/**10.86** | 53.02/**3.27** | 69.34/**0.94** | 91.49/**9.70** | 63.46/**1.35** |\n| MRCN    | 70.80/**2.36** | 96.50/**11.18** | 67.30/**5.87** | 95.10/**2.36** | 98.80/**8.87** | 89.20/**1.16** |\n\n[1] Dynamic dual-attentive aggregation learning for visible-infrared person re-identification. ECCV 2020\n\n[2] MRCN: A Novel Modality Restitution and Compensation Network for Visible-Infrared Person Re-identification. AAAI 2023\n\n**Q3: Verify the effectiveness of different modules and their combination**\n\nWe investigate the effectiveness of different modules and supplement our experiments with the effectiveness of each pair of specific modules. Comparing rows 5-7 with rows 1-3 in Table 5, our proposed modules are effective when used individually and show additive effectiveness when combined. Our experiments validate the superiority of the proposed method.\n\n| Noise | FSAM | SFM  | ${\\cal{L}}_{AQAL}$ | SYSU-MM01 |          | RegDB    |          |\n| ----- | ---- | ---- | -------------------- | --------- | -------- | -------- | -------- |\n|       |      |      |                      | Rank-1    | mAP      | Rank-1   | mAP      |\n|       |      |      |                      | 46.73     | 45.78    | 82.24    | 76.52    |\n| ✔     |      |      |                      | 16.67     | 18.28    | 33.00    | 30.60    |\n| ✔     | ✔    |      |                      | 12.75     | 12.17    | 15.74    | 12.91    |\n| ✔     |      | ✔    |                      | 11.04     | 13.19    | 20.73    | 19.81    |\n| ✔     |      |      | ✔                    | 14.62     | 15.52    | 15.86    | 15.53    |\n| ✔     | ✔    | ✔    |                      | 5.10      | 6.35     | 7.58     | 7.51     |\n| ✔     | ✔    |      | ✔                    | 1.07      | 3.27     | 0.58     | 1.35     |\n| ✔     |      | ✔    | ✔                    | 1.74      | 3.81     | 6.70     | 5.77     |\n| ✔     | ✔    | ✔    | ✔                    | **0.79**  | **2.81** | **0.49** | **0.85** |", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank all reviewers for their valuable feedback, with three reviewers (sAfc, jhhd, and 78wy) strongly supporting our work. We are pleased to see that reviewers consider our paper:

- The ideas presented are novel and interesting (Reviewer sAfc);
- The theoretical proofs are solid (Reviewer 78wy);
- Strong theoretical guarantees are provided (Reviewer jhhd);
- Extensive experiments demonstrate the effectiveness of the proposed method (Reviewer ksiy).

Reviewer ksiy's main concern is that adversarial examples may not be applicable to the real world and could cause information loss. We address this by highlighting the significance of adversarial attacks through their development history and applications across various fields, emphasizing their role in testing model robustness and providing a new perspective on system security. Additionally, we performed visualization experiments of adversarial samples and included more VIReID systems of different types to validate the effectiveness and generalizability of our method. Furthermore, we compared the effectiveness of our proposed modules through experiments on perturbed results, demonstrating the indispensability of each module.

All questions have been addressed in responses specific to each reviewer. Additionally, please refer to the attached PDF, which includes supplementary tables and figures. These are referenced and described in our individual responses to the reviewers.

Meta-review: After rebuttal, all the reviewers unanimously vote for acceptance of this work. After checking the rebuttal, the review, and the manuscript, the AC recommends acceptance.
Decision: Accept (poster)