Feature-Level Adversarial Attacks and Ranking Disruption for Visible-Infrared Person Re-identification

Authors: xi  yang
huanling  liu
de  cheng
nannan  wang
xinbo  gao
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 16:47:04

Abstract: Visible-infrared person re-identification (VIReID) is widely used in fields such as video surveillance and intelligent transportation, imposing higher demands on model security. In practice, the adversarial attacks based on VIReID aim to disrupt output ranking and quantify the security risks of models. Although numerous studies have been emerged on adversarial attacks and defenses in fields such as face recognition, person re-identification, and pedestrian detection, there is currently a lack of research on the security of VIReID systems. To this end, we propose to explore the vulnerabilities of VIReID systems and prevent potential serious losses due to insecurity. Compared to research on single-modality ReID, adversarial feature alignment and modality differences need to be particularly emphasized. Thus, we advocate for feature-level adversarial attacks to disrupt the output rankings of VIReID systems. To obtain adversarial features, we introduce \textit{Universal Adversarial Perturbations} (UAP) to simulate common disturbances in real-world environments. Additionally, we employ a \textit{Frequency-Spatial Attention Module} (FSAM), integrating frequency information extraction and spatial focusing mechanisms, and further emphasize important regional features from different domains on the shared features. This ensures that adversarial features maintain consistency within the feature space. Finally, we employ an \textit{Auxiliary Quadruple Adversarial Loss} to amplify the differences between modalities, thereby improving the distinction and recognition of features between visible and infrared images, which causes the system to output incorrect rankings. Extensive experiments on two VIReID benchmarks (i.e., SYSU-MM01, RegDB) and different systems validate the effectiveness of our method.

Main paper: 
 introduction: 
 VIReID is widely applied in key tasks such as security monitoring [1] [2] [3] . Suppose the law enforcement agency of a city uses ReID system to monitor public places for tracking criminal suspects. Internal personnel may attempt to deceive the system by modifying the images [4] [5] [6] of criminal suspects due to improper behavior or other reasons, in order to protect specific individuals. According to Figure1, infrared adversarial samples will erroneously match visible samples, while visible adversarial samples will also erroneously match infrared samples. The credibility and stability of VIReID are crucial in such special application scenarios. However, there is currently insufficient theoretical research on the security of VIReID. Therefore, this paper explores how to obtain better adversarial features and how to address the task characteristics of VIReID modality differences and re-ranking. The current research on adversarial attacks in the fields of face recognition and person re-identification mainly focuses on digital attacks [7] [8] [9] , which refer to the manipulation, distortion, or tampering of digital images to deceive, disrupt, or mislead systems, leading to erroneous identification results or reduced recognition accuracy [10] . Adversarial attacks based on ReID focus on crafting adversarial samples suitable for visible images and disrupting internal rankings. In contrast, VIReID requires more consideration regarding the generalizability of attack methods under different imaging mechanisms and how to align adversarial features between infrared and visible images. Moreover, dealing with two modalities necessitates comparing distances between modalities and within modalities. To address these challenges, we propose a feature-level adversarial attacks and ranking disruption for VIReID.First, we adopt the method of universal adversarial perturbation (UAP) [11] to generate adversarial samples, which seeks a set of universal perturbations independent of the image and can generalize well in deep neural networks. At the same time, it significantly lowers the threshold for implementing adversarial attacks and adapts to different systems. Secondly, to make visible and infrared images more consistent in the feature space, we propose a frequency-spatial attention module, achieving adversarial feature alignment by unifying frequency and spatial features. Visible and infrared images are generated under different imaging conditions; the former provides rich texture information, while the latter contains significant pixel amplitude information. This module uses fast Fourier transform to decompose features into amplitude and phase, corresponding to the texture details and spatial position information of images, respectively. Since these features are closely related to the spatial domain, a spatial attention module is chosen for these two components to further emphasize or suppress different regions in the feature map. Additionally, a weighted spatial attention module is applied to the shared features to maintain consistency in the feature space. This method not only focuses on the frequency domain features of visible and infrared images from different imaging conditions but also emphasizes their spatial features for significant pedestrian poses, thereby achieving adversarial feature alignment. To disrupt the ranking of identification results, we propose an auxiliary quadruple adversarial loss function. The visible and infrared pedestrian features extracted by the first-stage model are used as auxiliary features in the calculation process of the features loss function extracted in the second stage. By pulling the distance between the same modality and the same person closer and pushing the distance between different modalities and the same person farther, while also ensuring that the intra-class distance is smaller than the inter-class distance, the differences between modalities are expanded and the ranking under different modalities is disrupted. By utilizing generated features containing multiple types of information, the network's ability to explore features at different levels is enhanced. That is, with four types of features, we ensure a double guarantee to achieve the goal of disrupting the ranking. The main contributions of this paper can be summarized as follows:• We are the first to propose exploring the security of VIReID, considering the alignment of adversarial features across modalities in VIReID.• We propose a frequency-spatial attention module that integrates frequency-domain features with spatial features to enhance the consistency and representation ability of adversarial features.• We design an auxiliary quadruple adversarial loss function, which utilizes auxiliary features to amplify the differences within and between modalities, thereby disrupting the ranking results. 
related work: 
 Visible-Infrared Person Re-identification. VIReID [12] [13] [14] [15] refers to the technique of identifying and matching pedestrians from one modality to another using visible or infrared images. Moreover, VIReID finds wide application in the field of security, enhancing the intelligence level of security systems. The significant differences between different modalities make VIReID challenging. To alleviate the modality discrepancy at the feature level, some methods adopt single-stream, dual-stream, or multi-stream networks [16] [17] [18] [19] , extracting shared features from different modalities by designing various attention mechanisms and loss functions [20] [21] [22] [23] . Ye et al. [16] proposed the concept of metric learning, jointly optimizing modality-specific and modality-shared matrices. Subsequently, Zhu et al. [18] proposed a hetero-center loss, which for the first time shortens the distance between feature centers of the same identity, bridging the gap between features of the same pedestrian across different modalities. In addition, Ling et al. [24] devised a multi-constraint similarity learning approach to comprehensively explore the relationships between cross-modal information. Meanwhile, as the posture and shape of pedestrians provide important information in the recognition process, these methods mainly focus on spatially enhancing feature representation. However, there are also differences in frequency information between visible and infrared images. Li et al. [25] proposed a novel frequency-domain modality-invariant feature learning framework to reduce modality differences from a frequency-domain perspective.Adversarial Attacks. In the fields of computer graphics and pattern recognition, adversarial attacks [26] [27] [28] [29] are an important area of research. Many studies have revealed the vulnerability of deep models to carefully crafted small perturbation, resulting in significant errors with high confidence in predictions. Adversarial attacks aim to explore adversarial noise that causes deep learning models to behave abnormally. FGSM [30] belongs to the single-step attack algorithm, which optimizes the loss by quickly determining the direction of perturbation for input samples and calculates the adversarial perturbation through backpropagation. The concept of iterative thinking was subsequently incorporated into FGSM, thereby leading to the development of Projected Gradient Descent [31] (PGD). Subsequently, adversarial attacks on ReID were also studied, with the core idea of generating well-crafted adversarial examples or disturbing ranking results. LTA [32] used local grayscale iteration to generate adversarial examples, mainly focusing on disturbing the color of the original image. A mis-ranking formula was proposed by DMR [33] to increase the distance between images of the same pedestrian while decreasing the distance between images of different pedestrians, effectively disrupting the ranking results.Cross-modality Attacks. Adversarial instances are widely present across various domains of visible images. In recent years, exploration of adversarial instances in the field of infrared pedestrian detection has begun. Osahor et al. [34] discussed perturbation by altering pixel values within infrared images. Subsequently, Zhu et al. [35] attempted for the first time to alter the infrared radiation distribution of the human body by simulating additional heat sources using a set of small bulbs, generating physical adversarial examples. To be more easily implemented in the physical world, Wei et al. [36, 37] considered the different imaging mechanisms of visible and infrared sensors, proposing a unified adversarial patch to execute cross-modality physical attacks. Meanwhile, we chose to perform security evaluation on visible-infrared person re-identification models proposed in recent years in the digital world.Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [No] Justification: Our approach was tested on only a few datasets and systems, demonstrating the feasibility of the theory without exploring its application in real-world scenarios. Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
proposed method: 
 As shown in Figure 2 , the overall structure of the proposed method adopts a dual-stream ResNet network as the backbone. Firstly, Universal Adversarial Perturbation (UAP) are added separately to visible and infrared images. In the first stage, the Frequency-Spatial Attention Module (FSAM) is embedded to extract frequency-domain spatial correlated features as auxiliary features for images of two different modalities, visible and infrared. Subsequently, through a shared module, further focus is applied to the temporal domain features. This completes the focus on spatial features in both frequency and normal domains, making the learned information more diversified, thereby completing the learning of features in the second stage. During the training phase, the features from the first stage are used as auxiliary features and combined with the features from the second stage, all of which are inputted into the Auxiliary Quadruple Adversarial Loss for optimizing the entire module.Universal Adversarial Perturbation (UAP) aims to generate a single perturbation that can be added to any image from the same distribution, resulting in mis-classification when added. Deep neural networks are highly susceptible to this type of UAP, yet they remain imperceptible to the human eye. Defining a set of images I that satisfies distribution µ, where g(I) fits the output function, and after perturbation δ, the labels are not equal. That is: g(I + δ)̸ =g(I).(1) When δ is constrained by two conditions simultaneously, the optimization problem of finding universal perturbation can be described as follows:EQUATIONwhere P (•) represents probability, || • || p denotes the p-norm, ε indicates the magnitude of the perturbation to ensure that the adversarial perturbation is visually imperceptible, µ represents the data distribution, ϵ ∈ (0,1] denotes the success rate of deception to ensure the attack's success rate. The goal is to find an adversarial perturbation δ that can be added to all sample points and will result in misclassification of adversarial samples with a probability of 1 -ϵ. The UAP algorithm does not require solving optimization problems or gradients of the model and is applied in scenarios where a large number of adversarial samples need to be quickly generated.So we choose universal perturbation for addition, which can be used on both visible and infrared images and can also adapt to scenarios with a large number of pedestrian samples in real-world environments. The introduction of UAP can make the model more independent of specific data distributions and training sets, thereby reducing the model's dependence on specific data and making the system more flexible and adaptable to challenges in different environments and scenarios. Visible and infrared images generate their respective adversarial samples as follows:EQUATIONEQUATIONFor the input Q vi and Q ir , adding perturbation(δ vi ,δ ir ) related to the data distribution to each, the generated adversarial sample Qvi , Qir can deceive the system by exploiting visual similarity attacks.The Fast Fourier Transform (FFT) is widely utilized in the field of image processing to convert images into the frequency domain, enabling the analysis of the frequency components of the image. This aids in understanding the overall structure, texture, and edge information of the image. Therefore, we combine the Fast Fourier Transform with a spatial attention module to focus on the unique features in the frequency domain of the image, which aids in enhancing feature representation capabilities. First, we provide a brief introduction to the basic concepts of the FFT. Given the feature x ∈ R C×H×W output by network, its FFT can be expressed as follows:EQUATIONwhere j represents the imaginary unit, u and v are the horizontal and vertical coordinates of the x, and F(•) denotes the Fourier transform, C, H and W denote the number of the channel, height and width of features. The frequency-domain feature F(x) is represented asF(x) = R(x) + jI(x),where R(x) and I(x) represent the real and imaginary part of F(x). These real and imaginary parts can be converted to amplitude and phase spectrums, which can be formulated as follows:EQUATIONEQUATIONAs shown in Figure 3 the amplitude component of the visible pedestrian features after FFT is represented as x va and the phase component as x vp , while the amplitude component of the infrared pedestrian features after FFT is represented as x ia and the phase component as x ip .The spatial attention module generates spatial attention maps using the spatial relationships within the features. It focuses on the distribution of information, locating the position and shape of the subject while reducing background information interference. We apply max pooling and average pooling operations along the channel axis. These two pooling operations respectively extract the maximum and average value information from the feature map and concatenate them to form a richer feature descriptor. Pooling is performed along the channel axis, which helps highlight information-rich regions in the feature map and better locate key information. In summary, the specific calculation process is as follows:EQUATIONwhere M (x) denotes the weight distribution generated by applying the convolution layer and Sigmoid denotes the sigmoid function. f represents a convolution operation with the filter size of 7 × 7. M axP ool(x) and AvgP ool(x) represent the features after passing through the max-pooling layer and average-pooling layer, respectively.Given that the phase component involves more spatial information, we sequentially apply the spatial attention module to obtain the new phase component(x′ vp ,x ′ ip), which can be expressed as follows:xEQUATIONxEQUATIONwhere ⊗ denotes element-wise multiplication, M v (x vp ) and M i (x ip ) represent the attention weights generated by the feature phase components. The combination of the original amplitude component and the new phase component can reconstruct the original feature information through inverse fast Fourier transform(IFFT). At this point, the network outputs features x v1 and x i1 , which will serve as auxiliary features. This process can be described as:EQUATIONEQUATIONThe features are then further input into a shared module T in the model to complete feature extraction. At this point, choosing to pass through the spatial attention module allows for focusing on spatial information in the original domain. This two-stage process completes the focus on both frequency domain and original domain spatial information, enhancing the representation capability of distinguishing features. Thus, the final extracted visible and infrared features can be expressed as:EQUATIONEQUATIONwhere M ′ v and M ′ i indicates the weight distribution generated by the second-stage SAM module, resulting in the final stage features, namely x v2 and x i2 .We propose an auxiliary quadruplet adversarial loss function to disrupt the system's output ranking. This method effectively adapts to ReID issues by attacking the predicted ranking results. Additionally, considering the modality differences involved in VIReID, we introduce auxiliary features to fully leverage the information disparities between modalities, erroneously outputting cross-modality ranking results. In this process, there are both modality differences and identity differences. Therefore, we approach it from two aspects: under the condition of the same modality, minimizing the distance between the same identities and maximizing the distance between different identities; under the condition of the same identity, minimizing the distance between the same modalities and maximizing the distance between different modalities. Let's start by controlling the same modality case:EQUATIONwhere x j v1 represents the auxiliary feature, j and k denote different pedestrians, D(•, •) represents the Euclidean distance between two feature vectors and [x] + = max(x, 0). When the modalities are the same, the task is transformed into a single-modality ReID. This problem discusses the disruption of matching pairs across modalities, while still maintaining the original requirement for the same modality, namely, encouraging that the maximum distance between the most easily identifiable pairs of images across identities is still less than the minimum distance between the most easily identifiable pairs of images within an identity, ensuring that the sorting output within the same modality is normal.This part ensures that the original order and matching relationships within the modality are not disrupted.When the identities are consistent across modalities:EQUATIONwe encourage reducing the distance between pedestrians in the same modality while increasing the distance between pedestrians in different modalities. This strategy aims to narrow the distance within the same modality while widening the distance between different modalities, thereby increasing the modalities differences. It disperses the features in the feature space, making it more challenging for the model to cluster features. This is intended to impact the matching and sorting results across modalities. The loss function augmented with visible features can be expressed as:EQUATION)where N is the number of person ID in a mini-batch. Meanwhile, using more discriminative embedding centers(c v1 , c v2 , c i1 , c i2 ) for each class, we introduce a margin term α to balance the two terms. Thus, the loss function augmented with visible features and infrared features can be expressed as follows:EQUATIONEQUATIONFinally, the auxiliary quadruplet adversarial loss function (L AQAL ) is ultimately formulated as:EQUATIONThe L AQAL forces the distance between modalities to increase, preventing them from easily clustering together, thereby disrupting the overall ranking results.Besides the auxiliary quadruple adversarial loss L AQAL , we also have the identity loss L id . The training process of VIReID is considered an image classification problem, where each identity is a distinct class. During the testing phase, the output from the pooling layer or embedding layer is used as the feature extractor. Given an input image x i with label y i , the probability of x i being recognized as class y i is encoded using the softmax function and denoted as p(y i |x i ). The identity loss is then computed by the cross-entropyEQUATIONwhere N represents the number of training samples within each batch.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?Answer: [No] Justification: We will consider making the code details publicly available in the future.Guidelines:• The answer NA means that paper does not include experiments requiring code. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
implementation details: 
 The experiments are conducted on an NVIDIA GeForce 3090 GPU with Pytorch. We chose the powerful baseline model AGW [38] horizontal flipping and random erasing with a probability of 0.5. We optimize using the stochastic gradient descent (SGD) optimizer, with a weight decay set to 0.0005 and a momentum parameter set to 0.9. The initial learning rate for both datasets was set to 0.1, and it was decayed by a factor of 0.1 at the 20th and 50th epochs, respectively. Finally, the margin α in the auxiliary quadruplet adversarial loss function is set to 0.2. We adopt a warm-up learning rate scheme, with a total of 60 training epochs.SYSU-MM01 [1] is a cross-modality pedestrian re-identification dataset proposed in 2017. There are 287,628 visible images and 15,792 infrared images in total. Cameras 1 and 2 are installed in well-lit environments, cameras 3 and 6 operate under infrared conditions, and cameras 4 and 5 are placed in outdoor scenes. The dataset comprises two testing modes: all-search and indoor-search. The all-search mode is more challenging because the gallery includes images from all cameras.The RegDB [43] dataset comprises 412 individuals, with each person having 20 images, including 10 visible images and 10 infrared images. Among the 412 individuals, there are 254 females and 158 males, with 156 individuals captured in frontal views and 256 individuals captured in rear views.During the testing phase, RegDB offers two modes: visible to infrared and infrared to visible. In the visible to infrared mode, visible images serve as query images, while infrared images are used as gallery images. The infrared to visible mode operates in the opposite manner.We select six different methods to evaluate the security of the AGW model. Among them, FGSM and PGD are typical gradient-based methods for generating adversarial samples, while UAP generates universal perturbations that are independent of the image. Additionally, we chose SMA, DMR, and LTA, three pedestrian re-identification attack methods which primarily disrupt ranking and generate adversarial samples through local color iteration. The results, as shown in Table1, indicate a sharp decline in all evaluation metrics after attacking AGW across two datasets and two different tests. In the RegDB dataset, under two testing modes, our mAP decreased dramatically from 66.37% and 65.90% to 0.85% and 0.89%, nearly approaching 0. This demonstrates that our method is more suitable for testing robustness against VIReID tasks.To demonstrate the universality of our method, we select three approaches to improve the recognition accuracy of VIReID systems: HCLoss enhances intra-class cross-modality similarity through a heterogeneous center loss function, CAJ improves recognition accuracy based on data augmentation techniques, and MMN and DEEN utilize different network designs to better extract effective shared features. These methods are commonly used to improve VIReID accuracy at the feature level. As shown in Table2, our experimental results show that after applying our attack method, the mAP of all systems dramatically decreased across different datasets and test modes (for example, the Rank-1 of DEEN drops from 74.70% to 1.71%, and mAP falls from 71.80% to 3.55% in all-search mode), indicating that these systems cannot withstand our attacks. This demonstrates that our method can effectively test the robustness of various VIReID systems.As shown in Table3, the effectiveness of different modules is validated on two datasets. Using the SYSU-MM01 dataset as an example, after adding noise to generate adversarial samples, the mAP decreases from 45.75% to 18.28%. Subsequently, the three modules are individually tested on this basis, and all show a decrease in accuracy, with the FSAM module being the most effective, reducing the mAP to 12.17%. Additionally, since both FSAM and SAM are designed to enhance adversarial feature representation capabilities, the combined effect of these two modules is tested, further reducing the mAP to 6.35%. The experiments demonstrate that all three proposed modules are highly effective.Although targeting different aspects, their combined usage enhances the effectiveness of method.Determining which stage of ResNet-50 should have the FSAM module inserted. In this experiment, we use ResNet-50 as the backbone, which has five stages: block 0 to block 4. We study the impact on model performance by inserting the FSAM module after different stages of ResNet-50. This experiment controls only the insertion position of FSAM as the variable, keeping all other factors constant. As shown in Table4, inserting FSAM after block 0 and block 4 results in lower accuracy, indicating better performance. This suggests that in the shallow layers of the network, image processing captures more effective information from the initial frequency domain features, while in the deeper layers of the network, the feature representation capability is further enhanced, and spatial information becomes more important. Based on these results, we integrate the proposed FSAM module after block 0 and block 4 of the ResNet-50 model.As shown in Table 5 , we compared our frequency domain attention module with channel attention mechanisms like SE-Net [44] and ECA-Net [45] . The results reveal that while SE-Net and ECA-Net excel in certain areas, our frequency domain attention module is highly competitive across various metrics. It achieves the best Rank-1 and mAP scores on both datasets. Channel attention mechanisms often focus on global features, which may overlook local details and spatial relationships in cross-modal VIReID tasks, leading to potential information loss and reduced performance. We compare the t-SNE visualization results on the baseline and the proposed method. To ensure fairness, we randomly select several images of ten identities from all cameras. For each individual, 20 visible images and 20 infrared images are randomly chosen. As shown in Figure4, after the attack, the clustering of all pedestrians became more dispersed, with the visible and infrared modalities each forming their own clusters, and the relative distance between them increasing. The visualization experiments validated the effectiveness of our attack method, as the attack further exaggerated the gap between the two modalities, effectively suppressing the output performance. These results demonstrate that our method is highly effective for security testing in VIReID.Baseline Attacked  
conclusion: 
 In this paper, our objective is to conduct security validation of VIReID systems and propose a novel attack method suitable for cross-modality tasks. We introduce the frequency-spatial attention module, which are used at two stages of feature extraction, focusing on spatial information in both frequency and source domains to enhance the representation capability of adversarial features and strengthen the effectiveness of adversarial samples. Additionally, we propose an auxiliary quadruple adversarial loss function considering the modalities differences involved in VIReID tasks to interfere with the ranking of system outputs, completing the robustness test of the current VIReID system. Extensive experiments not only deepens the understanding of the security of cross-modality ReID systems but also provides a new direction for the development of VIReID and emphasizes the importance of ensuring their reliability and protection in practical applications. 
acknowledgement: 
 This Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes] Justification: We provide a detailed explanation in Section 4Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [No] Justification: Our experimental results do not include error bars, confidence intervals, or tests for statistical significance.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).• The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. • It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). • If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [Yes] Justification: We present the relevant computational information in Section 4.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines? 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes]Justification: While our method can be applied for robustness testing of models, it also implies that certain groups might use this method to attack systems, resulting in potentially destructive impacts. These are considerations that need to be taken into account in practical deployments.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [Yes]Justification: We use publicly available datasets and do not involve scraping data from the internet.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort. 
licenses for existing assets: 
 Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes]Justification: We ensure that all assets used are properly credited and adhere to the relevant licenses and terms of use.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. Justification: The pedestrian dataset used in this paper is publicly available, and it is stated by the publisher that each pedestrian captured in the dataset has signed a privacy release allowing the use of the images for scientific research and display in research papers. Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector. 
institutional review board (irb) approvals or equivalent for research with human subjects: 
 Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained? Answer: [No] Justification: The pedestrian dataset used in this paper is publicly available, and it is stated by the publisher that each pedestrian captured in the dataset has signed a privacy release allowing the use of the images for scientific research and display in research papers. Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: Justification: We have reviewed the relevant requirements and ensured compliance with ethical guidelines.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 


Decision: After rebuttal, all the reviewers unanimously vote for acceptance of this work. After checking the rebuttal, the review, and the manuscript, the AC recommends acceptance.

Reviews: review: paper_summary: paper_summary: This paper addresses the security of visible-infrared person re-identification systems by introducing a method for feature-level adversarial attacks. The proposed approach integrates universal adversarial perturbations and a frequency-spatial attention module to disrupt the output ranking of VIReID systems. The auxiliary quadruple adversarial loss is designed to amplify modality differences, enhancing the distinction between visible and infrared features. Extensive experiments on the SYSU-MM01 and RegDB benchmarks validate the effectiveness of this method in compromising VIReID systems' rankings.
strength_weakness: strength_weakness: 1. The paper could benefit from a more in-depth analysis of the failure cases and limitations of the proposed method. For instance, under what conditions does the attack fail, and why?
2. The paper primarily focuses on the effectiveness of the attack in terms of ranking disruption. Additional metrics, such as computational efficiency and impact on overall system performance, could provide a more comprehensive evaluation.
3. Minor issue: Equation 15 appears to have an error.
questions: questions: Can the authors provide more insight into the computational complexity of the proposed method?
limitations: limitations: No limitations or negative impacts have been identified in this paper.
paper_summary: paper_summary: This paper aims to explore the security of VIReID and introduces a Universal Adversarial Perturbations to simulate common disturbances in real-world environments. Additionally, a Frequency-Spatial Attention Module is proposed to integrate frequency information extraction and spatial focusing mechanisms. An Auxiliary Quadruple Adversarial Loss is proposed to amplify the differences between modalities, thereby improving the distinction and recognition of features between visible and infrared images. Extensive experiments on two VIReID benchmarks (i.e., SYSU-MM01, RegDB) and different systems validate the effectiveness of the proposed method.
strength_weakness: strength_weakness: 1) Since the author proposed a frequency domain attention module, the frequency domain and attention need to be introduced in the related work section.

2) What is the motivation of the proposed frequency domain attention module for the VIReID task? 

3) The proposed frequency domain attention module uses a spatial attention module to generate spatial attention maps using the spatial relationships within the features. I'm quite curious about the performance if other attention mechanisms (such as SE-Net, Channel) are used. 
Furthermore, could the authors provide the performance of the proposed frequency domain attention module on the VIReID method, thereby proving its effectiveness?
questions: questions: Please check the weakness.
limitations: limitations: The authors have adequately addressed the limitations of the proposed method.
paper_summary: paper_summary: There is currently a lack of research on the security of VIReID systems. This paper proposes to explore the vulnerabilities of VIReID systems and prevent potential serious losses due to insecurity. To obtain adversarial features, this paper introduces Universal Adversarial Perturbations (UAP) to simulate common disturbances in real-world environments. Additionally, the authors employ a Frequency-Spatial Attention Module (FSAM), integrating frequency information extraction and spatial focusing mechanisms, and further emphasize important regional features from different domains on the shared features. Extensive experiments on two VIReID benchmarks (i.e., SYSU-MM01, RegDB) and different systems validate the effectiveness of the proposed method.
strength_weakness: strength_weakness: 1. Adversarial attack in VI-ReID is a meaningful research topic. However, merely utilizing the noise-added image as an adversarial sample is singular and impractical. How could a real-world application employ this method for an attack in a visible-infrared surveillance system? Numerous other methods for generating adversarial samples are not discussed in the paper, thereby significantly diminishing its value.
2. This paper does not present the image with the added noise. If the image loses its original information after noise is introduced, the recognition results will be poor even without using the method of this paper.
3. FSAM is not a new thing, which has been proposed in many works [1] and is not innovative.
[1] Li Y, Zhang T, Zhang Y. Frequency Domain Modality-invariant Feature Learning for Visible-infrared Person Re-Identification[J]. arXiv preprint arXiv:2401.01839, 2024.
questions: questions: How could a real-world application employ this method for an attack in a visible-infrared surveillance system? The motivation for the methodology of this paper requires further elaboration
limitations: limitations: Authors need to consider more methods of attack than just adding noise.
paper_summary: paper_summary: There is currently a lack of research focused on the security of VIReID systems. In light of this, the authors are the first to propose a method to disrupt the output ranking of VIReID systems by leveraging feature-level adversarial attacks, considering the specific characteristics of VIReID. This paper introduces Universal Adversarial Perturbations (UAP) and adopts a Frequency-Spatial Attention Module (FSAM) to integrate frequency and spatial information, ensuring the consistency of adversarial features. Additionally, the authors propose an Auxiliary Quadruple Adversarial loss to amplify modality differences, thereby disrupting the system's output ranking results.
strength_weakness: strength_weakness: 1.The paper points out the differences from single-modality ReID research but does not clearly explain the specific distinctions between them. Additionally, it does not clarify why adversarial feature alignment and modality differences need to be emphasized.
2. What is the total loss function? What is the meaning of ‘L_id’ in Figure 2?
3.The authors suggest that the phase components of the features after Fourier transformation are more closely related to spatial information, but there are no corresponding visualization experiments to support this claim.
questions: questions: 1. In Section 3.2.2, what are the meanings of features x_v1 and x_i1, and how are they generated?
2. Regarding Table 2, the methods selected for attacking different VIReID systems are somewhat limited, which does not seem sufficient to validate the generalizability of the proposed approach. It is recommended to include more VIReID methods.
3. According to Table 3, the authors explore the effectiveness of different modules, but there are no experimental investigations into the effectiveness of the two specific modules, which I think is insufficient.
limitations: limitations: The authors explain the limitations of their work, and there is no negative societal impact.

Comments: comment: Thank you for the author's response. This rebuttal addresses my concerns. I decide to change my initial score to Weak accept.
Thanks for the author's response. I decide to raise my score to Weak Accept.
This rebuttal addresses most of my concerns. I decide to raise my score to weak accept.