Title: CSPCL: Category Semantic Prior Contrastive Learning for Deformable DETR-Based Prohibited Item Detectors
Venue: NeurIPS 2025
Accepted: yes
Keywords: object detection

X-ray inspection

detection transformer
Abstract: Prohibited item detection based on X-ray images is one of the most effective security inspection methods. However, the foreground-background feature coupling caused by the overlapping phenomenon specific to X-ray images makes general detectors designed for natural images perform poorly. To address this issue, we propose a Category Semantic Prior Contrastive Learning (CSPCL) mechanism, which aligns the class prototypes perceived by the classifier with the content queries to correct and supplement the missing semantic information responsible for classification, thereby enhancing the model sensitivity to foreground features. To achieve this alignment, we design a specific contrastive loss, CSP loss, which comprises the Intra-Class Truncated Attraction (ITA) loss and the Inter-Class Adaptive Repulsion (IAR) loss, and outperforms classic contrastive losses. Specifically, the ITA loss leverages class prototypes to attract intra-class content queries and preserves essential intra-class diversity via a gradient truncation function. The IAR loss employs class prototypes to adaptively repel inter-class content queries, with the repulsion strength scaled by prototype-prototype similarity, thereby improving inter-class discriminability, especially among similar categories. CSPCL is general and can be easily integrated into Deformable DETR-based models. Extensive experiments on the PIXray, OPIXray, PIDray, and CLCXray datasets demonstrate that CSPCL significantly enhances the performance of various state-of-the-art models without increasing inference complexity. The code is publicly available at https://github.com/Limingyuan001/CSPCL.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a plug-and-play contrastive learning mechanism, named Category Semantic Prior Contrastive Learning (CSPCL), to address the feature coupling issue caused by object overlap in X-ray images. CSPCL ingeniously leverages category semantic prior knowledge embedded in the classification head to guide and enhance content queries, enabling Deformable-DETR-based models to achieve stronger foreground feature perception. To support this guidance, the paper introduces a novel contrastive learning loss function, CSP loss, which consists of ITA loss for positive pairs within the same class and IAR loss for negative pairs across different classes. This design decomposes the contrastive learning problem into two sub-problems—inter-class repulsion and intra-class attraction—allowing for more targeted optimization aligned with specific tasks. CSP loss demonstrates superior performance compared to classical N-pair loss and InfoNCE loss. Extensive evaluations on four prohibited item detection datasets show that CSPCL can improve the detection accuracy of Deformable DETR-based models without introducing additional computational overhead and further establishes a new state-of-the-art benchmark for prohibited item detection models.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths：\n1.The idea proposed in this paper is highly novel. Instead of focusing on typical modifications of label assignment strategies or network structures in mainstream DETR-like models, the authors introduce a simple and efficient contrastive learning strategy that guides content queries to learn category semantic features, thereby improving detection accuracy. This is particularly effective for addressing the feature coupling issue between foreground and background caused by the unique overlapping phenomenon in X-ray images.\n2.CSPCL is a plug-and-play method that can be broadly applied to Deformable-DETR-based models. It improves detection accuracy without increasing computational complexity, which is a significant advantage in the current landscape that favors large models and scaling laws.\n3.The CSP loss is simple, intuitive, and effective. It decomposes the contrastive learning problem into two sub-problems: inter-class repulsion and intra-class attraction. This targeted design, tailored to the specific task, outperforms classical losses such as N-pair loss and InfoNCE loss.\n4.CSPCL demonstrates strong generalizability, achieving effective results across different models, backbones, and datasets. This contribution is commendable, as it represents a more fundamental and insightful approach rather than a simple model-specific modification.\n5.The paper is well-written, clearly structured, and easy to follow. The method is explained step by step, the experiments are comprehensive and rigorous, and the figures clearly illustrate how the proposed augmentation strategy is applied during training.\nWeaknesses：\n(1) The code for the paper does not appear to be fully open-sourced; instead, it only provides the implementation of the main innovations.\n(2) In Tables 3 and 4, the paper should supplement experimental results for AO-DETR (ResNet-50) + CSPCL to comprehensively demonstrate whether smaller-scale models can also achieve SOTA results with the CSPCL mechanism.\n(3) The Non-Maximum Suppression (NMS) post-processing step has not been cited.\n\nQUESTIONS:\n1.The authors should consider adding additional experiments, for example, to investigate whether CSPCL can help smaller models, such as AO-DETR (ResNet-50), surpass the performance of SOTA models when using CSPCL.\n2.Based on the paper and code, the current design of CSP loss only works for cases where the number of content queries is an integer multiple. The authors should consider how category-specific content queries should be allocated in scenarios where the number of content queries is not an integer multiple of the number of categories, such as with 100 content queries and 15 categories. If this aspect can be improved, CSPCL would be able to adapt to fine-tuning tasks based on Deformable-DETR models for various numbers of content queries, further enhancing the significance of this work.\n3.For better readability, it would be helpful to clarify the meaning of all variables. For example, in Lines 151-160, the dimensions N and C are not explained. Although this doesn’t affect understanding, it does impact reading fluency.\n4.The authors should carefully check the completeness of the references to ensure that all relevant citations are included. This will help readers better understand the content of the paper and the inspirations behind it.\n\nLIMITATIONS:\nYes. It's worth noting that, in natural-light images, densely appearing objects often lead to occlusion, resulting in the loss of foreground object features, while the kind of foreground-background feature coupling problem always found in X-ray images is rare. Therefore, the inability to demonstrate CSPCL’s effectiveness on the COCO dataset actually highlights the strength of the proposed method in addressing the specific challenge of overlapping phenomena.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-06-09"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a Category Semantic Prior Contrastive Learning (CSPCL) mechanism to improve prohibited item detection in Deformable DETR-based models, addressing the challenge of object overlapping in X-ray images. The core idea is to remedy the issue of content queries losing category-specific semantic information during the decoder's update process. CSPCL aligns content queries with class prototypes derived from the classifier's weights. This is achieved through a devised CSP loss, featuring two key components: an Inter-Class Adaptive Repulsion (IAR) loss that pushes dissimilar classes apart with a force proportional to their feature similarity, and an Intra-Class Truncated Attraction (ITA) loss that prevents feature homogenization by stopping attraction at a set threshold. The authors demonstrate that this plug-and-play module enhances performance across four datasets without increasing inference complexity.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n1. The paper is well-motivated, clearly identifying a plausible weakness in Deformable DETR models to provide a justification for its proposed method.\n2. The paper proposes a practical method, as the CSPCL module is plug-and-play and adds no computational overhead during inference.\n\nWeaknesses\n1. The paper's originality seems limited. This paper appears to be an incremental improvement on existing contrastive learning works, which has been widely explored in various object detection architectures, such as [1] and [2]. The idea of inter-class and intra-class contrasts have also been adopted in previous studies, such as [3] and [4]. Specifically, there also exists a prototype-based contrastive learning work in X-ray prohibited items detection [5]. The proposed adaptive loss, while thoughtful, refines this existing foundation rather than introducing a new paradigm. Therefore, it’s suggested that the author should further address the above originality concerns.\n\n2. The paper lacks a direct experimental comparison with other X-ray detection methods. In table 3, none of the anti-overlapping methods (e.g. [6][7]) mentioned in related works are compared. Additionally, the most relevant competitors MMCL[8] that uses contrastive learning on DETR-based models for X-ray images is neglected. The baseline model AO-DETR is not compared, either. \nTherefore, it’s suggested that the author should conduct head-to-head comparisons with more X-ray detectors to empirically validate the claimed benefits of the proposed CSPCL.\n\n3. The SOTA comparison in Table 3 is difficult to interpret fairly due to an inconsistent baseline. The presented C-AO-DETR uses a Swin-L backbone, which is more powerful than the ResNet/ResNeXt backbones used by most competing methods listed in the table. This makes it difficult to attribute performance gains to the CSPCL module itself rather than to the superior capacity of its baseline architecture. \n\n4. The proposed method appears to have limited applicability beyond Transformer-based architectures. Similar to techniques that are exclusively applicable to CNN-based detectors, this work seems constrained to Transformer architectures. Moreover, the results in Table 3 show that the performance of Transformer-based detectors does not surpass the CNN-based Mask R-CNN. Therefore, it is recommended that the authors provide further explanation of the practical value of CSPCL.\n\n5. The paper does not clearly explain how the proposed method solves the object overlapping problem. The argument relies on indirect reasoning and a single qualitative example, rather than a quantitative analysis that directly measures performance on overlapping objects.\n\n6. The paper lacks a sufficient discussion of its key design choices. This includes the selection and sensitivity analysis of new hyperparameters , as well as the justification for which decoder layer to apply the proposed loss.\n\n[1]Wei, Fangyun, et al. \"Aligning pretraining for detection via object-level contrastive learning.\" Advances in Neural Information Processing Systems 34 (2021): 22682-22694. Li M, Jia T, Lu H, et al. \n\n[2]Xie, Enze, et al. \"Detco: Unsupervised contrastive learning for object detection.\" Proceedings of the IEEE/CVF international conference on computer vision. 2021.\n\n[3]Li, Wenbin, et al. \"Modeling inter-class and intra-class constraints in novel class discovery.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2023.\n\n[4]Chen, Hao, Benoit Lagadec, and Francois Bremond. \"Ice: Inter-instance contrastive encoding for unsupervised person re-identification.\" Proceedings of the IEEE/CVF international conference on computer vision. 2021.\n\n[5]Tao, Renshuai, et al. \"Few-shot x-ray prohibited item detection: A benchmark and weak-feature enhancement network.\" Proceedings of the 30th ACM international conference on multimedia. 2022.\n\n[6]Y. Wei, R. Tao, Z. Wu, Y. Ma, L. Zhang, and X. Liu, “Occluded prohibited items detection: An x-ray security inspection benchmark and de-occlusion attention module,” in Proceedings of the 28th ACM International Conference on Multimedia, 2020, pp. 138–146.\n\n[7]C. Zhao, L. Zhu, S. Dou, W. Deng, and L. Wang, “Detecting overlapped objects in x-ray security imagery by a label-aware mechanism,” IEEE Transactions on Information Forensics and Security, vol. 17, pp. 998–1009, 2022\n\n[8]MMCL: Boosting Deformable DETR-Based Detectors with Multi-Class Min-Margin Contrastive Learning for Superior Prohibited Item Detection[J]. arXiv preprint arXiv:2406.03176, 2024.\n\nQUESTIONS:\n1.The proposed method defines class prototypes using the classifier's weights, which are simultaneously updated by both the classification loss and the new CSP loss. This raises a question about the stability of the prototypes. It is suggested that the authors discuss if this dual-update mechanism could cause the prototypes to become unstable, or merely reflect an average of the content queries rather than serving as a fixed semantic prior.\n\n2.The ITA loss is designed with a gradient truncation mechanism to preserve intra-class variance, with the stated goal of handling variations in object appearance. However, the direct benefit of this preserved variance is not empirically demonstrated. It is suggested that the authors provide a more direct analysis linking this specific design choice to a measurable improvement in performance, for example, on categories with high fine-grained diversity.\n\n3.The IAR loss utilizes an exponential function to scale the repulsion strength based on inter-class prototype similarity. The paper does not provide a justification for this specific mathematical choice. It is suggested that the authors explain why an exponential function was chosen over simpler alternatives, such as a linear scaling factor.\n\n4.The paper is ambiguous regarding which decoder layers the CSP loss is applied to, mentioning \"layer 1\" in one instance but a \"set of target layers\" () in the algorithm description. It is suggested that the authors clarify this crucial implementation detail and provide a clear justification for their choice.\n\n5.The core idea of using classifier weights as learnable class prototypes has been explored in prior literature such as ADAM[1]. It is suggested that the authors more clearly articulate the novelty of their approach in comparison to these existing works. A more explicit discussion of how the proposed CSP loss, particularly its adaptive and truncated nature, provides a distinct advantage over other prototype-based learning schemes would help to better situate the paper's contribution .\n\n[1]Zhou D W, Cai Z W, Ye H J, et al. Revisiting class-incremental learning with pre-trained models: Generalizability and adaptivity are all you need[J]. International Journal of Computer Vision, 2025, 133(3): 1012-1032.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-18"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes CSPCL, a Category Semantic Prior Contrastive Learning mechanism designed to enhance Deformable DETR-based object detectors for X-ray security imagery, particularly for detecting prohibited items. The method introduces a novel CSP loss, composed of Intra-Class Truncated Attraction (ITA) and Inter-Class Adaptive Repulsion (IAR), to align content queries with class prototypes (classifier weights), thereby correcting and supplementing the semantic information degraded by overlapping features. CSPCL is plug-and-play, architecture-agnostic, and demonstrates consistent performance gains across four datasets and multiple backbone models.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. This paper focuses on a critical issue in the field of X-ray detection.\n2. The proposed plug-and-play CSPCL outperforms classic contrastive learning methods—such as N-pair and InfoNCE losses—in enhancing anti-overlapping detection capability.\n3. Experiments demonstrate the effectiveness of the proposed CSPCL for X-ray image detection.\n\nWeaknesses:\n1. I observe that the backbone of proposed  C-AO-DETR is Swin-L, which is inherently more powerful than those used in the other baselines; moreover, the input image sizes vary across methods, making the comparison unfair\n2. The paper only compares against relatively old contrastive losses (InfoNCE, N-pair), none of which were designed specifically to improve anti-overlapping detection. The authors should include more recent loss functions to further validate their approach\n3. The models listed in Table 2 are quite similar; the authors should evaluate a broader range of more recent, diverse architectures to strengthen their analysis.\n\nQUESTIONS:\n1. Does the model trained with this loss generalize well to other downstream tasks, such as zero-shot or open-set detection?\n\n2. Could you provide additional visual results—for example, ten more examples—similar to those shown in Figure 5?\n\nLIMITATIONS:\nYes, the authors discuss limitations in supplementary material.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper focuses on improving prohibited item detection in X-ray images, where the main challenge lies in the overlapping of objects that causes foreground-background feature coupling—something general object detectors struggle with. To tackle this, the authors propose a Category Semantic Prior Contrastive Learning (CSPCL) framework, which enhances the model’s ability to focus on foreground features by aligning class prototypes with content queries during training. The method is evaluated on multiple X-ray datasets (PIXray, OPIXray, PIDray, and CLCXray), showing consistent improvements over existing state-of-the-art methods.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n1. A plug-and-play module that can be integrated into Deformable DETR-based detectors without increasing inference complexity.\n2. Better performance than baseline methods.\n\nWeaknesses\n1. Why aligning class prototypes with content queries can correct and supplement the missing semantic information is not immediately clear. The connection between the two is not direct, and the authors need to provide further explanation.\n2. Compared to existing DETR-based methods, what are the specific advantages of the approach proposed in this paper? In addition, the paper should also clarify the advantages of DETR-based methods over non-DETR-based methods.\n3. The performance of AO-DETR on PIDray in Table 3 is missing. This result is crucial for evaluating the effectiveness of the proposed method.\n4. The settings of C-AO-DETR (ours) and the comparison methods are different, which raises the question of whether the performance improvement comes from a better backbone or other configuration choices, rather than from the proposed method itself.\n5. The design of inter-class and intra-class losses is not novel, as it is conceptually very similar to approaches like triplet loss and contrastive loss. This limits the overall novelty of the paper.\n\nQUESTIONS:\nPlease see Weaknesses.\n\nLIMITATIONS:\nLack of sufficient (theoretical) explanation:\nThe paper lacks a clear theoretical justification or intuitive explanation for the core claim that aligning class prototypes with content queries can correct and supplement missing semantic information. The logical connection between these el（ements is not fully established.\n\nUnfair experimental comparisons:\nThere are differences in experimental settings between C-AO-DETR and the comparison methods (e.g., DN-DETR), such as using a stronger backbone or more favorable training configurations. This makes it difficult to determine whether the performance improvements truly come from the proposed method itself.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-15"}