Title: Normal-Abnormal Guided Generalist  Anomaly Detection
Venue: NeurIPS 2025
Accepted: yes
Keywords: Anomaly Detection

Normal-Abnormal Generalist Learning
Abstract: Generalist Anomaly Detection (GAD) aims to train a unified model on an original domain that can detect anomalies in new target domains. Previous GAD methods primarily use only normal samples as references, overlooking the valuable information contained in anomalous samples that are often available in real-world scenarios. To address this limitation, we propose a more practical approach: normal-abnormal-guided generalist anomaly detection, which leverages both normal and anomalous samples as references to guide anomaly detection across diverse domains. We introduce the Normal-Abnormal Generalist Learning (NAGL) framework, consisting of two key components: Residual Mining (RM) and Anomaly Feature Learning (AFL). RM extracts abnormal patterns from normal-abnormal reference residuals to establish transferable anomaly representations, while AFL adaptively learns anomaly features in query images through residual mapping to identify instance-aware anomalies. Our approach effectively utilizes both normal and anomalous references for more accurate and efficient cross-domain anomaly detection. Extensive experiments across multiple benchmarks demonstrate that our method significantly outperforms existing GAD approaches. This work represents the first to adopt a mixture of normal and abnormal samples as references in generalist anomaly detection. The code and datasets are available at https://github.com/JasonKyng/NAGL.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper aims to tackle generalist anomaly detection (GAD). Previous GAD methods primarily use only normal samples as references, overlooking the valuable information in anomalous samples. To address this, this paper proposes a more practical GAD method by innovatively incorporating few -shot anomalous samples as references to guide anomaly detection across diverse domains. The proposed Normal-Abnormal Generalist Learning framework consists of two parts: Residual Mining and Anomaly Feature Learning. The method can effectively utilize both normal and anomalous references for more accurate and practical cross-domain anomaly detection.\n\nSTRENGTHS AND WEAKNESSES:\n**[Strengths]**\n\n1. This paper focuses on a previously overlooked issue in GAD method: only considering to use few-shot normal samples as references without considering to effectively utilize information from few-shot abnormal samples. This is interesting and very valuable for practical usage. This paper proposes a good GAD method that can also effectively utilize abnormal samples when directly used in unseen product classes.\n\n2. The proposed method is simple but effective. The main innovative module in this paper is the RM-AFL module, the whole module is simply based on cross-attention to capture normal-abnormal differences in residual space. The finally learned anomaly proxies can represent the anomalous patterns in the query image, thus can highlight anomalous regions by further feature comparison.\n\n**[Major Weaknesses]**\n\n1. I think the anomaly classification in the title is not accurate because the model in the paper cannot generate classification labels for anomalous regions. Anomaly classification requires further classification of anomaly types based on detection, and the method in the paper only implements anomaly detection.\n\n2. In Introduction, the first point in contribution conclusion is not reasonable. “We propose a new generalist anomaly detection task and corresponding dataset”. Generalist anomaly detection has been proposed in previous papers, and this paper is only a further study. This paper still uses the existing datasets, only divides the data into new settings, and does not propose a new dataset. The main difference between this paper and previous papers is the consideration of using both normal and abnormal samples as references.\n\n3. Although the proposed method is simple and easy to follow, I think one main issue about the paper is that the method lacks corresponding explanations. Why the key and value in the first cross-attention are $\\mathcal{F}^a$ and $Res(\\mathcal{F}^a, \\mathcal{F}^n)$? Why the key and value in the second cross-attention are $Res(\\mathcal{F}^q, \\mathcal{F}^n)$ and $\\mathcal{F}^q$? Is this order necessary? What would happen if the order is swapped or different key and value are used? What do residual proxies and anomaly proxies represent? Do residual proxies represent abnormal patterns of reference abnormal samples in residual space? Do anomaly proxies represent abnormal patterns of the query sample in vision space? Why do we need attention in the residual space first? Can't we directly extract anomaly proxies from $\\mathcal{F}^q$ in the vision space?\n\n4. In Eq(5), the $\\mathcal{M}^{\\prime}$ is based ground-truth mask $\\mathcal{M}^a$. However, the ground-truth mask during testing is not known in advance. How is the Residual Mining part implemented during testing (how is the $\\mathcal{M}^\\prime$ part handled)?\n\n5. About implementation details. Is the RM-AFL module implemented with one layer? Or the RM-AFL module is implemented by stacking multiple layers. This critical detail is missing.\n\n6. About experiments. Why does PromptAD only appear in $N^1$ setting? Why only ResAD is reproduced? Is the ResAD reproduced based on the backbone (ViT-S) and 448x448 resolution used in your method (From appendix D, it seems that ResAD is reproduced based on wide-resnet50)? If for fair comparison, other methods should also be reproduced based on ViT-S and 448x448 resolution. You may need to reproduce ResAD based on the backbone used in your method and 448x448 resolution for more reasonable comparison. Also in Tab.2, the results of other methods are from ResAD paper, but why does ResAD need to be specifically reproduced. Additionally, the experiments in main text are not sufficient, you should conduct more cross-dataset experiments to validate your method (I see experiment results on MVTec3D, MVTecLOCO, BTAD, MPDD, it’s better to reorganize your paper to include these experiment comparison in main text).\n\n7. Strictly, your method does not just use an abnormal sample, as you randomly select a sample from each anomaly type. For a product class, there may be many anomaly types (e.g. Mvtec dataset), and the actual number of abnormal samples used may be even more than the normal samples. The more critical issue is that in practical use, we do not know the anomaly type in the query sample. In this case, how should we choose abnormal samples and which anomaly type should we choose from. You need to discuss how much influence it would have on the results if the anomaly type in the abnormal reference sample is different from the type in the query sample. And what influence does it have if multiple different anomaly types of abnormal reference samples are given to a query sample.\n\n**[Minor Weaknesses]**\n \nTypos: In line 118, $r^a$ misses the subscript k. In line 145, the shape of $\\mathcal{M}^a$ should be $\\mathbb{R}^{K_2L}$.\n\nI think the good explanations for Weakness 3 and influence about anomaly type in Weakness 7 are important. Weakness 4 and 6 also should be well addressed. Overall, I think this paper considers an interesting and very valuable task, and proposes a simple but effective method. The entire paper is easy to follow. However, due to the aforementioned weaknesses, I am currently unable to provide an accept. If my concerns can be effectively addressed, I am willing to raise my score.\n\nQUESTIONS:\nPlease see Strengths And Weaknesses.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-20"}

comments: SUMMARY:
This paper focuses on generalist anomaly detection that can detect anomalies in new target domains. The proposed method leverages both normal and anomalous samples as references to guide anomaly detection across diverse domains. Experiments are conducted on several popular datasets.

STRENGTHS AND WEAKNESSES:
1. The paper lacks important citations. In few-shot anomaly detection, the utilization of limited anomalous data has been extensively studied [a,b,c]. The authors should adequately cite these papers and compare their method with them. In the current experiments, none of the competing methods use anomalous data, while the proposed method requires it. This comparison is unfair.
2. The authors define an original data domain and a target domain. However, both data domains here consist of industrial defect detection data without fundamental differences. The authors should refer to MVFA [a] and adopt medical imaging data as the target domain to demonstrate the generalization capability of the proposed method.

[a] Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Images. CVPR 2024.
[b] Catching both gray and black swans: Open-set supervised anomaly detection. CVPR 2022.
[c] Explicit boundary guided semi-pushpull contrastive learning for supervised anomaly detection. CVPR 2023.

QUESTIONS:
See the weakness.

LIMITATIONS:
See the weakness.
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 2
DATE: 2025-06-30

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the problem of Generalist Anomaly Detection (GAD), detecting anomalies in new target domains with models trained on a source domain. To this end, the authors propose a new framework, NAGL, including two key components, Residual Mining (RM) and Anomaly Feature Learning (AFL). Given an abnormal reference, RM obtains residuals between normal and abnormal samples to learn abnormal patterns. AFL learns abnormal features of a query image based on the learned abnormal patterns. NAGL compares abnormal patterns between the query and reference images to detect anomalies. The authors demonstrate the effectiveness of NAGL on standard benchmarks, and show an expensive analysis.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n(+) NAGL incorporates abnormal references along with the normal ones, enabling effective anomaly detection across diverse domains. \n\n(+) The proposed method is highly practical due to its lightweight design and achieves faster training and inference than the previous works [51, 62].\n\n(+) The proposed method shows strong cross-domain generalization performance (e.g., MVTecAD → VisA, MVTecAD → BraTS). \n\n\n**Weaknesses**\n\n(-) The proposed method relies heavily on abnormal references. I am thinking that this might limit the applicability in certain real-world scenarios, where only normal data is available, making it less practical than the works of [51, 62].\n\n(-) The authors set 𝐾_1 ≥ 𝐾_2 assuming a real-world scenario where abnormal samples are scarce. The inference setting on MVTecAD, however, seems to contradict the assumption. Specifically, if K_1=1 and K_2=1, the number of normal and abnormal references are 15 and 73, respectively. Please clarify this. \n\n(-) The authors demonstrate that the proposed method generalizes well in cross-domain settings, such as the MVTecAD ⟶ BraTS scenario. However, the source domain is limited to MVTecAD, which contains a variety of abnormal references. It would be helpful if the authors could provide more results using other source domains.\n\n(-) Please clarify the experimental settings (iii) in Table 3. I am wondering why (iii) shows inferior performances than (i) although (iii) exploits abnormal samples.\n\n(-) The author shows in Table 3 the effectiveness of the proposed method that leverages abnormal references for GAD. However, it seems that NAGL adopts different backbone ones from [52, 61]. It would be more convincing if the authors provide additional results using the same backbone in [52, 61]. This would help to establish the validity of the proposed method more convincingly.\n\n(-) There are some typos.\nL82: augmentation-based -> and augmentation-based\nL148: pervious -> previous\nCaption of Figure1: cause -> causes\nCaption of Table 3: represent -> represents, merge -> merges\n\nQUESTIONS:\nPlease refer to the strengths and weaknesses section for the details.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel generalist anomaly detection (GDA) framework, termed Normal-Abnormal Generalist Learning (NAGL), which leverages both normal and abnormal reference samples during training and inference to reflect more realistic application scenarios where limited anomaly examples may be available. The framework integrates two key components: Residual Mining (RM), which extracts transferable anomaly representations from the residuals between normal and abnormal references, and Anomaly Feature Learning (AFL), which identifies discriminative anomaly features in query images by comparing them against these residual proxies. Designed for cross-domain generalization, NAGL achieves strong performance across diverse datasets such as MVTecAD, VisA, and BraTS. Extensive experiments, including ablation studies and efficiency analysis, demonstrate its effectiveness and computational efficiency, highlighting its potential for practical anomaly detection applications.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: \n1. The paper is the first to propose GAD with both normal and abnormal reference samples, aligning better with practical settings where limited abnormal data may exist.\n2. The RM-AFL mechanism is well-motivated and shown to resolve issues of false activations when using abnormal references naively.\n\nWeaknesses\n1. The selection of abnormal references is random (per defect type), but its potential impact on performance variability is not well studied or controlled.\n2. The RM and AFL modules, while empirically strong, could benefit from clearer theoretical motivation or formal insight into their robustness or transferability.\n3. The method assumes access to both normal and abnormal references at inference time, which may not always hold in fully open-world applications.\n\nQUESTIONS:\n1. How does the performance vary when abnormal references are from different defect types than the query (e.g., inter-type transfer)? Is there any observed instability or degradation?\n\n2. While BraTS provides a cross-domain test, have you evaluated zero-shot performance on unseen object classes in the same dataset? This would help assess the method’s generalization depth.\n\n3. AFL uses residual attention to avoid false activations. How does this behave in cluttered scenes or images with complex textures? Would AFL overfit to known anomaly shapes?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}