Normal-Abnormal Guided Generalist  Anomaly Detection
Authors: Yuexin Wang
Xiaolei Wang
Yizheng Gong
Jimin XIAO
Venue: NeurIPS 2025 poster ()
Keywords: Anomaly Detection
Normal-Abnormal Generalist Learning
Primary area: applications

Abstract: Generalist Anomaly Detection (GAD) aims to train a unified model on an original domain that can detect anomalies in new target domains. Previous GAD methods primarily use only normal samples as references, overlooking the valuable information contained in anomalous samples that are often available in real-world scenarios. To address this limitation, we propose a more practical approach: normal-abnormal-guided generalist anomaly detection, which leverages both normal and anomalous samples as references to guide anomaly detection across diverse domains. We introduce the Normal-Abnormal Generalist Learning (NAGL) framework, consisting of two key components: Residual Mining (RM) and Anomaly Feature Learning (AFL). RM extracts abnormal patterns from normal-abnormal reference residuals to establish transferable anomaly representations, while AFL adaptively learns anomaly features in query images through residual mapping to identify instance-aware anomalies. Our approach effectively utilizes both normal and anomalous references for more accurate and efficient cross-domain anomaly detection. Extensive experiments across multiple benchmarks demonstrate that our method significantly outperforms existing GAD approaches. This work represents the first to adopt a mixture of normal and abnormal samples as references in generalist anomaly detection. The code and datasets are available at https://github.com/JasonKyng/NAGL.

Normal-Abnormal Guided Generalist Anomaly Detection
Yuexin Wang Xiaolei Wang k.2 Yizheng Gong 1,2, Jimin Xiao
University Xi' an University of Liverpool
Abstract
Generalist Anomaly Detection (GAD) aims train a unified model on original domain that can detect anomalies in new target domains. Previous GAD methods primarily use only normal samples as references, overlooking the valuable informa- tion contained in anomalous samples that are often available in real- world scenarios. To address this limitation, we propose a more practical approach: normal-abnormal- guided generalist anomaly detection, which leverages both normal and anomalous samples as references to guide anomaly detection across diverse domains. We intro- duce the Normal-Abnorme Generalist Learning (NAGL) framework, consisting of two key components: Residual Mining (RM) and Anomaly Feature Learning (AFL). RM extracts abnormal patterns from Normal-abnorman reference residuals to estab- lish transferable anomaly representations, while AFL adaptively learns anomaly features in query images through residual mapping identify instance- anomalies. Our approach effectively utilizes both normal and anomalous refer- ences for more accurate and efficient cross- cross-domain anomaly detection. Extensive experiments across multiple benchmarks demonstrate that our method significantly outperforms existing GAD approaches. This work represents the first to adopt a mix- ture of normal and abnormal samples as references in generalist anomaly detection. The code and datasets are available atihtpss: //githubb...t... tigt...t...... cww cww/Jasonnygg///a NAGL
Introduction
Visual Anomaly Detection (AD) 13 30 63
5
14 67 56
59
17 41 6 plays crucial role in 18,
20 Its primary objectives are to classify images as normal or anomalous and to localize anomalies within those images. Traditional focus on training and testing model on single domain, without considering how detection capabilities might transfer to different target domain. However, many real-world AD scenarios prohibit training on the target domain due to data scarcity and privacy issues, making it difficult to achieve the desired outcome in that target domain. To address the challenge, InCTRL [68] and ResAD 660] propose Generalist Anomaly Detection (GAD) that aims to train a unified model the original domain while enabling AD on the target domain. As shown in Fig. [T, the GAD framework adopts a meta-learning strategy. This strategy trains the model in the original domain to localize the anomalous regions of query image by referring to limited number of normal references. Subsequently, the learned ability is transferred to the new target domain.
industrial quality inspection 48
69 25 and medical diagnosis [4
AD methods 31 15 53
Although previous GAD methods 688 60 have made substantial progress, they are not yet practical for real-world applications. Models trained exclusively on normal samples often lack the discriminative power to reliably distinguish anomalies [14] 61]. However, real-world scenarios often provide a small number of anomalous samples (e.g., defective parts or diagnosed discase cases). These anomalous
"Equal Contribution.
Coneeponning Author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Training on Original Domaln
Inference en Targel Domain
Query image W.
Anemoloss Fontr
Our Approach
Activation Map
Activation Map
Query
Normal Reference
Model
Query 1
Normal Reference
Model
Result
Query
Normal Reference
Query
Reference
Result
(a) Vanilla Generalist Anomaly Detection Setting
(c) False Activation Anomallou Features
Training on Original Domain
Inference Target Domains
AUROC (%)
Ours
Query
Normal Abrommal
N Reference
Model
Query
Nompal Abnormal Referenee
Model
Result
ResAD
LnCTRL
Quayy
Annormal
Normal Abtormal Reference
Result
12 1a IS
Reference
Inference Speed (FPS) (d) Efficiency Analysis
Normal-Abnormal
Generalist Anomaly
Setting
Figure I: Overview of existing and proposed GAD paradigms. (a) Vanilla GAD only adopts few normal samples as references. (b) Our approach combines normal and abnormal references to enhance detection. (c) Direct application of the KNN-based method detection task causes to false activations (middle), while our approach eliminates them (right). (d) Comparisons of different methods in terms of AUROC sample (y-axis), inference speed (x-axis), and model size (circle radius). Our approach achieves the highest AUROC sample metric for anomaly detection while being 2x faster than ResAD and 14x faster than InCTRL.
normal-abnormal guided
samples contain valuable information on anomaly characteristics that could be leveraged to improve detection. Given this context, we propose a more practical and effective approach: normal-abnormal guided generalist anomaly detection (illustrated in Fig. [Tp). This approach leverages both normal and anomalous samples as references, guiding the model to detect anomalies across diverse domains. The core of this approach learning the relationships between a query and these references in the original domain, and then applying this learned understanding to the target domain.
Previous methods 668 60) leverage residuals between queries and normal references ensure transferability, but these approaches are specifically tailored for scenarios where few-shot normal samples are provided, which cannot be adapted the normal-abnormal guided paradigm. When provided with a reference set with both normal and abnormal samples, KNN-based approaches solution, where sample regions far away from normal references and close to abnormal references are considered as abnormal. However, these KNN-based approaches are typically training-fres free, lacking the adaptability offered by data-driven learning. Additionally, we experimentally find that the KNN-based method suffers from false activation problems (see Fig.[p> To address these limitations, we propose a Normal-Abnorma Generalist Learning (NAGL) framework, which utilizes the residual features to model the differences between normal and anomalous references. The proposed method achieves superior detection performance at faster speeds (see Fig. It), demonstrating its significant potential for practical applications.
[2] can serve
The core of our NAGL framework consists two parts: Residual Mining (RM) from normal-abnormal references and Anomaly Feature Learning (AFL) for query images through residual mapping. To fully explore abnormal reference patterns and maintain transferability, RM leverages normal-abnormal residuals to learn abnormal reference patterns through designed attention operation, obtaining residual proxies. AFL adaptively learns abnormal features of a query image by comparing residual proxies with residuals between the query feature and normal references, obtaining anomaly proxies. Finally, anomaly localization results are acquired by similarity computation between the query feature and anomaly proxies. Our NAGL only relies on residual training to adaptively capture the similarities and differences between query and normal-abnormal reference samples, so can transferred between different domains.

---

Leveraging RM and AFL, our proposed NAGL framework effectively achieves -ormallabnormal guided generalist anomaly detection. The main contributions of this work are summarized as follows:
We propose a different generalist anomaly detection task and a corresponding dataset split. This task is the first to adopt a mixture of normal and abnormal samples references.
We propose novel Normal-Abnorma Generalist Learning framework to effectively lever- age abnormal reference in GAD that adapts normal-abnonna reference residuals to mine potential anomalies in the query.
Extensive experiments across multiple anomaly detection benchmarks demonstrate that our method significantly outperforms existing GAD approaches.
2 Related Work
2.1 Anomaly Detection
Artificial intelligence techniques based on deep learning have been widely applied 227
38 57
with anomaly detection (AD) being a significant application. AD can be divided into various tasks
according to real-world requirements, e.g.. unsupervised AD [55
54
69
37
64
49
few-shot
AD (FSAD) 21
15
28
42 50 33 zero-shot AD (ZSAD) [23
65 7. noisy AD 8 (26.. 3D AD
Among these tasks, unsupervised AD generally adopts one-class classification paradigm to train the detection model, i.e., the model is only trained on normal samples and can detect unseen abnormal patterns during the inference phase. Existing unsupervised methods can be divided into three main categories: reconstruction-bent based [5]] 40 19 feature-
16 I
36 66 and open-set AD I4 67
embedding-based [44 48 29 58), and augmentation-basee [62 64 34 methods. PatchCore [48],
simple feature-embedding based method, firstly constructs a memory bank of normal embedding features, then introduces a nearest neighbour search find several nearest neighbours for each test embedding feature, and computes the distance between the test embedding feature and its neighbours as an anomaly score. PatchCore can also be extended to a few-shot AD task well, where the used memory bank only requires a few normal reference samples to construct. Based on this work [48), FastRecon 15 leverages ridge regression on normal features to quickly reconstruct test features. However, these methods only have a testing phase and detect anomalies on single domain of data, lacking transferability. Therefore, some 7SAD (65] and GAD [60] 68] methods have begun to study cross-domain detection. And [43] also proposes a general few-shot defect classification framework that addresses real-world applications of defect type identification. Building upon the GAD task, our proposed normal-abnormal guided generalist AD framework addresses cross-domain anomaly detection by effectively utilizing abnormal samples from the original domain to improve performance.
2.2 Generalist Anomaly Detection
FSAD is designed to identify anomalies using only limited number of normal samples from target datasets. Existing FSAD methods can be divided into seening-free-baseee 48 15
and vhea--carning- based methods [2 68 60) Meta-learning- based methods focus on generalizing the detection of the model the new target domain. RegAD 21 trains registration network using samples from seen categories, which can register samples from unseen categories and achieve cross-category anomaly detection. However, RegAD remains dependent on domain relevance between training and testing data. To achieve dagnoie anomaly detection, InCTRL 688 firstly proposes a generalist anomaly detection (GAD) task and utilizes residual distance to discriminate anomalies. Subsequently, ResAD (60) applies residual features to eliminate domain dependencies to implement the GAD task. The reference set used by the vanilla GAD task only contains normal samples. However, in practical scenarios, small number of abnormal samples also be obtained. Therefore, we propose the normal-abnormal guided GAD task, where the reference set combines normal and abnormal samples.
3 Methodology
3.1 Normal-Abnormal Guided GAD Task
In the normal-abnormal guided GAD task, we focus on training a unified model on the original domain and detecting anomalies in new target domain using both normal and abnormal samples
3

---

Nemal Refenmox
Frezen
Trarabie
C= +
Sinz
RM-AFL Module
Normal-Guitted Detection
Asonull Proxies
R
S
S
gr
Fivion Spare
N'W Soarch
Merga
AAFZ
baput Query
F
Residoed Mapiic
Res Fe.
Ficoder
Vision
Ghoormak-Guileed Detection
Residual
p
Spane
(K-)
Res (.
S
ARM
Abruerud Reforence
F
Fivion
Reridial
F
Space
R
Learnable Prixics
P
RM-AFI
Attestion
Mork Deverommpl
Figure 2: Overview of our proposed NAGL framework. Given test image and its corresponding reference images (normal and abnormal), features are extracted through pre-trained backbone net- work. The extracted normal features guide the generation of a normal-guided score map. Meanwhile, abnormal features are processed through the RM-AFL module to produce an abnormal-guided score map. This module implements a transformation process: from learnable proxies in vision space to residual proxies capturing normal-abe aormaalabnormal differences in residual space, and finally to anomaly proxies vision space that highlight specific anomalous regions in the query image. The final anomaly score is computed by merging both normal and abnormal guided score maps.
as references. Formally, let D Dorigin {X, }} be the training dataset from original domain, where X' { consists of N normal and abnormal samples, ) = J is corresponding ground truth masks, and each normal sample equipped a zero mask. During training phase, we organize the data into many episodes, where each episode consists of reference set R (R", Ro} and
K; query input x from D Dorigin. The reference set R contains normal samples R [r²) [k= and abnormal samples Ra { k= K₁ and K2 denote the number of normal and abnormal reference samples, respectively. We train unified detection model on D Dorigin. During inference, the model is evaluated on the target domain dataset Dtarget containing unseen categories compared to Dorigin- The testing process is consistent with the training phase, where each test sample is equipped with normal- normal-abnormal reference set. our proposed cormal-abnormal guided GAD task, considering the scarcity of abnormal samples, we set K1 2 K2, making it highly applicable to real-world situations.
3.2 Overview of Proposed Method
Our proposed Normal-Abnorma Generalist Learning (NAGL) framework is designed to detect anomalies in the target domain through original domain training. As shown in Fig. [ given a query input and normal-abnormal references, we extract these features by applying pre-trained backbone network for further processing. After feature extraction, we directly leverage the Nearest Neighbor (NN) search between the query and normal references, obtaining an initial anomaly score map. Next, to apply an abnormal reference to improve the initial anomaly map, we propose an RM-AFL module. This module consists of two attention parts: Residual Mining (RM) from normal-abnormal references and Anomaly Feature Learning (AFL) for query images with residual mapping. RM relies on abborrlll residuals learn the representation for abnormal reference patterns, generating several residual proxies. AFL applies these proxies to learn abnormal areas in the query image and obtain anomaly proxies. Finally, we compute the cosine similarity between the query feature and anomaly proxies for anomaly localization results.
3.3 Feature Extraction
In this section, we describe the feature extraction process for an episode. For each episode, we have normal-abnormal reference set R = {R", } and input query image x E IR #' x3 x3 For input query and references, we follow the common practice of using a pre-trained backbone network o(-) to extract their features. Subsequently, we obtain query patch features F ( normal reference patch features JF
{f hel and abnormal reference patch features F {f. K.L where
JiJl

---

L h E. t is the number of patches in one feature map, h and w are length and width of feature map, and C' the channel dimension of each patch feature. For abnormal reference features, we downsample the anomaly mask to match the length of Fa, yielding M E RR224.
3,4 Normal-Guided Anomaly Score
In this section, apply query and normal reference features obtain initial anomaly localization result. Following the previous work [48]. we directly apply the nearest neighbor search between query features Ft and normal reference features F" generate an anomaly score map Sn E R'. Specifically, we apply the NN method to find the closest reference patch feature f. for each query patch feature / in normal reference patches F",
argmin (d(s), E"E
where d(:,)) is the cosine distance function and i E {1,2, ... ,). Therefore, each patch anomaly score is defined
S. = f(f
(2)
where S O indicates the most normal region and S indicates the most abnormal region.
3.5
Abnormal-Guided Anomaly Score
In this section, we focus on applying abnormal reference to improve the initial anomaly map. This section mainly consists of Residual Mining (RM) from normal-abnorma reference and Anomaly Feature Learning (AFL) for query images by residual mapping. The whole RM-AFL mainly relies on residual training to adaptively capture the similarities and differences between query and reference samples, so it can transferred between different domains.
3.5.1 Residual Mining from References
The RM module introduces learnable query proxy to adaptively capture abnormal patterns in anomalous references and utilizes normal-abnormal residuals to learn anomalous variations through a cross-attention mechanism. Specifically, for given abnormal reference features FF", we employ the NN method to identify the closest normal reference patch feature for each f in F and compute the residual between the two. For simplicity, denote this process as
Re = =" F.
(3)
where F. E IR K2LCC consists of fr corresponding to each f We then design an attention layer (ARM) that utilizes normal-abnormal residuals learn representations of abnormal variations, thereby generating residual proxies. We state Query, Key, and Value in the attention computation:
WWP, K. F V1 = WW"Res(F" FF),,
(4)
where P MxC is learnable proxies and randomly initialized, parameters W W and WY transform input features into query, key, and value vectors, M is a hyperparameter representing the number of P. Therefore, the sateetion-based RM mechanism can be described
P SA, Softtma
Q1KT
Vd + +M))
where P E MC is called residual proxies, (-1(-) denotes self-attention operation, and d is scaling factor [52] The attention mask M a(1 Mn) contains either zero or negative infinity values, where a is a large negative value (e.g. - -00). This mask ensures cross-attention focuses exclusively on abnormal regions. Residual proxies P learn abnormal reference patterns and apply residuals as Value represent variations for anomalies.
3.5.2 Anomaly Feature Learning for Query Images
In this section, AFL. focuses on applying obtained residual proxies to learn potential abnormal patterns in F obtaining anomaly proxies. The anomaly localization results acquired by similarity
5

---

computation between query patch features and anomaly proxies. This process can also be described by another attention layer (AAFI.) The Query. Key, Value of the attention module is designed as:
Q2 - W K2 = w Res(FF F"), V2 F2
(6)
where W W W are learnable parameters. Our proposed AFL can be written as:
Q22k
p SSAA (oottmax
V2
where (-2(-) denotes another self-attention module different from SA1(), P RR is called anomaly proxies. P captures abnormal patterns in FV by comparing reference residuals Res(F", FF) with the query-normal residuals (FF,F"))) P has the most distinguishable patch feature informa- tion Ft. Therefore, we calculate the mean of similarity between each anomaly proxy Pm and the query patch features F an anomaly-gume guiddd anomaly score map S., Each patch anomaly score is
1 Σ S.¹ = M
(8)
where S. RRL, Finally, we merge the initial normal- oormal-guided score map and abnormal-guie guided score map to acquire final anomaly localization results:
S S + S..
(9)
calculate the image-level anomaly score by averaging the top 1% highest values in the score map S E R4, Specifically, we denote this operation as To.01 (S), where To.01 (S) represents the average of the 1% highest scores in S.
Following 12]
3.6 Training on Original Domain
The whole NAGL framework trained on original domain data. Our optimization objective that the predicted map S should be consistent with the query ground truth mask M Therefore, we apply Focal loss 35 and Dice loss 32 to achieve anomaly segmentation training, i.e.
Leeg
Focal(S, M?) + Mi))
(10)
where MM9 denotes the downsampled and reshaped ground truth mask of the query image, Focal(.) and Dice(-) represent the Focal loss and Dice loss, respectively. Morcover, we also guarantee that the image-level predicted label is consistent with the classification label of the query image. Binary Cross-Entropy (BCE) loss is leveraged to optimize anomaly classification,
Lels BCE(s,
(11)
where denotes predicted image-level anomaly score, y is ground truth classification label, and (CE(-) denotes BCE loss function. Finally, hyper-parametes used to balance the classification and segmentation losses,
Cola + ACsegg
(12)
3.7 Inference on Target Domain
During inference, for given target domain data Diarget, similar to the training phase, we first use the NN method obtain normal-guided score map S,, Next, we apply RM and AFL modules to generate an abnormal-guided score map S,- According to Eq. (9) we merge the two score maps as the final score map E LR Next, the size of S is reshaped to R and up-sampled TR HxW
Experiment
4.1 Experimental Setup
Datasets. validate the efficiency of our NAGL. framework, we construct three benchmarks using the MVTecAD (2), VisA (69), and BraTS [4] datasets. The benchmarks include (1) training

---

Table 1: Comparison of the proposed method with the previous methods on MVTecAD and VisA datasets. No and A' represent the number of normal and abnormal reference samples in i-shot learning, respectively. Results marked with are quoted from [23], while those marked with are based on our re-implementation. The best/runner-up results are highlighted in bold/underline.
MVTecAD
VisA
Image-level AP
Pixel-level PRO
Image-level AP
Pixel-level PRO
Setring
Method
AUROC
Fl-max
AUROC
Fi-max
AUROC
Fi-max
AUROC
Fl-max
SPADE
10
81.0
90.6
90.3
91.2
83.9
42.4
79.5
82.0
80.7
95.6
84.1
35.5
PaDiM
13
88.1
89.3
73.3
40.2
62.8
75.3
89.9
64.3
PatchCore
95.4
N'
WinCLIP
93.1
96.5
93.7
95.2
87.1
55.9
83.8
85.1
83.1
96.4
85.1
41.3
PromptAD
94.6
97.1
95.9
87.9
86.9
88.4
96.7
85.1
ResAD"
92.7
83.7
95.9
Nº + A'
Ours
95.8
97.5
95.7
96.6
92.9
58.9
88.5
89.4
85.4
97.5
91.1
41.8
SPADE!
82.9
91.7
91.1
92.0
85.7
44.5
80.7
82.3
81.7
96.2
85.7
40.5
PaDiM!
R3
78.9
89.3
89.2
91.3
78.2
43.7
67.4
71.6
75.7
92.0
70.1
21.1
PatchCore
48
86.3
93.8
92.0
93.3
82.3
53.0
81.6
84.8
82.5
96.1
82.6
41.0
N²
WinCLIP
94.4
97.0
94.4
96.0
88.4
58.4
84.6
85.8
83.0
96.8
86.2
43.5
PremptAD
97.9
InCTRI
68
94.0
96.9
85.8
87.7
ResAD"
93.9
88.3
96.5
Nº + A
Ours
96.8
97.9
96.3
96.8
93.2
59.8
89.8
90.6
86.8
97.6
91.4
43.3
SPADE!
84.8
92.5
92.7
87.0
46.2
83.4
82.1
96.6
87.3
43.6
PaDiM
13
80.4
90.5
90.2
92.6
81.3
46.1
72.8
75.6
78.0
93.2
72.6
24.6
PatchCore
48
88.8
94.5
92.6
94.3
84.3
55.0
85,3
87.5
84.3
96.8
84.9
43.9
N
WinCLIP
23
95.2
97.3
94.7
96.22
89.0
59.5
87.3
88..8
84.2
97.2
87.6
47.0
PromptAD
98.5
89.1
97.4
IcCTRL 68
94.5
97.2
87.7
90.2
ResAD*
60
90.7
95.7
93.9
95.8
88.7
53.0
89.3
90.7
86.5
96.8
84.1
41.6
No + A
Ours
97.1
98.0
96.4
97.0
93.5
60.1
91.2
91.7
87.8
97.8
91.5
44.0
VisA and testing on MVTecAD, (2) training MVTecAD and testing VisA, and (3) training on MVTecAD and testing on BraTS, The first two benchmarks assess generalization capabilities across different industrial domains, while the third evaluates cross-domain transfer capabilities from industrial to medical applications. During inference, the normal references are sampled from the training set of target datasets, while abnormal references are sampled from each anomaly type. Further details are provided in Appendix A
Evaluation Metrics. We evaluate both image-level anomaly classification and pixel-level segmen-
tation performance using three metrics for each task, following previous works (23] 12 60 For detection performance, we employ the Area Under the Receiver-Operator Curve (AUROC, [3] ) maximum F1-score at the optimal threshold (Fl-max), and the Average Precision (AP), calculated using image-level anomaly scores. Similarly, for segmentation performance, we utilize the AUROC, F1-max, and per-region overlap (PRO) metrics, computed using pixel-wise anomaly scores.
model as the vision encoder, specifically utilizing its most lightweight version (ViT-S, 21 M parameters) to ensure low latency in practical applications. We freeze the parameters of the vision encoder throughout the experiments and only update the parameters of the proposed attention modules. The model is optimized using AdamW W [39] with an initial learning rate of 1 x 10-5, which is reduced by a factor of 1 at epoch 10 and 15. The training process converges within 20 epochs, with each epoch comprising 500 sampled episodes. Input images are resized to 448 x 448 resolution without data augmentation. We set the number of learnable proxies (P) to M 25 by default and use loss balance weight (A) of 1.0. Following
Implementation Details. Our approach employs the ViT-based 47
previous works 23 BI), we set the number of normal references as K₃ [1, 4]. Considering the
scarcity of abnormal samples, we only use one abnormal reference (K₂ = 1), making our approach highly applicable in real-world scenarios. The implementation is based on PyTorch@2.11 1.1. and
the experiments are conducted on a single NVIDIA RTX 4090 24GB GPU. To ensure statistical reliability, we report results averaged across independent runs with different random seeds.
7

---

Table 2: The image/pixel-level AUROC scores of the proposed method and previous methods on the BraTS dataset. Results marked with are based on our re-implementation. Other
Table 3: Ablation study on the MVTecAD and VisA datasets. Image/pixel-level AUROC are reported. /A' represents the one normal/abnormal refer- ence sample, "NAGL"" indicates the use of our pro- posed framework. There are four different settings. i/ii uses only normal/abnormal samples through NN search, ii merges the results of and ii without ad- ditional processing, and includes our proposed process.
results are reported from [60].
Sctting
Method
Image
Pixel
Mean
N²
ResAD
6U
73.5
91.0
82.3
+ A'
Ours
78.1
96.5
87.3
SPADE 10
PaDiM
58.0
59.4
90.2
92.8
75.4
74.8
Na A NAGL
Image
MVTsaAD
Pixel
Image
VisA Pixel
PatchCorc
48
58.2
93.5
75.9
93.2
94.5
81.5
95.3
N²
WinCLIP
RegAD
20
22
54.6
55.9
91.5
81.4
73.7
68.0
95.8
90.7
70.1
96.6
92.1
88.5
77.2
84.5
97.5
93.5
94.6
88.4
InCTRI
74.6
ResAD
60
66.2
91.5
78.9
Table 4: Comparison of the total parameters, training time, and inference speed of the proposed method with InCTRI. and ResAD. The training time of InC-
N² + A
Ours
82.1
96.8
89.5
SPADE 1D
66.3
94.8
80.6
PaDiM 1
60.6
94.5
77.6
TRL and ResAD is measured under the N setting,
PatchCore
48
71.2
95,9
83.6
while our method is based on the N A setting.
N
RegAD
21
60.0
87.3
73.7
WinCLIP
23
67.3
93.2
80.3
MVFA
75.2
92.7
84.0
Total Parameters (M)
Training Time (H)
Inference Speed (FPS)
loCTRL
68
76,9
Method
ResAD
60
74.9
94.3
84.6
InCTRL. ResAD
117.5
0.7
1.2 7,8
59.2
20.6
No + A"
Ours
84.9
97.1
91.0
Ours
24.4
0.3
17.1
Comparison Methods. We select some representative few-normal-shot AD methods to comparison, including SPADE [0). PaDiM [3], PatchCore [48), RegAD [21). WinCLIP [23], MVFA [22], and
PromptAD B1. For generalist AD, primarily compared
approach with InCTRL (68 and
ResAD 660 Tab. D and Tab. jobseprreeettt the comparative results of our method against these baselines.
The results marked with i are reported by [23]. Due to incomplete ResAD results (only reporting 2/4-shot settings with AUROC metrics, we reproduce the results using their official code (marked with *). For further details, please refer to Appendix
4.2 Main Results
Tab. I and Tab. present comparative results between our proposed method and cxisting approaches across MVTecAD, VisA, and BraTS datasets. The experimental results demonstrate three key advantages of our method. Additionally, to demonstrate the generalization capabilities, we also
present the performance on MVTec3D MVTecLOCO ①. BTAD 46), and MPDD 24 datasets in Appendix
Benefits of Single Abnormal Sample. Incorporating just one abnormal reference sample (A)) yields substantial performance gains across all scenarios. On industrial datasets (Tab. [ our N' + AA setting achieves 95.8% and 96.6% for image and pixel AUROC, respectively on MVTeCAD, surpassing the best N baseline method (WinCLIP: 95.29 2%. 96.2%) by 0.6 and 0.4 percentage points while using fewer reference samples. A similar trend is observed on VisA, where our N2 + A' setting achieves 89.8% and 67.6% for image and pixel AUROC, outperforming the best baseline method (ResAD with 44; 89.3%, 96.8% by 0.5 and 0.8 percentage points, respectively. These results show significant benefits from abnormal reference data.
Generalization. Our method exhibits superior generalization capabilities compared to ResAD across multiple datasets. On MVTecAD, under the N + A' setting, our approach achieves 95.7% image F1-max and 58.9% pixel F1-max, outperforming ResAD by 4.5 and 10.7 percentage points, respectively. Similarly, on VisA, we attain 85.4% image Fl-max and 41.8% pixel F1-max, exceeding ResAD by 4.1 and 4.0 percentage points. Furthermore, shown in Tab. 2 our method demonstrates superior cross-domain generalization, achieving 89. 59.5% average AUROC on the BraTSS medical dataset with just two normal and one abnormal sample, a 10.6 percentage point improvement over ResAD
8

---

Industrial Domain
Industrial
Medical Domain
Image
GT
N
N' 4'
Figure 3: Qualitative results. The first row displays the input images, and the second row shows the ground truth. The third row illustrates anomaly score maps using 1 normal sample, while the bottom row shows anomaly score maps produced by our method using 1 normal and abnormal samples.
(78.9% with 2 setting). Notably, across all other experimental settings, method consistently outperforms existing approaches in both anomaly detection and segmentation. These results clearly demonstrate the effectiveness of our method in generalist AD.
Scalability. Through comparative analysis, our method demonstrates excellent scalability as the number of normal samples increases across all three datasets. For example, on VisA, we observe consistent performance improvements when progressing from N + A to N + A and N + A settings. Specifically, the image Fl-max score increases from 85.4% to 86.8% and 87.8%, while the pixel Fl-max score improves from 41.8% to 43.3% and reaches 44.0%. On BraTS, the average AUROC improves from 87.3% to 89.5% and reaches 91.0%. These results indicate that our method effectively utilizes additional normal samples to enhance the detection ability.
4.3 Ablation Study
To further investigate the effectiveness of our method, we conduct an ablation study the MVTecAD and VisA datasets. Tab. Tab.3pprese presents the results from different experimental settings. The results clearly demonstrate several important findings.
First, comparing the results of i, ii, and iii, observe that naively incorporating abnormal references leads to significant performance degradation both at the image and pixel levels. This aligns with our argument in Sec. that the abnormal features cause severe false activation, where the unrefined abnormal features introduce misleading guidance that incorrectly highlights normal patterns as anomalies, thereby compromising detection accuracy.
Second, when comparing settings iii and iv, the benefits of our proposed method become evident. By implementing the NAGL framework in setting iv, method manages to effectively mitigate the false activation problem observed when naively using an abnormal reference. Properly mining the abnormal variations in residual space guide detection can effectively suppress the over-activation, resulting in 6.2 average AUROC improvements from 88.4% to 94.6%.
Finally, the comparison between and iv reveals that our method fully unleashes the potential of abnormal references, with gains of +2.6 Image AUROC on MVTecAD and +7.0 on VisA. proving that properly processed abnormal samples provide complementary discriminative signals.
4.4 Analysis of two attentions in RM-AFL module
In the first attention (RM), we set the value as Res(F", (F), which provides the differences in the residual space. We define the key as Jua which are the original abnormal patterns. Since the query is learnable, the output is the optimally aggregated residual features (termed as residual proxies P).
The second attention (AFL) aims to learn query-related abnormal patterns by comparing normal- abnormal differences and normal-query differences. We set the query as Po and the key as Res(Fi )"). This comparison is achieved by computing the attention map between P and

---

Res(F", Then the query features F are aggregated by the attention map to obtain the query- related abnormal patterns (termed as anomaly proxies P).
The residual proxies P represent abnormal patterns of reference abnormal samples in residual space, and anomaly proxies P represent abnormal patterns of the query sample in vision space. As validated in Appendix [ residual features exhibit a common distribution, even among residual features of different anomalies. Residual features of known anomalies can provide references; if the residual feature of query is similar to them, the corresponding visual feature is likely be an anomaly.
4.5 Efficiency Analysis
We compare the efficiency of our method with InCTRL and ResAD in terms of total parameters, training time, and inference speed. Tab. 4 shows that our method is more efficient than InCTRL and ResAD regarding total parameters and traiinn time. Our method has 24.4 million parameters, which around 5x smaller than InCTRL. Our method only requires 0.3 hours for training, which is 69x faster than ResAD. In terms of inference speed, our method achieves 17.1 FPS, which is 14x faster than InCTRL and 2x faster than ResAD, These results demonstrate the efficiency of our method for real-world deployment.
4.6 Qualitative Results
Fig. presents qualitative results from both industrial and medical datasets. Comparing the score maps, we observe that our method effectively generates more accurate results, which demonstrate that incorporating abnormal reference samples provides valuable guidance boundaries for the detection process, and our method effectively leverages this information to improve detection accuracy.
5 Conclusion
In this paper, we introduce a novel task for anomaly detection using both normal and abnormal refer- ences. Our approach addresses the limitations of traditional methods by leveraging limited known anomalies to guide the detection of unseen anomalies. We propose a NAGL framework that extracts discriminative features in the residual space, effectively capturing the essence of anomalies. Experi- mental results across multiple benchmarks demonstrate that our method significantly outperforms existing approaches, particularly in detecting challenging unseen anomalies. The performance gains are consistent across various datasets and settings, highlighting the robustness of our approach. This work opens new research directions for anomaly detection with limited supervision, with potential applications in industrial inspection and medical diagnosis. Future research could delve into more effective methods for leveraging scarce anomaly samples expanding prompts. For instance, could involve enabling language-based descriptions for normal or abnormal references.
6 Acknowledgements
This work is supported by the National Natural Science Foundation of China (No. 62471405. 62331003, 62301451). Jiangsu Basic Research Program Natural Science Foundation (BK20241814 Suzhou Basic Research Program (SYG202316) and XJTLU REF-22-01-011 -0-000000 XITLU University Research Center, Jiangsu Province Engineering Research Center of Data Science and Cognitive Computation at XJTLU and SIP innovation platform (YZCXPT2022103),
References
[1] Paul Bergmann, Kilian Batzner, Michael Fauser, David Sattlegger, and Carsten Steger. Beyond dents and scratches: Logical constraints in unsupervised anomaly detection and localization. International Journal of Computer Vision, 130(4):947 969, 2022.
[2] Paul Bergmann, Michael Fauser, David Sattlegger, and Carsten Steger. MVTec AD - a comprehensive real-world datasct for unsupervised anomaly detection. In CVPR, 2019.
[3 Paul Bergmann, Michael Fauser, David Sattlegger, and Carsten Steger. Uninformed students: Student- teacher anomaly detection with discriminative latent embeddings. In CVPR, 2020.
10

---

[4] Paul Bergmann, Xin Jin, David Satdlegge, and Carsten Steger. The mvtec 3d-ad dataset for unsupervised 3d anomaly detection and localization. arXiv preprini 1rX1v:2112.00045, 2021.
[5] Tri Cao, Jiawen Zhu, and Guansong Pang. Anomaly detection under distribution shift. In ICCV, 2023.
[6] Yunkang Cao, Xiaohao Xu, Yuqi Cheng, Chen Sun, Zongwei Du, Liang Gao, and Weiming Shen. Person- alizing vision-language models with hybrid prompts for zero-shot anomaly detection. IEEE Transactions on Cybernetics, 55(4):1917-192) 2025.
[7] Yunkang Cao, Jiangning Zhang, Luca Frittoli, Yuqi Cheng, Weiming Shen, and Giacomo Boracchi. AdaCLIP: Adapting CLIP with hybrid learnable prompts for zero-shot anomaly detection. In ECCV, 2024.
[8] Yuanhong Chen, Yu Tian, Guansong Pang, and Gustavo Cameiro. Deep one-class classification via interpolated gaussian descriptor. In AAAI, 2022.
[9] Zining Chen, Xingshuang Luo, Weiqiu Wang. Zhicheng Zhao, Fei Su, and Aidong Men. Filter or compensate: Towards invariant representation from distribution shift for anomaly detection. IAAI, 2025.
[10] Niv Cohen and Yedid Hoshen. Sub-image anomaly detection with deep pyramid correspondences. ArXiv, abs/2005. 022557. 2020.
[11] Alex Costanzino, Pierluigi Zama Ramirez, Giuseppe Lisanti, and Luigi Di Stefano. Multimodal industrial anomaly detection by crossmodal feature mapping. In CVPR, 2024.
[12] Simon Damm, Mike Laszkiewicz, Johannes Lederer, and Asja Fischer. AnomalyDINO: Boosting patch- based few-shot anomaly detection with dinov2. arXiv, 2024.
[13] Thomas Defard, Aleksandr Setkov, Angelique Loesch, and Romaric Audigier. PaDiM: A patch distribution modeling framework for anomaly detection and localization. In ICPR, 2021.
[14] Choubo Ding, Guansong Pang, and Chunhua Shen. Catching both gray and black swans: Open-set supervised anomaly detection. In CVPR, 2022.
[15] Zheng Fang. Xiaoyang Wang. Haocheng Li, Jiejie Liu, Qiugui Hu, and Jimin Xiao. FastRecon: Few-shot industrial anomaly detection via fast feature reconstruction. In ICCV, 2023.
[16] Zhihao Gu, Jiangning Zhang. Liang Liu. Chen, Jinlong Peng. Zhenye Gan, Guannan Jiang, Annan Shu, Yabiao Wang, and Lizhuang Ma. Rethinking reverse distillation for multi-modal anomaly detection. In AAAI, 2024.
[17] Jia Guo, Shuai Lu, Lize Jia, Weihang Zhang, and Huiqi Li. Recontrast: Domain-specific anomaly detection via contrastive reconstruction. In NeurlPS, volume 36, pages 10721-10740, 2023.
[18] Jia Guo, Shuai Lu, Weihang Zhang, Fang Chen, Huiqi Li, and Hongen Liao. Dinomaly: The less is more philosophy in multi-class unsupervised anomaly detection. In CVPR, pages 20405-20415, 2025.
[19] Haoyang He, Jiangning Zhang. Hongxu Chen, Xuhai Chen, Zhishan Li, Xu Chen, Yabiao Wang. Chengjie Wang, and Lei Xie. A diffusion- based framework for multi anomaly detection. In AAAI, 2024.
[20] Kelei He, Chen Gan, Zhuoyuan Li, Islem Rekik, Zihao Yin. Wen Ji. Yang Gao. Qian Wang, Junfeng Zhang. and Dinggang Shen. Transformers in medical image analysis. Intelligent Medicine, 3(1):59-78, 2023.
[21] Chaoqin Huang, Haoyan Guan, Aofan Jiang, Ya Zhang, Michael Spratling, and Yan-Feng Wang. Registra- tion based few-shot anomaly detection. In ECCV, pages 303-319, 2022.
[22] Chaoqin Huang, Aofan Jiang, Jinghao Feng, Ya Zhang, Xinchao Wang, and Yanfeng Wang. Adapting visual-language models for generalizable anomaly detection in medical images. In CVPR, pages 11375- 11385, 2024.
[23] Jongheon Jeong, Yang Zou, Taewan Kim, Dongqing Zhang, Avinash Ravichandran, and Onkar Dabeer. WinCLIIP Zero-/few-shot anomaly classification and segmentation. In CVPR, 2023.
[24] Stepan Jezek, Martin Jonak, Radim Burget, Pavel Dvorak, and Milos Skotak. Deep learning- based defect detection of metal parts: Evaluating current methods in complex conditions. In ICUMT, 2021.
[25] Xi Jiang, Jian Li, Hanqiu Deng, Yong Liu, Bin-Bin Gao, Yifeng Zhou, Jialin Li, Chengjie Wang, and Feng Zheng. MMAD: A comprehensive benchmark for multimodal large language models in industrial anomaly detection. In ICLR, 2025.
11

---

[26] Xi Jiang, Jianlin Liu. Jinbao Wang, Qiang Nie, Kai Wu. Yong Liu, Chengjie Wang, and Feng Zheng. SoftPatch: Unsupervised anomaly detection with noisy data. In NewrlPS, 2022.
[27] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In NeurlPS, volume 25, 2012.
[28] Mingyu Lee and Jongwon Choi. Text-guided variational image generation for industrial anomaly detection and segmentation. In CVPR, 2024.
[29] Jiarui Lei, Xiaobo Hu, Yue Wang, and Dong Liu. PyramidFlow: High-resolution defect contrastive localization using pyramid normalizing flow. In CVPR, 2023.
[30] Chun-LLiagg Li, Kihyuk Sohn, Jinsung Yoon, and Tomas Pfister. CutPaste: Self-supervised learning for anomaly detection and localization. In CVPR, 2021.
[31] Xiaofan Li, Zhizhong Zhang, Xin Tan, Chengwei Chen, Yanyun Qu, Yuan Xie, and Lizhuang Ma. PromptAD: Learning prompts with only normal samples for few-shot anomaly detection. In CVPR, 2024.
[32] Xiaoya Li, Xiaofei Sun, Yuxian Meng. Junjun Liang, Fei Wu, and Jiwei Li. Dice loss for data-imbalanced nlp tasks. In ACL, 2020.
[33] Yiyue Li, Shaoting Zhang, Kang Li, and Qicheng Lao. One-to-Normal: Anomaly personalization for few-shot anomaly detection. In NeurlPS, 2024.
[34] Jiang Lin and Yaping Yan. A comprehensive augmentation framework for anomaly detection. In AAAI, 2024.
[35] Tsung- Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In ICCV, 2017.
[36] Jiaqi Liu, Guoyang Xie, Ruitao Chen, Xinpeng Li, Jinbao Wang. Yong Liu, Chengjie Wang, and Feng Zheng. Real3D-AD: A dataset of point cloud anomaly detection. In NeurlPS, 2024.
[37] Wenrui Liu, Hong Chang, Bingpeng Ma. Shiguang Shan, and Xilin Chen. measurbbee anomaly detection. In CVPR, 2023.
[38] Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmenta- tion. In CVPR, pages 3431-3440, 2015.
[39] Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In ICLR, 2019.
[40] Ruiying Lu, YuJie Wu, Long Tian, Dongsheng Wang, Bo Chen, Xiyang Liu, and Ruimin Hu. Hierarchical vector quantized transformer for multi-class unsupervised anomaly detection. In NewrIPS, 2023.
[41] Wei Luo. Yunkang Cao. Haiming Yao. Xiaotian Zhang, Jianan Lou, Yuqi Cheng, Weiming Shen, and Wenyong Yu. Exploring intrinsic normal prototypes within single image for universal anomaly detection. In CVPR, pages 1974-9988, June 2025.
[42] Wenxi Lv, Qinliang Su, and Wenchao Xu. One-for-all few-shot anomaly detection via instance-induced prompt learning. ICLR, 2025.
[43] Shuai Lyu, Fangjian Liao, Zeqi Ma. Rongchen Zhang, Dongmei Mo, and Waikeung Wong. MVREC: A general few-shot defect classification model using multi-view region-contex: In AAAI, 2025.
[44] Declan MeIntosh and Alexandra Branzan Albu. Inter-realization channels: Unsupervised anomaly detection beyond one-class classification. In CVPR, 2023.
[45] Bjoern H Menze, Andras Jakab, Stefan Bauer, Jayashree Kalpathy- Kalpathy-Cramer, Keyvan Farahani, Justin Kirby. Yuliya Burren, Nicole Porz, Johannes Slotboom, Roland Wiest, et al. The multimodal brain tumor image segmentation benchmark (brats). IEEE Iransactions on medical imaging, 34(10):193322044 2014.
[46] Pankaj Mishra, Riccardo Verk, Daniele Fornasier, Claudio Piciarelli, and Gian Luca Foresti. VT-ADL: A vision transformer network for image anomaly detection and localization. In ISIE, pages 01-06, 2021.
[47] Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V. Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel HAZIZA, Francisco Massa, Alaaeldin El-Nouby, Mido Assran, Nicolas Ballas, Wojeiech Galuba, Russell Howes, Po- Yao Huang, Shang Wen Li, Ishan Misra, Michael Rabbat, Vasu Sharma, Gabriel Synnaeve, Hu Xu, Herve Jegou, Julien Mairal, Patrick Labatut, Armand Joulin, and Piotr Bojanowski. DINOv2: Learning robust visual features without supervision. Transactions on Machine Learning Research, 2024.
12

---

[48] Karsten Roth, Latha Pemula, Joaquin Zepeda, Bernhard Schölkopf, Thomas Brox, and Peter Gehler. Towards total recall in industrial anomaly detection. In CVPR, 2022.
[49] Luc PJ Sträter, Mohammadreza Salehi, Efstratios Gavves, Cees GM Snoek, and Yuki M Asano. Generalad: Anomaly detection across domains by attending distorted features. In ECCV, 2024.
[50] Fenfang Tao, Guo-Sen Xie, Fang Zhao, and Xiangbo Shu. Kernel-aware graph prompt learning for few-shot anomaly detection. In AAAI, 2025.
[51] Tran Dinh Tien, Anh Tuan Nguyen, Nguyen Hoang Tran, Ta Duc Huy, Soan Duong, Chanh D Tr Nguyen, and Steven QH Truong. Revisiting reverse distillation for anomaly detection. In CVPR, 2023.
[52] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Eukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NeurlPS, 2017.
[53] Xiaolei Wang, Xiaoyang Wang, Huihui Bai, Eng Gee Lim, and Jimin Xiao. CNC: Cross-modal normality constraint for unsupervised multi-class anomaly detection. In AAAI, 2025.
[54] Xiaolei Wang, Xiaoyang Wang, Huihui Bai, Eng Gee Lim, and Jimin Xiao. Cne: Cross-modal normality constraint for unsupervised multi-class anomaly detection. In AAAI, volume 39, pages 7943-7951, 2025.
[55] Xiaolei Wang, Xiaoyang Wang. Huihui Bai, Eng Gee Lim, and Jimin Xiao. Decad: Decoupling anomalies in latent space for multi-class unsupervised anomaly detection. In ICCV, 2025.
[56] Xiaolei Wang, Xiaoyang Wang, Huihui Bai, Eng Gee Lim, and Jimin Xiao. Ice: Intra-cluster contraction for pedestrian anomaly detection under one-class classification setting. IEEE Transactions on Artificial Intelligence, 2025.
[57] Xiaoyang Wang, Huihui Bai, Limin Yu, Yao Zhao, and Jimin Xiao. Towards the uncharted: Density- descending feature perturhation for semi-supervised semantic segmentation. In CVPR, pages 3303-3312, 2024.
[58] Guoyang Xie, Jinbao Wang. Jiaqi Liu, Yaochu Jin, and Feng Zheng. Pushing the limits of fewshot anomaly detection in industry vision: Graphcore. In ICLR, 2023.
[59] Hui-Yue Yang, Hui Chen, Ao Wang, Kai Chen, Zijia Lin, Yongliang Tang, Pengeheng Gao, Yuming Quan, Jungong Han, and Guiguang Ding. Promptable anomaly segmentation with sam through self-perception tuning. In AAAL, 2025.
[60] Xincheng Yao, Zixin Chen, Chao Gao, Guangtao Zhai, and Chongyang Zhang. ResAD: A simple framework for class generalizable anomaly detection. In NeurlPS, 2024.
[61] Xineheng Yao, Ruoqi Li, Jing Zhang, Jun Sun, and Chongyang Zhang, Explicit boundary guided semi- push-pull contrastive learning for supervised anomaly detection. In CVPR, 2023.
[62] Vitjan Zavrtanik, Matej Kristan, and Danijel Skočaj. DRAEM-a discriminatively trained reconstruction embedding for surface anomaly detection. In ICCV, 2021.
[63] Hui Zhang, Zuxuan Wu, Zheng Wang. Zhineng Chen, and Yu- Yu-Gang Jiang. Prototypical residual networks for anomaly detection and localization. In CVPR, 2023.
[64] Ximiao Zhang, Min Xu, and Xiuzhuang Zhou, RealNet: A fcature selection network with realistic synthetic anomaly for anomaly detection. In CVPR, 2024.
[65] Qihang Zhou, Guansong Pang, Yu Tian, Shiho Hc, and Jiming Chen. AnomalyCLII: Object-agnostic prompt learning for zero-shot anomaly detection. In ICLR, 2024.
[66] Qihang Zhou, Jiangtao Yan, Shibo Hc, Wenchao Meng, and Jiming Chen. PointAD: Comprchending 3d anomalies from points and pixels for sero-shot 3d anomaly detection. In NeurlPS, 2024.
[67] Jiawen Zhu, Choubo Ding, Yu Tian, and Guansong Pang. Anomaly heterogeneity learning for open-set supervised anomaly detection. In CVPR, 2024.
[68] Jiawen Zhu and Guansong Pang. Toward generalist anomaly detection via in-context residual learning with shot sample prompts. In CVPR, 2024.
[69] Yang Zou, Jongheon Jeong, Latha Pemula, Dongqing Zhang, and Onkar Dabeer. Spot-the-differnce self-supervised pre-training for anomaly detection and segmentation. In ECCV, 2022.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
Our contributions are summarized in the Abstract and Introduction sections.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Please see Appendix
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
14

---

Justification:
This paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: Please the Implementation Details in Sec. 4.1
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
15

---

Answer: [Yes]
Justification:
As mentioned in the Abstract, our code and data will be made publicly
available.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to to reproduce the results. See the NeurlPS code and data submission guidelines (https
ssiinyolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Please see Sec. 4.1 and Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
we report results averaged across independent runs with different random seeds. The detailed results can be found in Appendix
Justification: As mentioned in Sec. 4.1
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
16

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπхо rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Please see the Implementation Details in Sec. and Efficiency Analysis at Sec.4.5
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: We have read the NeurIPS Code of Ethics and confirm that our research complies with these ethical standards.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Please see the Appendix
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
17

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
Justification:
We think that our paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: The datasets utilized in this paper are all open-source, and we have cited the corresponding papers accordingly.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the Datasets.papersstt
package should be provided. For popular datasets, paperswi thcode. cww/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
18

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer:
[NA]
Justification: Our paper introduces new anomaly detection framework without releasing new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [NA]
Justification: Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
19

---

16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: The LLM is used only for editing, and the core method development in our paper does not involve LLMS as any important.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should be described.
20

---

A Dataset Collection
To implement our normal-abnormal guided generalist anomaly detection, we construct new bench- marks using three popular public datasets, including MVTecAD 22, VisA [69] and BraTS [45] We conduct evaluations from two perspectives: (1) industrial-to-industii
evaluation across different industrial scenarios (MVTecAD < VisA), and (2) industriall-o-mmdicca evaluation across industrial and medical domains (MVTecAD + BraTS). Specifically, when MVTecAD serves as training set, we test on VisA and BraTS; when testing on MVTecAD, we use VisA as the training set.
Original Dataset Structure. Each dataset individually divided into its own training and test sets. For all datasets, their respec- tive training sets contain only good samples, while each dataset has unique structure for its corresponding test set. As shown Tab. [ MVTecAD contains 15 object classes with 73 different defect types, distinguishing between various defect categories. VisA includes 12 object classes with various defect types, but all anomalies within each object class grouped together without specific categoriza- tion. BraTS exclusively comprises brain MRI scans, with abnormal samples including fumour segmentation.
Table 5: Statistics of the datasets. |C| denotes the number of classes, [7] denotes the number of defect types, N and A represent the normal and abnormal samples, respectively. K1/K₂ denotes the number of normal/abnormal reference samples.
Test
Reference
Dataset
[C]
[T]
N
MVTecAD VisA BraTS
73
467
1258
73 x K2
962
1200
12 X
K1
12 K2
1 154
1097
1
K
1 x K2
Our Input Data. During both training and testing, we implement a random sampling strategy to construct our input data. Specifically, our input data consists query input and reference set that includes both normal and abnormal samples. We first randomly select sample from the entire test set the query input. Then, based on the query input's category, randomly select Ki normal samples from the training set as normal references. Additionally, according to the query input's 's defect-type we randomly select K2 abnormal samples as abnormal references. During training, each episode input are randomly combined samples. Unlike the training process, during testing, we randomly selected reference set only once for each anomaly type and then use this to all samples of this anomaly type. The detailed statistics of our reference set are shown in Tab.
Defect type between Query and Reference. As shown in Fig. 4 there are some input cases: (a) the input query and abnormal reference
Query
Abnormal Reference
Normal Reference
samples belong to the same defect type; (b) the imput query and abnormal reference sam- ples exhibit different defect types; (c) when the input query is normal, the abnormal ref- erence randomly selects defect type. The (b)
(a)
same defect type
MVTecAD dataset categorizes different de- fect types separately, thereby ensuring that the input query (x") and abnormal reference
different
defect type
samples (R") belong the same defect cat- egory (Fig. 44p). In contrast, the VisA dataset combines all anomaly samples for each prod- uet type without distinguishing defect types, which results in cases where the input query and abnormal reference may exhibit different types of anomalies (Fig. 4D). Additionally, when the input query is normal, the abnormal reference randomly selects one defect type (Fig.
(c)
random defect type
Figure 4: Some examples of our input data
B Residual Features
To explore the guidance of abnormal residual features in anomaly detection, we visualize the feature distribution in the original vision space and the residual feature space. As shown Fig. 5 the features from different defect types are significantly different in the original vision space, while these features are more concentrated in their distribution in the residual feature space, suggesting it provides unified cross- representation for anomalies. Morcover, shown in Fig. 6 the
21

---

L2-norm of normal and abnormal features are overlapped in the original vision space, while they are separated in the residual feature space, indicating that residual features offer a more discriminative attribute for distinguishing between normal and abnormal samples.
(st Origine Voiio Space
(b) Redibal Sgace
co 12-kumm Dermherte of brage Temene
(h) 1254omm Dkonhetion of Rtedall Fomer
abons
Figure 5: T-SNE visualization of features. Differ-
Figure 6: (a) In the original visual feature space, the L2-norm distributions of normal and anoma-
ent colours denote different defect types. (a) In
the original vision space, the features from differ-
lous features exhibit significant overlap. (b) In the residual feature space, the L2-norm distribu- tions of normal and anomalous features are more
ent defect types are significantly different. (b) In
the residual feature space, these features are more
overlapped in their distribution.
clearly separated.
C More Abnormal References
Considering the scarcity of anomalous samples in real-world scenarios, we focused on using only one anomalous reference sample in the main text. To further explore the scalability of the NAGL framework, we conducted exper- iments using two anomalous reference samples. As shown in Table 6 using two anomalous ref- erence samples improved the performance of the NAGL framework both MVTecAD and VisA datasets. This shows NAGL effectively multiple anomalous references improve detection performance.
Table 6: The AUROC performance comparison between and 2 abnormal references (A and A²).
Setting
MVTeaAD
VisA
Mean
N² A' N1 Aa
96.5
89.5
93.0
+.0.8
N A
96.8
89.8
93.3
+ A2
97.3
90.5
93.9
+ 0.6
N1 + 4
97.1
91.2
94.2
No + A
97.5
92.2
94.9
+ 0.7
D Compared with ViT-based Results
Due to the extensive training time required for ResAD, we only reproduced the CNN-based (WideRes- Net50) results in the main text, maintaining identical training hyperparameters as specified in the original paper. As shown in Tab. 7 for more equitable comparison, we also compare our approach with ViT-based results. It is clear that our method still achieves competitive performance.
Table 7: The AUROC performance comparison of ViT-based backbones on MVTecAD and VisA datasets.
MVTccAD
VisA
MVTecAD
VisA
Setting
Method
Image
Pixel
Image
Pixel
Setting
Method
Image
Pixel
Image
Pixel
WinCLIP
94.4
96.0
84.6
96.8
WinCLIP
95.2
96.2
87.3
97.2
2
InCTRL
04.0
85.8
InCTRL
94.5
87.7
ResAD
94.4
95.6
84,5
95.1
ResAD
94.2
96.9
90.8
97.5
2 A
Ours
96.8
96.8
89,8
97.6
N
A
Ours
97.1
97.0
91.2
97.8
E
Discussion
Limitations: A drawback of our study is that it focuses solely image data for experimentation. It would be highly beneficial to apply our approach to other data types, like video and time series, to thoroughly assess the adaptability of method.
Social Impacts: a unified framework for generalist anomaly detection, the introduced approach does not raise specific ethical issues or adverse societal effects. The datasets utilized are publicly available. All qualitative illustrations are derived from industrial product imagery, ensuring no violation of personal privacy.
22

---

Hyperparameter Analysis
Fig. shows the impact of the number of learn- able proxies M. We observe that the perfor- mance of our method increases as M increases. This is because a larger number of proxies can better capture the diversity of the feature space, which is beneficial for the refinement process. However, the performance improvement dimin- ishes M exceeds 25. One possible reason the model becomes over wver-parametrrize, lead- ing to overfitting. Therefore, we set M 25 in our experiments to balance performance and
MYTAcAD
Pixel
Imngee
Pisel
Figure 7: Impact of the number of learnable prox-
ies M on MVTecAD and VisA datasets.
efficiency.
G
Additional Results
Additionally, we evaluated the performance of our proposed method on the MVTec3D and MVTe- cLOCO datasets further investigate its generalizability, As shown in Tab. our method achieves competitive results compared to ResAD in both 3D anomaly detection and logical anomaly detection, demonstrating its generalizability.
Table 8: Comparison of the proposed method with the ResAD on MVTec3D and MVTecLOCO datasets.
MVTecLOCO
Sctting
Method
Image-level
Pixel-level AUROC PRO Fl-max
Image-level
Pixel-level
AUROC
Fl-max
AUROC AP Fl-max AUROC
PRO Fl-max
N
ResAD" Ours
63.8
86.4
88.7
94.2
88.2
21.2
60.6
73,9
77.8
65.9
60.8
17.9
N1 + A
83.0
95.0
91.7
94.7
98.4
50.3
63.3
76.3
78.4
66.6
69.9
19.4
2/2
ResAD' Ours
66.7
88.6
88.9
94.9
90.1
24.5
62.3
76.3
77.7
66.0
62.3
18.3
N + A
82.8
94.9
91.6
94,8
98.4
51.3
64.6
77.4
78.0
66.3
69.3
18.5
N
ResAD'
70.1
89.4
88.9
95.0
91.4
26.0
65.7
77.5
77.7
67.5
60.7
19.1
N4 + A
Ours
86.9
96.4
92.3
95.4
98.6
54.2
71.5
81.5
79.1
66.9
69.2
18.6
As shown in Fig. 8 we provide further qualitative results obtained from our NAGL for pixel-level anomaly detection. The results demonstrate that our approach accurately localizes both large and small surface defects across various test cases. Furthermore, (mean + std) of NAGL on MVTecAD, VisA, BraTS, MVTec3D and MVTecLOCO datasets, under N + A' (Tab. 9). N + A' (Tab. 10 and N A' (Tab. 11 settings. We also report performance on the BTAD and MPDD datasets.
report the detailed subset-level results
Zipper
Capsule
Capules
Figure 8: Qualitative results for eight subcategories with different defect types.
23

---

Table The comprehensive results of our NAGL. framework, evaluated under settings with N A
Image-level AP
Pixel-level
Dataset
Objects
AUROC
Fi-max
AUROC
Fi-max
bottle
99.240.9
99.7+0.4
99. 29.2+0.0
98. 08.2+0.1
95. 84,840.2
76.2+0.8
cable
93.1+0.9
96.3+0.6
90.4=2.0
94.244 94.220.6
87.8+0.5
58.9+0.8
capsule
91 5+7.1
98.1+1.7
95.5+0.7
98. 180.2
96.7+0.4
46.8+5.3
carpet
100.0=0.0
100.0=0.0
100.0=0.0
300.0
98.5+0.0
68.940.0
grid
99 810.2
99.910.1
99.420.5
99,400.0
96.920.3
51,420.9
hazelnut
99.5a0.4
99.740.2
98.3+11
99. 99.5
1:0.44
77.5
leather
100.0+0.0
100,0=0.0
100.0=0.0
99.1
98.5+0.3
1..
motal not
99.4#0.1
99 9+0.0
98.2+003
95, 25.220.33
93 0..00.05
68.5
MVTecAD
pill
93.6:1.6
98.7+0.3
95.9:0.5
96.2+0.11
52.942.3
screw
77.2+0.9
91.5+00.4
86.7+0.2
97.8+0.4
92.2++0.8
43.2+2.4
tile
99.8+0.1
99,940.0
99.0=0.3
96.1
4+10.3
70.1
toothbushh
99.4a0.2
99.810.0
98.320..
99.240.1
95.120.2
65.942.2
transistor
86.1
79.3+6.9
76.7+4.1
84.3+3.7
64.7+1.77
39.1
wood
99.8+0.1
99.9+0.0
98.9+0.5
96.0+0.9
95.8+0.5
67.9+0.7
zipper
99.1+0.2
99.8d0.1
98.540.5
97.9w0.1
94.08003
54,5+0.6
Mean
95.8+0.7
97.5+00.6
95.7=0.3
96.6+0.2
92.9:40.0
58.9+0.5
candle
93.940.7
94.340.3
87.3+1.0
99.050.0
97.120.3
37.6 6...9999
cupsules
96.7a0.4
98.04002
93.5+0.1
97.920.
96.440.4
44.414.4
cashew
88.5+2.3
94.140.9
87.2+1.8
98.7+0.2
6.1.1
56.7+1.5
chewinggum
97.3
98 9+0.1
95.7+0.3
99. 3+0
84. 84.6+0066
68.7+0.4
fryum
95.720.7
98.110.3
92.8+0.9
91.120.4
40.8+2.4
macaronil
89.2+2.6
18.7+
83.1+2233
99.4+0.0
9..1+1.00
27.6++2.9
macaroni2
59.5+7.5
55,916.3
68.3+1.0
260.6
86.2+2.5
22.6644.2
Visk
pebl
94.9aU.7
93.410.7
90.61112
98.9402
94,2a0.1
60.71:4.6
pch2
86.3+0.9
84.3+1.4
80.0=1.4
95.6+0..
86.1+0.3
35.5
pch3
85.642.1
1.6
78.5+0.7
93.7+0.1
86.640.1
42,0+2.5
peb4
79.1=11.1
80.347.4
76.349.7
94.9 14.9
81.943.7
25.7+2.2
pipe_Iryum
95.820.6
Mean
88.5+1.4
89,4+1.2
85,420.7
97.5+0.1
91.1+0.4
41,810.5
BraTS
brain
78.143.4
95.4+10.9
93.820.3
96.5+0.1
79.420.5
49.4+1.0
bugel
95.5+2.0
98.940.5
95.9=1.6
28.99
99.740.1
66.940..
cable_glame
77.2=11.1
93.42-4.2
98.420.8
38.51:20.1
carrol
87.4a5.0
96.8
93.0&1.3
99.1
99.840.0
54.2+2.8
cookie
95.3+1.8
92,9+0.3
97.7+0.2
57.7+4.2
dowel
74.8+2,7
92.9+1.2
90.7+1.0
94. 3+0.5
98 6d0.2
40.0+3.5
foam
78.0+2.2
94. 611..5
88.920.0
79 11.88:1.2
840.7
43.5+:1.7
MVTec3D
peach
93.8411.
98 D1009
94.910.8
99.2+0.1
99, 8a0.0
62.51:4.4
potato
67.748.5
89.74336
89.9=1.0
98, 1+0.3
99. 540.1
42.5+4.6
rope
50,420.2
tire
75.3+5.8
92.5&1.8
87.9=0.4
91.920.3
98.2+0.1
46.4
Mean
83.011.5
95.0200.6
91.720..4
94.720.2
98.420.1
50.3+2.4
breakfast_bog
77.112.8
87.111.7
79.3&1.8
67.121.6
79.7&1.2
32.340.9
juice bottle
50.8+2.1
74.6+119
83.510.3
71.4+0.5
85.940.4
30.8
pushpins
62.710.6
66.4111.4
53.121.4
54.341.4
3.120.1
MVTecLOCO
screw bug
59.7&1.1
75.8
78.540.3
67.8
63.7&1.2
10.840.1
splicing connectors
76.9+0.4
65.743.5
Mean
63.3+1.2
76.3+11.0
78.4+0.5
66.6+0..
69.9t110
19,4+00.
01
85 9+3.0
95.0+1.1
86 96.0
65.4+1.7
93.3+1.00
50.9+2.2
02
93.664
99.040.2
95.6+0.6
68.4+5.8
96.940.4
64.6433.2
BTAD
03
99 840.0
96.740.5
93.242.1
97.3+0.3
99.5+0.0
65 1.6
77.1=2.0
bracket,
66.6446.5
92.0&1.2
95.340.6
2
brocket_browrg
57 37..644.2
75.0+3.8
81.6+0.7
87.0+1.3
94.9+0.2
10.9+0.3
bracket white
76.242.8
68.947.3
84.6+2..1
95.9+0.7
99.740.0
24.744.1
87.143.2
70.946.
76.4+3.9
91.2+1.5
97.450.5
28.8+4.77
MPDD
metal_plate
motal plate
99.940.0
100.0=0.0
99. 30.30000
89.7+0.9
96.540.5
73.2
tubes
95.520..5
Mean
78.5+0.8
79.9+1.1
85.0=0.5
91.94006
97.1+0.2
37.542.5

---

Table 10: The comprehensive results our NAGI. framework, evaluated under settings with N A
Tmage-level
Pixel-level PRO
Dataset
Objects
AUROC
Fi-max
AUROC
Fi-max
bottle
99.2+1.0
99.7+0.4
99.5+0.5
98.2=0.2
96.0+0.5
76.5+I
cable
93.7+0.4
96.6t0.3
90.8=0.5
94.7=0.1
88.5+0.1
60.0+1.4
capsule
90.8+8.3
97.8+2.
95.4+1.3
98. 1=0.1
96.8+0.5
47 47.4+5.2
earpet
100.0=0.0
100.0=0.0
100.0=0.0
99.3=0.0
98.5+0.0
68.940.4
grid
99.920.1
100.0=0.0
99.720.5
99.420.0
97.0+0.0
51.340.8
hazelnut
840.
99.920.1
99.3
99.6.0.1
97.4+0.4
79.8
leather
100.0+0.0
100.0=0.0
100.0+0.0
99. 1=0.0
98.4+0.3
41.8+1.0
metal nut
99 8+t.
99. 90.900.0
99.3+0.3
95.5=0.3
93.5+0.5
70.2
MVTecD
95.5+1.6
120.4
96.620.2
94.3=0.3
06.220.2
54.4+1.1
screw
86,0+0.9
95.2+0.4
88.6
98. 28..
93,5+0.3
48 8+0
99.9+0.1
100.0=0.0
99.6=0.3
96. 1=0.3
91,4+0.4
69.940.4
toothbrush
99.420.7
99.7s0.3
98.421.6
99.2s0.1
95.540.2
65.542.0
transistor
88.0s4..
80.8s4.6
79.2=4.7
85.2s1
66.0&1.8
39.044.0
wood
99.840.1
99.9+0.0
99.2+0.0
96.4=0.8
95.9+0.3
68.8+1.4
zipper
99.5+0.2
99.9+0.0
98.920.2
98.0=0.2
94.14
54.7+0.7
Mean
96.8+01.8
97.9+10.4
96.3+0.5
96.8=0.2
93.2+0.2
59.8+0.3
candle
93.740.7
94.2s0.5
87.920.5
09.000.0
97.120.2
37.9&1.2
capsules
96.9+1.0
98.2s0.5
93.4
97.9=0.2
96.540..4
48.0433.7
cashew
90.143.6
9..3+1.7
88.9=2.6
98.7
97.8+0.8
57.2+0.5
chewinggum
97.7+0.5
99 0s0.
95.5+0
99. 99.3=0.1
15.1
68. +
fryum
95.920.1
94.020.2
91.12003
macaroni
89.1+0.3
88.5+0.8
83,4+0.5
99.4=0.0
93.8+0.6
1..1
macaroni2
63.2+5.4
59.3+-4.3
68.8=114
98.3=0.2
86.7+0.4
17.34+3.8
pcbl
94,820.8
92.8&1.2
91.820.44
99.0=0.1
94.340.1
61.1&188
pch2
86.6+1.6
84.4+1.4
81.5+2.
95.7=0.1
86.5+0.1
36.940.8
pch3
88.9+1.1
91.2+0.7
83.1=1.8
93.9=0.1
87.3+0.0
44.1
pcb4
84.6210.7
87.4
80.0=9.6
95.2=0.5
83.2+4.0
29.7+1.8
pipe_ fryufrymm
90.620.9
86.8=1.1
97.620.0
91.4=0.4
43.34003
BraTS
brain
82.1+3.0
96.621.0
93.920.1
96.820.2
79.920.4
51.811.5
bagel
95.642.6
98.910.7
95.3
99.0=0.1
99.6+0.0
6..8+1.0
aable_gland
95.421.1
91.021.2
98.420.5
48.5+2.3
carrul
87.1=1.9
96.910.5
92.4+1.2
99 1=0.1
99.8
55. $5.1+2.9
56.9+3.44
dowel
74.3++2.
92.4+1.3
90.6=0.5
94.8=0.4
98.8+0.1
40,7+2.6
foam
80.5+2.1
95.2:0.5
89.420.5
79.9=0.7
92.9+0.4
43.71006
MVTec3D
94.2+2.5
1.1
94.811.4
99.2s0.1
99.000.0
62.813.7
potato
67.4+2.8
89.7
89.5+0.4
98.2=0.1
99.6+0.0
42.743.3
rope
95.320.5
99440.00
$0.410.3
lire
69.3+112.0
89.2+6.2
87.6.0.3
91.7=0.9
98. 180.2
45.1
Mean
82.812.9
94.911.1
91.620.6
94.820.3
98.420.1
51.3+1.7
breakfast_ box
78.142.1
88.340.9
78.821.5
64.421.9
77.7&1.5
29.941.9
juice bottle
58.8+3.7
79.2s1.6
83.9+0.8
70.9=0.7
85.6+0.4
29 1.0
puslipinns
72.020.5
53,911.2
MVTeLLOCO
screw bug
60.0a2.7
76.2+2.0)
78.2+0.0
69.2:0.6
65.2&1.3
10.940.1
splicing connectoes
77.2+0.4
Mean
64.6+1.4
77.4+1.7
78.0=0.2
66.3:0.4
69.3+0.4
18.5+0.5
01
86.4#0.2
95.0t0.1 I
85.9=0.6 6
65 65.6+2.7
93.5+1.1
12.1
93.140.7
98.9+0.1
95.6+0.2
65.56220
96.940.1
6..2
BTAD
99.740.1
95.5
91.8+2.8
97.2=0.0
99.5+0.0
64.6+0.5
96.5+00.5
91.1=0.9
96.740.4
bracket,
96.5+0.6
26.5+338
bracket, brown
60.8+1.4
75.6++1.6
82.0=1.0
89.0=0.8
95.4+0.5
13.1+1.1
bracket white
81.1+6.7
77.1
84.7+1.0
96.5a1.3
99.840.1
23.144.5
connector
87,8+4.0
72.6+6.3
77.8+2.1
92.0=1
97.6+0.5
30.7+4.5
MPDD
matal_ plate
99.9 39.940.1
100.0=0.0
09.5=0.44
90.0=0.7
96.6t0.4
73.5+1.6
94.010.6
97.610..3
91.820.8
05.620.5
64.91008
81.4+2.9
82.743.0
85.6=0.3
92.9:0.5
97.5+0.1
38.741.9
25

---

Table 11: The comprehensive results our NAGI. framework, evaluated under settings with N A
Tmage-level
Pixel-level PRO
Dataset
Objects
AUROC
Fi-max
AUROC
Fi-max
bottle
99.844 99.840.2
99. 99.9+0.
99.2+0.0
98. 180.2
95.7+0.2
76.6+00.7
cable
94.7+0.7
97.2+0.3
91.2=0.5
95.2=0.3
89.4+0.8
61 0t
capsule
97.4+1.0
99.5+0.2
97.3+0.8
98.4=0.0
97.3+0.1
52.4+0.4
earpet
100.0=0.0
100.0=0.0
100.0+0.0
99.3=0.0
98.4+0.0
68.5+0.2
grid
99.810.3
99.920.1
99.720.5
99.420.0
97.0x0.1
51.3+0.9
hazelnut
8:0..
99.920.1
99.3
99.6.0.1
97.540.77
79.7&1.5
leather
100.0=0.0
100.0=0.0
100.0=0.0
99. 1=0.0
98.2+0.3
41,0+1.1
metal nut
99 8+0.00
100.0=0.0
0
96.0=0.3
94.0+0.1
72.6tt
MVTecD
95.3&1.4
99.
96.320.5
95.3=0.2
$5.020.9
screw
84,4+442
94.5+1.8
88.4+0
98.3=0.3
93.8+0.7
100.0=0.0
100.0=0.0
99.6=0.3
96.0=0.3
91,2+0.2
69.5400.5
toothbrush
99.8a0.2
99.910.1
98.920.9
99.2s0.0
95.540.5
64.7a1.4
transistor
85.745.9
79.9
79.0=4.4
85.8=1
67.8&1.7
38.440.9
wood
99.840.3
99.9+0.
98.9=0.5
96.4=0.3
95.7+0.3
69.1
zipper
99.5+0.2
99.9+0.1
98.7=0.4
98.3=0.4
94,2+0.5
$4.7#0.6
Mean
97.1
98.0+11.5
96.4=0.2
97.0=0.0
93.5+00.
60.1
candle
94.0a0.4
94.420..
87.520.7
99.1=0.1
96.800.1
38 28.260.6
capsules
97.5+0.2
98.4s0.1
94.5
98. 1
96.8
48 6:1466
cashew
92.7+1.5
96.7+0.6
89.6=1
98.7=0.2
98.0+0.5
+0.1
chewinggum
97. 97.6650.1
09.0=0.00
95.5+0.3
99 10.220011
85.7+1.2
68. +0.8
fryum
2:0.2
90.84966
41.720.7
macaroni
89.5+2.
88.8+119
83.5+0.9
99.4 99.4=0.1
93.6++0.5
26.5+12
macaroni2
66,2+5.0
63. 63.214488
70.0=1.1
98.5=0.1
87.6+0.2
22.341.8
pcbl
95.410.7
93.3s0..8
92.720.8
98.920.1
94.240.1
60.422.0
pch2
88.4+1.1
85.8+0.8
82.9=0.6
95.7=0.1
86.5+0.2
38.5
pch3
89.0+1.4
91.6+1.1
83.1+
95.2=0.5
87.4+0.3
42.5+3.9
pcb4
91.0H335
91. 11.922.7
86.1=4.0
95.3=0.3
83.1+1.3
30.3++1.0
pipe_ Pryuf
97.121.2
93.721.3
91.2+1.1
91.7:0.8
87.8=0.9
97.8:0.0
91.5++02
44.01:0.6
BraTS
brain
84.911.7
97.3+0.4
94.1.0.1
97.1=0.4
80.740.5
54.7+2.8
bagel
96.0#1.9
99.0s0.5
95.8=1.4
99.0=0.1
99.740.0
67. 67.0+33.2
aable_gland
91.520.7
92.9=1.5
1=0.2
99.210.1
56.310.4
carrul
90.1#1.1
97.7+0.3
93.820.4
99.3.0.1
99.8
$7.740.5
97.7+0.0
dowel
83.4+t4.9
95.4+:2.1
90.3+1.0
95.8=0.6
46 66.0+6.4
foam
79.9+5.0
95.1:1.2
90.1=1.0
79.7=0.7
92.7+0.3
43.9+1.4
MVTec3D
96.8a.1.5
99.2s0.3
96.3a1.2
99.4s0.0
94.840.0
69.010.6
potato
68.7+3.2
90.9+0.7
89.6=0.2
98.2=0.2
99.6+0.0
13.9&1.7
rope
lire
80.4s4.8
94.2+1.3
88.6&1.3
93.2=0.2
98.5+0.1
48.5+0.4
Mean
86.911.8
96.420.5
92.3e0.7
95.420.0
98.610.0
54,210.4
breakfast_ box
79.5+2.2
89.121.0
79..21.7
64.921.1
77.7&1.0
30.6110.8
juice bottle
75.0+7.4
88.515.4
84.1
70.2=0.2
85.240.2
28.340.6
puslipinns
65.018.2
74.523.0
MVTeLLOCO
screw bug
65.545.3
79.3+2.5
79.0&1.4
70.0=0.9
64.7+0.2
10.940.1
splicing connectoes
78.5+11.7
65.244.0
Mean
71.5+t.1
81.5+:22
79.1=0.8
66.9:0.6
69.2+0.5
18.6+t0.4
01
90.2+2.7
96.4+0.9
89.0=1.7
67.8=0.8
93.8+0.7
52.9+0.9
93.440.9
98, 18.940.2
95.7+0.2
64.3a1.7
96.940.1
6..9+116
BTAD
99.740.
96.3+
92.7+2.2
97.2=0.2
99 99.5+0.0
65.740.9
92.5+0.9
61.2400.4
bracket,
68.9
95.4=1
97.6+11.2
27.2433.0
bracket, brown
60.7+8.8
73.3+5.4
83.2+11.1
90.4=1.7
96.0+0.5
15.3+22.0
bracket white
72.6+a17.4
69.7420.9
82.2+6.2
96.5a1 4
99.8+0.1
28.243.6
connector
86.342.7
74.6+3.5
77.243.9
92.2=1.1
97.740.3
36.2+6.0
MPDD
matal_ plate
100.0=0.0
100.0=0.0
09.5=0.4
90.4=0.5
96.740.4
73.0&1.9
66.111..
80.7+3.0
82.0+33.
85.8=1.3
93.5+0.7
97.8+0.3
41.0+1.1
26


Reviews:
[{"review_id": "lnTwMh1xBF", "reviewer": "Reviewer_LrM6", "summary": "This paper proposes a novel generalist anomaly detection (GDA) framework, termed Normal-Abnormal Generalist Learning (NAGL), which leverages both normal and abnormal reference samples during training and inference to reflect more realistic application scenarios where limited anomaly examples may be available. The framework integrates two key components: Residual Mining (RM), which extracts transferable anomaly representations from the residuals between normal and abnormal references, and Anomaly Feature Learning (AFL), which identifies discriminative anomaly features in query images by comparing them against these residual proxies. Designed for cross-domain generalization, NAGL achieves strong performance across diverse datasets such as MVTecAD, VisA, and BraTS. Extensive experiments, including ablation studies and efficiency analysis, demonstrate its effectiveness and computational efficiency, highlighting its potential for practical anomaly detection applications.", "questions": "1. How does the performance vary when abnormal references are from different defect types than the query (e.g., inter-type transfer)? Is there any observed instability or degradation?\n\n2. While BraTS provides a cross-domain test, have you evaluated zero-shot performance on unseen object classes in the same dataset? This would help assess the method’s generalization depth.\n\n3. AFL uses residual attention to avoid false activations. How does this behave in cluttered scenes or images with complex textures? Would AFL overfit to known anomaly shapes?", "limitations": "yes", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths: \n1. The paper is the first to propose GAD with both normal and abnormal reference samples, aligning better with practical settings where limited abnormal data may exist.\n2. The RM-AFL mechanism is well-motivated and shown to resolve issues of false activations when using abnormal references naively.\n\nWeaknesses\n1. The selection of abnormal references is random (per defect type), but its potential impact on performance variability is not well studied or controlled.\n2. The RM and AFL modules, while empirically strong, could benefit from clearer theoretical motivation or formal insight into their robustness or transferability.\n3. The method assumes access to both normal and abnormal references at inference time, which may not always hold in fully open-world applications.", "was_revised": true, "final_justification": "After the discussion, my concerns have been addressed. As the response covers many aspects that need to be improved, these should be carefully incorporated into the final version. I keep my score as \"borderline accept\".", "rebuttal": "**Dear LrM6,**\n\nThank you for reviewing our paper. Your insightful suggestions have significantly improved its clarity. We have addressed your comments one by one. Please do not hesitate to let us know if you have any further questions.\n\n_**Q1. Impact of random abnormal reference selection on performance variability.**_\n> **A1.** To clarify, our training strategy employs random sampling to construct input pairs (normal images, an abnormal image, and a query image), thereby enriching the diversity of our training samples. **For testing**, we maintain consistency by using the same abnormal reference for a given anomaly type. For instance, when evaluating the 'broken_large' anomaly type for the 'bottle' product in the MVTecAD dataset, we select the first image as the abnormal reference and apply it to test the remaining data. This approach ensures that our testing benchmark is well-controlled. We will also provide the list of reference files used during testing when we release our code. We also provide a detailed description of our dataset structure in the **Appendix A**. Please refer to \"Dataset Collection\" (lines 451-498) to pursue a clear understanding. \n\n> Moreover, the paper rigorously evaluates performance variability through statistical reporting: results are averaged across multiple independent runs with different random seeds (Lines 238-239), and detailed subset-level results include standard deviations (e.g., $mean \\pm std$ in **Appendix, Tables 9–11**). For comparative purposes, we present the variability of different methods in **Table R1**. WinCLIP and InCTRL performance data are sourced from Tables [1, 4] and Table [1] of their original papers, respectively. The results indicate that our method is more stable, exhibiting a smaller standard deviation.\n\n**Table R1. Comparison of stability ($mean \\pm std$) under $N^2(+A^1)$**\n\n|Dataset$\\rightarrow$|MVTecAD|MVTecAD|VisA|VisA|\n|:-:|:-:|:-:|:-:|:-:|\n|Method$\\downarrow$|I-AUROC|P-AUROC|I-AUROC|P-AUROC|\n|WinCLIP|94.4±1.3|96.0±0.3|84.6±2.4|96.8±0.3|\n|InCTRL|94.0±1.5|-|85.8±2.2|-|\n|Ours|96.8±0.8|96.8±0.2|89.8±1.5|97.6±0.0|\n\n_**Q2. Clearer theoretical motivation or formal insight for RM and AFL modules.**_\n> **A2.**  The RM and AFL are two attentions. In attention mechanisms, the Query and Key are initially used to compute the attention map. Then, this map is used to weight and combine the Value, meaning the output of the attention mechanism primarily represents the Value.\n\n> In the first attention (RM), we set the value as $\\text{Res}(\\mathcal{F}^a,\\mathcal{F}^n)$, which provides the \"normal-abnormal differences\" in the residual space.  We define the key as $\\mathcal{F}^a$, which are the original abnormal patterns. Since the query is **learnable**, the output is the **optimally** aggregated residual features (termed as residual proxies $\\tilde{\\mathcal{P}}$).\n\n> The second attention (AFL) aims to learn query-related abnormal patterns by comparing \"normal-abnormal differences\" and \"normal-query differences\". We set the query as $\\tilde{\\mathcal{P}}$ and the key as $\\text{Res}(\\mathcal{F}^q,\\mathcal{F}^n)$. This \"comparison\" is achieved by computing the attention map between $\\tilde{\\mathcal{P}}$ and $\\text{Res}(\\mathcal{F}^q,\\mathcal{F}^n)$. Then the query features $\\mathcal{F}^q$ are aggregated by the attention map to obtain the query-related abnormal patterns (termed as anomaly proxies $\\hat{\\mathcal{P}}$). \n\n> Detailed analysis of transferability in residual space: As validated in **Appendix B**, features in the original visual space cluster by anomaly type (**Figure 5 a**), indicating strong visual correlations that impede cross-domain learning. Conversely, features in the residual space (**Figure 5 b**) do not exhibit such clustering; instead, they converge to a common distribution. This demonstrates that residual features effectively reduce type-specific information, enhancing their transferability.\n\n_**Q3. Assumption of abnormal reference access at inference time.**_\n> **A3.** As discussed in Lines 34-35, real-world scenarios often present a limited number of anomalous samples (e.g., defective parts or diagnosed disease cases), which originates from our direct collaboration with factories. More specifically, in industrial production, the scarcity of anomalous samples is common during the initial production phase. However, as production increases, anomalous samples become more readily available. The practical relevance of this challenge is further highlighted by the VISION'24 Data Challenge, organized by ECCV 2024, which focused on industrial defect segmentation with limited anomalous samples. The ability to leverage anomaly samples to enhance detection performance is highly desirable for many factories, and this paper addresses precisely this scenario. Our method also provides preliminary evidence that a small number of anomalous samples hold significant potential for achieving both speed and accuracy. Furthermore, reviewer QNYq positively evaluated this problem, stating that \"this paper considers an interesting and very valuable task.\" Therefore, we believe this setting is valuable for research and application\n\n_**Q4. Performance when abnormal references are from different defect types.**_\n> **A4.** Our experiments already include inter-type scenarios, as detailed in **Appendix A**. The VisA dataset combines all anomaly samples for each product. This can lead to situations where the input query and the anomalous reference display different types of anomalies (**Figure 4 b**). Nevertheless, our method demonstrates an advantage over ResAD in the experimental results (**Table 1**).\n\n> For a clearer comparison, we conducted further experiments (**Table R2**) on the MVTecAD dataset. We tested \"hazelnut\" and \"carpet\" products under the $N^1+A^1$ setting, using 'crack' and 'hole' as anomaly types. Results show that while the inter-type setting causes slight degradation, our method still outperforms ResAD (which lacks abnormal references), even with different anomaly types. This aligns with our ***Q2*** discussion, indicating the transfer capacity of these residual features.\n\n**Table R2. Comparison of different defect types as abnormal references**\n\n|Method|Test Type|Abnormal Reference Type|I-AUROC|P-AUROC|\n|:-:|:-:|:-:|:-:|:-:|\n|ResAD|crack from hazelnut|-|90.2|96.7|\n|Ours(intra)|crack from hazelnut|crack|98.9|99.2|\n|Ours(inter)|crack from hazelnut|hole|98.8|99.2|\n|Ours(inter)|crack from hazelnut|print|98.6|99.0|\n|ResAD|hole from carpet|-|90.2|96.7|\n|Ours(intra)|hole from carpet|hole|100.0|99.5|\n|Ours(inter)|hole from carpet|color|99.1|98.4|\n|Ours(inter)|hole from carpet|cut|99.8|99.2|\n\n_**Q5. Evaluation of zero-shot performance on BraTS.**_\n> **A5.** Both zero-shot methods [r1] and GAD [52, 61, Ours] train models on one domain (e.g., MVTecAD) and test on another (e.g., VisA or BraTS) to achieve cross-domain generalization. Thus, when BraTS is used as the test set, the training set is MVTecAD, and all classes in BraTS are unseen. Unlike zero-shot methods, which directly test samples without needing reference samples, GAD requires reference samples during testing. For comparison, we report the zero-shot performance of AnomalyCLIP[r1] on BraTS, as shown in the **Table R3**. The results indicate that incorporating references (e.g., ResAD and our methods) improves detection performance, demonstrating the generalization capability of these methods.\n>\n**Table R3. Comparison with zero-shot methods  on BraTS**\n\n|Method (reference number)|I-AUROC|P-AUROC|avg.|\n|:-:|:-:|:-:|:-:|\n|AnomalyCLIP (None)|69.3|93.2|81.3|\n|ResAD ($N^1$)|73.5|91.0|82.3|\n|Ours ($N^1+A^1$)|78.1|96.5|87.3|\n \n***Q6. AFL's behavior in cluttered scenes and potential for overfitting.***\n> **A6.** According to [1], the MVTecAD dataset comprises both texture and object images. To evaluate the detection capabilities in complex texture scenarios, we specifically analyzed the texture image products from the MVTecAD dataset (carpet, grid, leather, tile, wood). The results are presented in **Table R4**. (Detailed subset-level results are provided in Appendix G for reference.) For comparative purposes, we also included the product-wise performance of PromptAD [25] (drawing data from Tables 11 and 12 of the original paper). The comparison indicates that our method does not exhibit a disadvantage in detecting texture images, even though PromptAD requires training a specific model for each product. \n\n**Table R4. Performance Comparison of Texture Images**\n\n|Products|Ours (I-AUROC)|Ours (P-AUROC)|PromptAD (I-AUROC)|PromptAD (P-AUROC)|\n|:-:|:-:|:-:|:-:|:-:|\n|carpet|100.0|99.3|100.0|95.9|\n|grid|99.8|99.4|99.8|95.1|\n|leather|100.0|99.1|100.0|93.3|\n|tile|99.8|96.1|100.0|92.8|\n|wood|99.8|96.0|97.9|89.4|\n\n> Additionally, while residual features offer some cross-domain capability, the risk of overfitting remains. We observed this phenomenon during training, where training for more epochs negatively impacted model performance to some extent. We mitigated the impact of overfitting by setting fewer epochs (This technique is also used by AnomalyCLIP[r1]) and minimizing the number of training parameters, as described in Appendix F. Since figures cannot be provided here, we will supplement the performance curves regarding training epochs in the main text.\n\n*[r1] AnomalyCLIP: Object-agnostic Prompt Learning for Zero-shot Anomaly Detection. ICLR 2024*"}, {"review_id": "v8rPCSpI3w", "reviewer": "Reviewer_QEzS", "summary": "This paper addresses the problem of Generalist Anomaly Detection (GAD), detecting anomalies in new target domains with models trained on a source domain. To this end, the authors propose a new framework, NAGL, including two key components, Residual Mining (RM) and Anomaly Feature Learning (AFL). Given an abnormal reference, RM obtains residuals between normal and abnormal samples to learn abnormal patterns. AFL learns abnormal features of a query image based on the learned abnormal patterns. NAGL compares abnormal patterns between the query and reference images to detect anomalies. The authors demonstrate the effectiveness of NAGL on standard benchmarks, and show an expensive analysis.", "questions": "Please refer to the strengths and weaknesses section for the details.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\n(+) NAGL incorporates abnormal references along with the normal ones, enabling effective anomaly detection across diverse domains. \n\n(+) The proposed method is highly practical due to its lightweight design and achieves faster training and inference than the previous works [51, 62].\n\n(+) The proposed method shows strong cross-domain generalization performance (e.g., MVTecAD → VisA, MVTecAD → BraTS). \n\n\n**Weaknesses**\n\n(-) The proposed method relies heavily on abnormal references. I am thinking that this might limit the applicability in certain real-world scenarios, where only normal data is available, making it less practical than the works of [51, 62].\n\n(-) The authors set 𝐾_1 ≥ 𝐾_2 assuming a real-world scenario where abnormal samples are scarce. The inference setting on MVTecAD, however, seems to contradict the assumption. Specifically, if K_1=1 and K_2=1, the number of normal and abnormal references are 15 and 73, respectively. Please clarify this. \n\n(-) The authors demonstrate that the proposed method generalizes well in cross-domain settings, such as the MVTecAD ⟶ BraTS scenario. However, the source domain is limited to MVTecAD, which contains a variety of abnormal references. It would be helpful if the authors could provide more results using other source domains.\n\n(-) Please clarify the experimental settings (iii) in Table 3. I am wondering why (iii) shows inferior performances than (i) although (iii) exploits abnormal samples.\n\n(-) The author shows in Table 3 the effectiveness of the proposed method that leverages abnormal references for GAD. However, it seems that NAGL adopts different backbone ones from [52, 61]. It would be more convincing if the authors provide additional results using the same backbone in [52, 61]. This would help to establish the validity of the proposed method more convincingly.\n\n(-) There are some typos.\nL82: augmentation-based -> and augmentation-based\nL148: pervious -> previous\nCaption of Figure1: cause -> causes\nCaption of Table 3: represent -> represents, merge -> merges", "was_revised": true, "final_justification": "Thank you for the rebuttal. The major concerns, regarding abnormal references, experimental settings, and the effectiveness of the methods, have been sufficiently addressed. Considering both the questions raised by other reviewers and your response, I will raise the rating accordingly. I have no follow-up questions.", "rebuttal": "**Dear QEzS,**\n\nThank you for reviewing our paper. Your insightful suggestions significantly improved its clarity. We address your questions individually below and are available to clarify any further questions you may have.\n\n***Q1. The applicability of using anomalous samples in real-world scenarios.***\n> **A1.** Our task significantly differs from [51, 62]. Therefore, we guess the reviewer is interested in a discussion comparing the practicality of ours with [52, 61]. While current mainstream research [52, 61] primarily focuses on scenarios where only normal data is available, our collaborations with factories reveal that the presence of a small number of anomalous samples is often more common. This is because, as production progresses, some anomalous samples or new anomaly types can always be collected. Additionally, the VISION'24 Data Challenge, organized by ECCV 2024, focuses on industrial defect segmentation using a limited number of anomalous samples (more details are available on their official website). Over 80 teams worldwide participated, underscoring the practical applicability of addressing industrial defects with minimal anomalous data.\n> Driven by real-world industrial needs, we propose this task that utilizes a limited number (e.g., even just one) of anomalous samples. Our method also provides preliminary evidence that a small number of anomalous samples hold significant potential for achieving the goal of both speed and accuracy. Reviewer QNYq positively evaluated this problem, stating that \"this paper considers an interesting and very valuable task.\" Therefore, we believe this setting has both research and application value. \n\n***Q2. The number of normal and abnormal references.*** \n> **A2.** It should be noted that the MVTecAD dataset distinguishes between different anomaly types, containing $15$ products and $73$ anomaly types. When $K_1=1$ and $K_2=1$, based on the product category and anomaly type of the query sample to be tested, we select $1$ normal sample of the corresponding product as a normal reference and $1$ abnormal sample of the corresponding anomaly type as an abnormal reference. The numbers $15$ and $73$ here indicate that there are a total of $15$ normal references and $73$ abnormal references for the MVTecAD dataset, but it does not mean that we use all references for each query sample. Additionally, for the VisA dataset, which does not distinguish anomaly types, we select $1$ normal reference and $1$ abnormal reference for each product. Therefore, there are $12$ normal references and $12$ abnormal references in the VisA dataset. This strategy for selecting normal references is consistent with existing few-normal-shot AD methods (e.g., PromptAD [25], InCTRL [61], and ResAD [52]).\n\n***Q3. Evaluation on different training datasets.***\n> **A3.** Training on MVTeAD and testing on datasets such as VisA and BraTS, is to align with previous works [r1, 52, 61], facilitating comparison and analysis with these existing studies. Furthermore, to further validate the cross-domain capabilities, we conducted additional experiments, as shown in the **Table R5**. In this experiment, we used VisA and BTAD as training sets. These datasets contain less diversity with fewer products and do not distinguish between anomaly types. Specifically, VisA has $12$ products ($962$ normal samples and $1200$ abnormal samples), while BTAD has only $3$ products ($176$ normal samples and $282$ abnormal samples). \n\n> **Table R5** shows that training on VisA did not result in significant performance degradation. However, performance degradation is observed when training on BTAD for both our method and ResAD. This is likely due to the limited data volume in BTAD, which may hinder the models from adequately capturing the \"differences\" between normal and anomalous samples. It is noteworthy that our method experienced a smaller performance decrease compared to ResAD. And we also report the performance on other datasets, including MVTec3D, MVTecLOCO, BTAD, and MPDD, in the **Appendix Table 8-11**, which indicates our method has good cross-domain capabilities.\n\n**Table R5. Evaluation on different training datasets.**\n\n| Method | Training Dataset | Test Dataset | I-AUROC | P-AUROC | $\\Delta$  |\n| :----: | :--------------: | :----------: | :-----: | :-----: | :-------: |\n| ResAD  |     MVTecAD      |    BraTS     |  73.5   |  91.0   |           |\n| ResAD  |       BTAD       |    BraTS     |  71.8   |  89.6   |   - 3.1   |\n|  Ours  |     MVTecAD      |    BraTS     |  78.1   |  96.5   |           |\n|  Ours  |       VisA       |    BraTS     |  77.6   |  96.7   |   - 0.3   |\n|  Ours  |       BTAD       |    BraTS     |  77.2   |  95.8   | - 1.6 |\n\n\n> *[r1] AnomalyCLIP: Object-agnostic Prompt Learning for Zero-shot Anomaly Detection. ICLR 2024*\n\n***Q4. Clarification of settings in Table 3.***\n> **A4**. We will now explain the different settings in Table 3 individually.\n>\n> **setting (i):** As discussed in Section 4.3, we apply a nearest neighbor (NN) search between query features ($\\mathcal{F}^q$) and normal reference features ($\\mathcal{F}^n$), as detailed in **Equations (8,9)**.\n>\n> **setting (ii):** Similar to setting (i), we also apply the NN search between query features ($\\mathcal{F}^q$) and abnormal reference features ($\\hat{\\mathcal{F}}^{a}$). It is important to note that $\\hat{\\mathcal{F}}^{a}$ is a subset of $\\mathcal{F}^{a}$,  and exclusively comprises abnormal patch features from $\\mathcal{F}^{a}$. This process yields an additional anomaly score map derived from the abnormal reference, which can be formulated as:\n> \n> $f^a_* :=\\mathop{\\mathbf{argmin}}\\_{f^{a} \\in \\hat{\\mathcal{F}}^{a}}\\left(\\mathbf{d}(f\\_i^{q},f^{a})\\right),$\n> \n> $\\mathcal{S}^{i}_{a}=1-\\mathbf{d}(f^q_i, f^a _*).$\n> \n> **setting (iii)** is the result of merging settings (i) and (ii), such that:\n>\n> $\\mathcal{S}^i\\_{\\text{(iii)}} = \\frac{1}{2}(\\mathcal{S}^i_{n}+\\mathcal{S}^i_{a}).$\n> \n> Setting (ii) indicates that simply using a Nearest Neighbor (NN) search between query and abnormal features leads to severe over-activation (visualization results can be found in Figure 1(c)). This over-activation causes setting (iii) to perform worse than setting (i).\n\n***Q5. Verify the effectiveness of the proposed method more convincingly.***\n> **A5.** *(It should be noted that the results in Table 3 do not use different backbones; all are based on the ViT-based [41] backbone. Next, we will primarily discuss the backbones in Tables 1 and 2.)* As described in Lines 244-245, ResAD[52] only reported results for 2/4-shot settings using AUROC metrics. To provide a more comprehensive comparison, we reproduced their results using the official ResAD code. Training ResAD is time-consuming; for instance, as shown in **Table 4**, the CNN-based backbone alone requires 20 hours of training. Training with the official ViT-based backbone was estimated to take more time. Therefore, to meet the submission deadline, we initially only provided reproduction results for the CNN-based architecture. For a fair comparison, we also included a comparison with ViT-based methods in **Appendix D**, with results presented in **Table 7**. Furthermore, for a more rigorous comparison, we also reproduced ResAD with the DINOv2 ViT-S [41] backbone used in our method, and the results are shown in **Table R6-1/2**. *(Since InCTRL[61] uses text prompts, implying its reliance on a CLIP-based backbone, we maintained the original InCTRL backbone when reporting our results.)*  **Table R6-1/2** indicates that our method maintains its advantage.\n\n**Table R6-1. Different Backbone Comparison under $N^2(+A^1)$**\n\n|Method|backbone|Total Parameters (M)|MVTecAD I-AUROC|MVTecAD P-AUROC|VisA I-AUROC|VisA P-AUROC|BraTS I-AUROC|BraTS P-AUROC|\n|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|InCTRL|CLIP ViT-B|117.5|94.0|-|85.8|-|74.6|-|\n|ResAD|Imagebined_huge|442.6|94.4|95.6|84.5|95.1|67.9|94.3|\n|ResAD$^\\ast$|WideResNet50|59.2|87.2|94.8|86.6|96.5|66.2|91.5|\n|ResAD$^\\ast$|DINOv2 ViT-S|82.4|90.3|95.3|85.7|96.2|66.4|93.5|\n|Ours|DINOv2 ViT-S|24.4|96.8|96.8|89.8|97.6|82.1|96.8|\n\n**TableR6-2.Different Backbone Comparison under $N^4(+A^1)$**\n\n|Method|backbone|Total Parameters (M)|MVTecAD I-AUROC|MVTecAD P-AUROC|VisA I-AUROC|VisA P-AUROC|BraTS I-AUROC|BraTS P-AUROC|\n|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|InCTRL|CLIP ViT-B|117.5|94.5|-|87.7|-|76.9|-|\n|ResAD|Imagebined_huge|442.6|94.2|96.9|90.8|97.5|84.6|96.1|\n|ResAD$^\\ast$|WideResNet50|59.2|90.7|95.8|89.3|96.8|74.9|94.3|\n|ResAD$^\\ast$|DINOv2 ViT-S|82.4|93.5|96.5|89.0|96.5|74.8|94.1|\n|Ours|DINOv2 ViT-S|24.4|97.1|97.0|91.2|97.8|84.9|97.1|\n\n***Q6. Some typos***\n> **A6.** We have corrected typos in the main text and carefully reviewed all statements."}, {"review_id": "q2pxDIMF4f", "reviewer": "Reviewer_RxZT", "summary": "This paper focuses on generalist anomaly detection that can detect anomalies in new target domains. The proposed method leverages both normal and anomalous samples as references to guide anomaly detection across diverse domains. Experiments are conducted on several popular datasets.", "questions": "See the weakness.", "limitations": "See the weakness.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "1. The paper lacks important citations. In few-shot anomaly detection, the utilization of limited anomalous data has been extensively studied [a,b,c]. The authors should adequately cite these papers and compare their method with them. In the current experiments, none of the competing methods use anomalous data, while the proposed method requires it. This comparison is unfair.\n2. The authors define an original data domain and a target domain. However, both data domains here consist of industrial defect detection data without fundamental differences. The authors should refer to MVFA [a] and adopt medical imaging data as the target domain to demonstrate the generalization capability of the proposed method.\n\n[a] Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Images. CVPR 2024.\n[b] Catching both gray and black swans: Open-set supervised anomaly detection. CVPR 2022.\n[c] Explicit boundary guided semi-pushpull contrastive learning for supervised anomaly detection. CVPR 2023.", "was_revised": true, "final_justification": "The authors have addressed part of my concerns about the comparison with existing methods.", "rebuttal": "**Dear RxZT,**\n\nThank you for reviewing our paper. We appreciate your suggestion to strengthen our citations and comparisons, as this helps contextualize our work. We address each of your questions below and are available to answer any further questions promptly.\n\n***Q1. Lack of citations.***\n> **A1.** We will include references to [a] in our main text, while [b, c] have already been cited (please refer to [11, 53]). Since the settings of [a, 11, 53] differ significantly from ours, it is difficult to compare our methods with theirs. Instead, our method is more similar to [52, 61]; therefore, we primarily discussed [52, 61] in the main text. Additionally, for clearer comparison, we will now detail the differences between our method and those in [a, 11, 53]:\n> \n> **Comparison with [a].** The method described in [a] is specifically designed for medical image detection. It involves transferring knowledge from one medical dataset to another, not from an industrial dataset to a medical one, as we do in **Table 2**. Additionally, to directly compare the differences, we provide some experimental results in ***Q2***.\n>\n> **Comparison with [11, 53].** The method described in [11, 53] focuses on _open-set supervised anomaly detection (OSAD)_, which has two experiment protocols, including multi-class and one-class settings. For the multi-class setting, a few labeled anomaly samples are randomly drawn from all possible anomaly classes in the test set per dataset. These sampled anomalies are used for training, and then removed from the test data. For the one-class setting, the anomaly example sampling is limited to one single anomaly class only, and all anomaly samples in this anomaly class are used for training and removed from the test set to ensure that the test set contains only unseen anomaly classes. \n\n> This means that these methods [11, 53] require training a specific model for each dataset for anomaly detection, which is different from our method. Our method applies meta-learning on the original domain to facilitate cross-domain transfer (**no training on target domains**). \n \n> **Table R7** presents a comparison of our method with methods in [11] and [53]. *(The results for [11] are obtained from Tables 1 and 2 of their original paper, while the results for [53] are obtained from Tables 1 and 4 of their original paper. And our subset-level results are reported in **Appendix G**)*. Through comparison, we found that our method performs comparably to OSAD on \"carpet\" data and exhibits a slight performance decrease on \"metalnut\" data. However, our method remains good practical applicability due to its domain-agnostic ability.\n\n**Table R7. Comparison with OSAD methods**\n\n|   Method   |       Setting       | Test Product |    Training anomaly types    | I-AUROC | P-AUROC |     Model Type      |\n| :--------: | :-----------------: | :----------: | :--------------------------: | :-----: | :-----: | :-----------------: |\n|  DRA [11]  |     multi-class     |    carpet    | random sampling from MVTecAD |  94.0   |    -    |   domain-specific   |\n|  DRA [11]  |      one-class      |    carpet    |  select one type in carpet   |  93.5   |    -    |   class-specific    |\n| BGAD [53]  |     multi-class     |    carpet    | random sampling from MVTecAD |  99.6   |  98.9   |   domain-specific   |\n| BGAD [53]  |      one-class      |    carpet    |  select one type in carpet   |  99.9   |  99.5   |   class-specific    |\n|    Ours    | general ($N^1+A^1$) |    carpet    |             VisA             |  100.0  |  99.3   | **domain-agnostic** |\n|  DRA [11]  |     multi-class     |   metalnut   | random sampling from MVTecAD |  99.7   |    -    |   domain-specific   |\n|  DRA [11]  |      one-class      |   metalnut   | select one type in metalnut  |  94.5   |    -    |   class-specific    |\n| BGAD [53]  |     multi-class     |   metalnut   | random sampling from MVTecAD |  99.6   |  97.0   |   domain-specific   |\n| BGAD [53]  |      one-class      |   metalnut   | select one type in metalnut  |  99.8   |  97.5   |   class-specific    |\n|    Ours    | general ($N^1+A^1$) |   metalnut   |             VisA             |  99.4   |  95.2   | **domain-agnostic** |\n\n\n***Q2. Evaluation in the medical domain.***\n> **A2.** As described in **Lines 32-41**, our study employs a cross-domain setting and proposes a meta-learning approach for anomaly detection across diverse domains. We also validate the cross-domain transfer capabilities from industrial to medical applications in **Table 2** and **Figure 3**. Following the reviewer's suggestion, we compared our method with MVFA[a] on additional medical datasets (LiverCT and RESC), as shown in the **Table R8**. Results indicate our method demonstrates strong competitiveness, despite being trained on MVTecAD (industrial domain $\\rightarrow$ medical domain). \n\n**Table R8. Comparison in the medical domain**\n\n|  Setting  |  Method  | BraTS (I-AUROC) | BraTS (P-AUROC) | LiverCT (I-AUROC) | LiverCT (P-AUROC) | RESC (I-AUROC) | RESC (I-AUROC) |\n| :-------: | :------: | :-------------: | :-------------: | :---------------: | :---------------: | :------------: | :------------: |\n|   $N^4$   | WinCLIP  |      67.3       |      93.2       |       67.2        |       96.8        |      88.8      |      96.7      |\n|   $N^4$   | MVFA [a] |        -        |        -        |       81.2        |       99.7        |      96.2      |      98.7      |\n| $N^4+A^1$ |   Ours   |      84.9       |      97.1       |       84.3        |       99.4        |      97.1      |      99.0      |"}, {"review_id": "GCvtMPBOVn", "reviewer": "Reviewer_QNYq", "summary": "This paper aims to tackle generalist anomaly detection (GAD). Previous GAD methods primarily use only normal samples as references, overlooking the valuable information in anomalous samples. To address this, this paper proposes a more practical GAD method by innovatively incorporating few -shot anomalous samples as references to guide anomaly detection across diverse domains. The proposed Normal-Abnormal Generalist Learning framework consists of two parts: Residual Mining and Anomaly Feature Learning. The method can effectively utilize both normal and anomalous references for more accurate and practical cross-domain anomaly detection.", "questions": "Please see Strengths And Weaknesses.", "limitations": "yes", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**[Strengths]**\n\n1. This paper focuses on a previously overlooked issue in GAD method: only considering to use few-shot normal samples as references without considering to effectively utilize information from few-shot abnormal samples. This is interesting and very valuable for practical usage. This paper proposes a good GAD method that can also effectively utilize abnormal samples when directly used in unseen product classes.\n\n2. The proposed method is simple but effective. The main innovative module in this paper is the RM-AFL module, the whole module is simply based on cross-attention to capture normal-abnormal differences in residual space. The finally learned anomaly proxies can represent the anomalous patterns in the query image, thus can highlight anomalous regions by further feature comparison.\n\n**[Major Weaknesses]**\n\n1. I think the anomaly classification in the title is not accurate because the model in the paper cannot generate classification labels for anomalous regions. Anomaly classification requires further classification of anomaly types based on detection, and the method in the paper only implements anomaly detection.\n\n2. In Introduction, the first point in contribution conclusion is not reasonable. “We propose a new generalist anomaly detection task and corresponding dataset”. Generalist anomaly detection has been proposed in previous papers, and this paper is only a further study. This paper still uses the existing datasets, only divides the data into new settings, and does not propose a new dataset. The main difference between this paper and previous papers is the consideration of using both normal and abnormal samples as references.\n\n3. Although the proposed method is simple and easy to follow, I think one main issue about the paper is that the method lacks corresponding explanations. Why the key and value in the first cross-attention are $\\mathcal{F}^a$ and $Res(\\mathcal{F}^a, \\mathcal{F}^n)$? Why the key and value in the second cross-attention are $Res(\\mathcal{F}^q, \\mathcal{F}^n)$ and $\\mathcal{F}^q$? Is this order necessary? What would happen if the order is swapped or different key and value are used? What do residual proxies and anomaly proxies represent? Do residual proxies represent abnormal patterns of reference abnormal samples in residual space? Do anomaly proxies represent abnormal patterns of the query sample in vision space? Why do we need attention in the residual space first? Can't we directly extract anomaly proxies from $\\mathcal{F}^q$ in the vision space?\n\n4. In Eq(5), the $\\mathcal{M}^{\\prime}$ is based ground-truth mask $\\mathcal{M}^a$. However, the ground-truth mask during testing is not known in advance. How is the Residual Mining part implemented during testing (how is the $\\mathcal{M}^\\prime$ part handled)?\n\n5. About implementation details. Is the RM-AFL module implemented with one layer? Or the RM-AFL module is implemented by stacking multiple layers. This critical detail is missing.\n\n6. About experiments. Why does PromptAD only appear in $N^1$ setting? Why only ResAD is reproduced? Is the ResAD reproduced based on the backbone (ViT-S) and 448x448 resolution used in your method (From appendix D, it seems that ResAD is reproduced based on wide-resnet50)? If for fair comparison, other methods should also be reproduced based on ViT-S and 448x448 resolution. You may need to reproduce ResAD based on the backbone used in your method and 448x448 resolution for more reasonable comparison. Also in Tab.2, the results of other methods are from ResAD paper, but why does ResAD need to be specifically reproduced. Additionally, the experiments in main text are not sufficient, you should conduct more cross-dataset experiments to validate your method (I see experiment results on MVTec3D, MVTecLOCO, BTAD, MPDD, it’s better to reorganize your paper to include these experiment comparison in main text).\n\n7. Strictly, your method does not just use an abnormal sample, as you randomly select a sample from each anomaly type. For a product class, there may be many anomaly types (e.g. Mvtec dataset), and the actual number of abnormal samples used may be even more than the normal samples. The more critical issue is that in practical use, we do not know the anomaly type in the query sample. In this case, how should we choose abnormal samples and which anomaly type should we choose from. You need to discuss how much influence it would have on the results if the anomaly type in the abnormal reference sample is different from the type in the query sample. And what influence does it have if multiple different anomaly types of abnormal reference samples are given to a query sample.\n\n**[Minor Weaknesses]**\n \nTypos: In line 118, $r^a$ misses the subscript k. In line 145, the shape of $\\mathcal{M}^a$ should be $\\mathbb{R}^{K_2L}$.\n\nI think the good explanations for Weakness 3 and influence about anomaly type in Weakness 7 are important. Weakness 4 and 6 also should be well addressed. Overall, I think this paper considers an interesting and very valuable task, and proposes a simple but effective method. The entire paper is easy to follow. However, due to the aforementioned weaknesses, I am currently unable to provide an accept. If my concerns can be effectively addressed, I am willing to raise my score.", "was_revised": true, "final_justification": "The authors' rebuttal can address my main concerns (please see my response to the authors), thus I decide to raise the final score for the apper.", "rebuttal": "**Dear QNYq,**\n\nThank you for reviewing our paper. We address your questions individually below. Please let us know if you have further questions. We will answer them promptly.\n\n***Q1. About the title.***\n> **A1.** \"Anomaly Classification and Segmentation\" in the title follows from [19], titled \"WinCLIP: Zero-/Few-Shot Anomaly Classification and Segmentation.\" In their introduction, they define image-level abnormality as classification. For a more precise expression, we are considering changing the title to \"Normal-Abnormal Guided Generalist Anomaly Detection\" *(if the rules allow)*.\n\n***Q2. About introduction.***\n> **A2.** We will change it to \"We introduce a new task that utilizes a mixture of normal and anomalous samples as references for generalizable anomaly detection and have constructed benchmarks based on existing datasets.\"\n\n***Q3. The explanations of two attentions in RM-AFL module.***\n\n> **A3.**  Yes, your understanding is correct, residual proxies represent abnormal patterns of reference abnormal samples in residual space, and anomaly proxies represent abnormal patterns of the query sample in vision space.  As validated in **Appendix B**, residual features exhibit a common distribution, even among residual features of different anomalies  (**Figure 5**). Residual features of known anomalies can provide references; if the residual feature of a query is similar to them, the corresponding visual feature is likely to be an anomaly. \n\n> In the first attention (RM), we set the value as $\\text{Res}(\\mathcal{F}^a,\\mathcal{F}^n)$, which provides the \"normal-abnormal differences\" in the residual space.  We define the key as $\\mathcal{F}^a$, which are the original abnormal patterns. Since the query is **learnable**, the output is the **optimally** aggregated residual features (termed as residual proxies $\\tilde{\\mathcal{P}}$).\n\n> The second attention (AFL) aims to learn query-related abnormal patterns by comparing \"normal-abnormal differences\" and \"normal-query differences\". We set the query as $\\tilde{\\mathcal{P}}$ and the key as $\\text{Res}(\\mathcal{F}^q,\\mathcal{F}^n)$. This \"comparison\" is achieved by computing the attention map between $\\tilde{\\mathcal{P}}$ and $\\text{Res}(\\mathcal{F}^q,\\mathcal{F}^n)$. Then the query features $\\mathcal{F}^q$ are aggregated by the attention map to obtain the query-related abnormal patterns (termed as anomaly proxies $\\hat{\\mathcal{P}}$). \n   \n> Based on the above analysis, we need to ensure that the **Value** of RM and the **Key** of AFL reside within the same space. Accordingly, we have included the experimental results as presented in the **Table R9**.\n>\n> **Experiment (I)** demonstrates that swapping the **Key** and **Value** in RM-AFL leads to a significant performance drop. This is because the output of RM-AFL is aggregation of  $\\text{Res}(\\mathcal{F}^q,\\mathcal{F}^n)$ (residual features), which are different from the original query features ($\\mathcal{F}^q$) when computing the score by **Equation (8)**. Similar phenomena are also observed in (II).\n>\n> **Experiment (III)** shows that using the original visual features as the **Key** and **Value** in RM-AFL yields satisfactory performance, but a gap still exists compared to (IV). This is because the transferability of the original visual space is limited (**Appendix B Figure 5**), making it difficult to effectively capture class-agnostic information for the final detection.\n\n**Table R9. Ablation study of RM-AFL design**\n\n||RM-*Query*|RM-*Key*|RM-*Value*|AFL-*Key*|AFL-*Value*|I-AUROC|P-AUROC|\n|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|(I)|$\\mathcal{P}$|$\\text{Res}(\\mathcal{F}^a,\\mathcal{F}^n)$|$\\mathcal{F}^a$|$\\mathcal{F}^q$|$\\text{Res}(\\mathcal{F}^q,\\mathcal{F}^n)$|93.1|93.9|\n|(II)|$\\mathcal{P}$|$\\text{Res}(\\mathcal{F}^a,\\mathcal{F}^n)$|$\\text{Res}(\\mathcal{F}^a,\\mathcal{F}^n)$|$\\text{Res}(\\mathcal{F}^q,\\mathcal{F}^n)$|$\\text{Res}(\\mathcal{F}^q,\\mathcal{F}^n)$|92.8|94.0|\n|(III)|$\\mathcal{P}$|$\\mathcal{F}^a$|$\\mathcal{F}^a$|$\\mathcal{F}^q$|$\\mathcal{F}^q$|94.4|95.1|\n|(IV) Ours|$\\mathcal{P}$|$\\mathcal{F}^a$|$\\text{Res}(\\mathcal{F}^a,\\mathcal{F}^n)$|$\\text{Res}(\\mathcal{F}^q,\\mathcal{F}^n)$|$\\mathcal{F}^q$|95.8|96.6|\n\n***Q4. About $\\mathcal{M}^a$.***\n> **A4.** $\\mathcal{M}^a$ denotes the label for an abnormal reference. When an abnormal reference is obtained, its corresponding mask is also acquired. This design originated from our collaboration with the factory. They believe it would be easy to provide a very limited number of abnormal references(e.g., even just one) and their corresponding masks. Additionally, the VISION'24 Data Challenge, organized by ECCV 2024, can also access a few masks of abnormal samples, further underscoring the practical applications of this design.\n\n***Q5. The layers of  RM-AFL module.***\n> **A5.** As discussed in ***Q3***, the output of RM-AFL is a query-related abnormal feature. The input of RM-AFL is a learnable proxy. The output cannot be used as input, making concatenating layers unreasonable. Additionally, while residual features offer cross-domain capability, the risk of overfitting remains (**Appendix F**). Therefore, the RM-AFL module is implemented with only one layer. As shown in **Table R10**, more layers lead to performance degradation.\n\n**Table R10. Comparing the number of RM-AFL modules**\n\n|RM-AFL|I-AUROC|P-AUROC|\n|:-:|:-:|:-:|\n|1|95.8|96.6|\n|2|95.7|96.3|\n|3|93.1|94.2|\n\n ***Q6. About experiments.***\n> **A6.**  We address these concerns as follows:\n> - We will include comparison of PromptAD under $N^2$ and $N^4$ in our main text. Although PromptAD requires training a specific model for each product, our method consistently demonstrates superior performance. **Table R11-1/2** presents comparisons for $N^2$ and $N^4$.\n> - As stated in **Lines 244-245**, ResAD only reported AUROC results for 2/4-shot settings. For a more comprehensive comparison, we reproduced their results using the official ResAD code. And training ResAD is time-consuming; therefore, we initially only provided reproduction results for the CNN-based architecture in our main text.\n> - For a fair comparison, we also included a comparison with ViT-based methods in **Appendix D Table 7**, and reproduced ResAD with the DINOv2 ViT-S [41] backbone and $448\\times 448$ resolution used in our method (**Table R11-1/2**). These results indicate that our method maintains an advantage when ResAD uses the same backbone and resolution. We will use these results for comparison in the main text.\n> - Due to space constraints, more comprehensive results and comparisons with MVTec3D, MVTecLOCO, BTAD, and MPDD will be included in the main text.\n\n**Table R11-1. Different Backbone Comparison (AUROC) under $N^2(+A^1)$**\n\n|Method|backbone|Total Parameters(M)|MVTecAD I|MVTecAD P|VisA I|VisA P|BraTS I|BraTS P|\n|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|PromptAD|CLIP ViT-B|-|95.7|96.2|88.3|97.1|-|-|\n|ResAD$^\\ast$|WideResNet50|59.2|87.2|94.8|86.6|96.5|66.2|91.5|\n|ResAD$^\\ast$|DINOv2 ViT-S|82.4|90.3|95.3|85.7|96.2|66.4|93.5|\n|Ours|DINOv2 ViT-S|24.4|96.8|96.8|89.8|97.6|82.1|96.8|\n\n**TableR11-2.Different Backbone Comparison (AUROC) under$N^4(+A^1)$**\n\n|Method|backbone|Total Parame ters(M)|MVTecAD I|MVTecAD P|VisA I|VisA P|BraTS I|BraTS P|\n|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|PromptAD|CLIP ViT-B|-|96.6|96.5|89.1|97.4|-|-|\n|ResAD$^\\ast$|WideResNet50|59.2|90.7|95.8|89.3|96.8|74.9|94.3|\n|ResAD$^\\ast$|DINOv2 ViT-S|82.4|93.5|96.5|89.0|96.5|74.8|94.1|\n|Ours|DINOv2 ViT-S|24.4|97.1|97.0|91.2|97.8|84.9|97.1\n\n_**Q7. Using different defect types.**_\n> **A7.** Our experiments, detailed in **Appendix A**, already include different defect type scenarios. The VisA dataset aggregates all anomalous samples per product without distinguishing between defect types. This can result in scenarios where the input query and the anomalous reference display different anomaly types (**Figure 4 b**, termed as inter-type). The **Table 1** also indicates that our method maintains a performance advantage over ResAD.\n\n> To facilitate a more intuitive comparison, we conducted additional experiments as detailed in **Table R12**. We test the *\"hazelnut\"* and *\"carpet\"* products from MVTecAD, using 'crack' and 'hole' as the test type, respectively. The results indicate that while the inter-type setting causes some performance degradation, the impact is minimal. Additionally, compared to the ResAD, our method demonstrates an advantage even when using different anomaly types. Therefore, our setting and method are not limited to the same anomaly type, which aligns with practical applications.\n\n**Table R12. Comparison of different defect types as abnormal references**\n\n|Method|Test Type|Abnormal Reference Type|I-AUROC|P-AUROC|\n|:-:|:-:|:-:|:-:|:-:|\n|ResAD|crack from hazelnut|-|90.2|96.7|\n|Ours(intra)|crack from hazelnut|crack|98.9|99.2|\n|Ours(inter)|crack from hazelnut|hole|98.8|99.2|\n|Ours(inter)|crack from hazelnut|holeandprint|98.8|99.3|\n|ResAD|hole from carpet|-|90.2|96.7|\n|Ours(intra)|hole from carpet|hole|100.0|99.5|\n|Ours(inter)|hole from carpet|color|99.1|98.4|\n|Ours(inter)|hole from carpet|colorandcut|99.7|99.4|\n\n> Our method, which is based on NN search, benefits from an increased number of references, as also discussed in **Appendix C**.\n\n***Q8. Typos.***\n> **A8.** We have corrected typos in the main text and carefully reviewed all statements."}]

Author rebuttal:

Meta-review: This paper introduces the Normal-Abnormal Generalist Learning framework for anomaly detection. In this setting models must generalize across domains using both normal and limited abnormal references. The method comprises Residual Mining, which extracts transferable residual representations and Anomaly Feature Learning, which adaptively identifies anomaly features in query images. Experiments on MVTecAD, VisA, and BraTS demonstrate notable improvements over prior generalist anomaly detection approaches. The paper addresses an important and practical gap in anomaly detection by explicitly incorporating abnormal references. The design is both conceptually sound and empirically validated through extensive experiments. The main weakness is unfair comparisons with existing works (they do not use abnormal samples) and some claims, e.g. first work utilizing abnormal samples as reference, are obivously exggerated. After discussion among reviewers and AC, all reviewers are leaning towards acceptance. Therefore, I recommend to accept this submission and the authors are strongly encouraged to address the remaining issues, in particular, comparison with existing works in the camera ready version.
Decision: Accept (poster)