One-for-All Few-Shot Anomaly Detection via Instance-Induced Prompt Learning
Authors: Wenxi Lv
Qinliang Su
Wenchao Xu
Venue: ICLR 2025 Poster (2025)
Keywords: Anomaly detection
few-shot
vision-language model
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Anomaly detection methods under the 'one-for-all' paradigm aim to develop a unified model capable of detecting anomalies across multiple classes. However, these approaches typically require a large number of normal samples for model training, which may not always be feasible in practice. Few-shot anomaly detection methods can address scenarios with limited data but often require a tailored model for each class, struggling within the 'one-for-one' paradigm. In this paper, we first proposed the one-for-all few-shot anomaly detection method with the assistance of vision-language model. Different from previous CLIP-based methods learning fix prompts for each class, our method learn a class-shared prompt generator to adaptively generate suitable prompt for each instance. The prompt generator is trained by aligning the prompts with the visual space and utilizing guidance from general textual descriptions of normality and abnormality. Furthermore, we address the mismatch problem of the memory bank within one-for-all paradigm. Extensive experimental results on MVTec and VisA demonstrate the superiority of our method in few-shot anomaly detection task under the one-for-all paradigm.

Published as a conference paper at ICLR 2025
ONE-FOR-ALL FEW-SHOT ANOMALY DETECTION VIA INSTANCE-INDUCED PROMPT LEARNING
Wenxi Lv Qinliang Su Wenchao Xu²
1 School of Computer Science and Engineering. Sun Yat-sen University, Guangzhou, China.
2 Guangdong Key Laboratory of Big Data Analysis and Processing, Guangzhou, China.
3 Department of Computing, The Hong Kong Polytechnic University, Hong Kong SAR suql eemliaangmaiii 1. edu.c..
1 wwxx@maill sysu. edu. cn wenchao, xuêpolyu edu.hk
ABSTRACT
Anomaly detection methods under the 'one-for-all' paradigm aim develop a unified model capable of detecting anomalies across multiple classes. However, these approaches typically require large number of normal samples for model training, which may not always be fulfilled in practice. Few-shot anomaly de- tection methods can address scenarios with limited data but require a tailored model for each class, following the 'one-for-one" paradigm. In this paper, we first proposed a one-for-all few-shot anomaly detection method with the assis- tance of vision-language models. Unlike previous CLIP-based methods that learn fixed prompts for each class, our method learns a class-shared prompt genera- tor to adaptively generate suitable prompts for each instance. The prompt gen- erator is trained by aligning the prompts with the visual space and utilizing guidance from general textual descriptions of normality and abnormality. In addition, we further propose a method to address the problem of how to re- trieve valid similar features from the visual memory bank under the one-for-all paradigm. Extensive experimental results on MVTec and VisA demonstrate the superiority of our method in few-shot anomaly detection task under the one-for- all paradigm. Our code is available in ht htt.t::::... http//gtttuuu thub. com/ Vanssssry/ ODne-For-lll Shot Anoma lyw.-aaoomaaeeeeeeeeee Detetction
INTRODUCTION
Visual anomaly detection aims to detect anomalies in images, which has widespread applications across various fields like industrial damage inspection (Bergmann al.] [019 Zou et al., (022). medical diagnosis (Zhang et al. 2020 Fang et al., (0024), etc.. Due to the countless forms and types of potential anomalies in real-world applications, as well as the difficulties of collecting anomalies.
existing anomaly detection methods (Deng & Li] 2022, Tien et al. 2023; Roth et al. 2022 Liu cet al., 2023 Schlüter et al. 2022) mostly assume the only availability of normal samples during training and then use them train normality model that can capture any samples that deviate from the normality. Despite achieving high detection accuracy, these methods follow the 'one-for-one' paradigm, which requires bespoke anomaly detection model to be trained for each class in the dataset, which, obviously, is cumbersome to use and expensive to train. To increase the flexibility and reduce the training complexity, some recent works have proposed resort the 'one-for-all' paradigm& as proposed in UniAD in (You et al.) 22222.. DiAD in (He et al. 2024), and VPDM in (Li al. 2024c). Different from the 'one-for-one paradigm, methods under the 'one-for-all paradigm only need to train a common model for all classes of data, and thus could significantly increase the using flexibility and training efficiency.
However, all of the 'one-for-one' and 'one-for-all' methods above assume the availability of large amount of normal instances for model training. But for some application scenarios, this requirement may not be easy to meet, egg, at the initial phase of manufacturing new product in scenarios with difficulties for instance collection. To deal with these scenarios, few-shot anomaly detectors
"oorresoonding author.

---

Published as a conference paper at ICLR 2025
have been proposed, such as RegAD in (Huang al. 2022) and FastRecon in (Fang et al. 2023 which use several available images to estimate the distribution of normal instances or use them to build coreset for instance reconstruction. But due to the scarcity of training data, the distribution or coreset estimated by these methods is very inaccurate, causing their detection performance to lag far behind their full-shot counterparts. Recently, inspired by advancements of vision-language models (VLMs) (Radford et al. 2021; Zhou et al. 2022b), WinCLIP (Jcong et al., (023) proposed to manually craft a set of prompts that describe the general properties of various anomalies and then use CLIP to assess the alignment between image patches and any of the anomaly prompts. Patches with high anomaly alignment scores are considered anomalous. However, as CLIP struggles with capturing local information, WinCLIP generates hundreds of image windows of different sizes and employs CLIP's visual encoder to extract their features, which obviously is computationally expen- sive. Moreover, the diversity and complexity of anomalies make it impossible to have manually crafted prompts cover or accurately describe all possible types of anomalies. To address the two is- sues, the recent work PromptAD (Li et al.] (2024b) employs the V-V attention mechanism (Li al. 2023b) to preserve local information in the output tokens of CLIP and then further proposes to learn set of vectors (prompts) from data automatically. Despite remarkable performance improvement and significant complexity reduction, PromptAD, as well as WinCLIP, still fall within the 'one-for- one' paradigm. That is, it requires to learn a separate prompt for every category of data, which could incur a huge computation burden for training and cause inconvenience for practical utilization.
In this paper, we propose the first one-for-all few-shot anomaly detection method by harnessing the zero/few-shot recognition ability of vision-language models CLIP (Radford et al., (021) and BLIP- Diffusion (Li et al., 2024a) via instance-induced prompt learning, naming our method instance- induced prompt anomaly detection (IIPAD). Specifically, instead of learning fixed prompt for every category as in WinCLIP and PromptAD, we propose to learn a common prompt generator for all categories. The prompt-generator is designed to assemble a common sequence of vectors and a set of instance-specific tokens output from CLIP and Q-Former in BLIP-Diffusion (Li et al.) 2024a) via concatenation and addition. In this way, the prompt generator can generate an exclusive prompt for every instance by taking its own characteristies into account, making the prompts better at extracting instance-specific normal and abnormal details. Given the designed prompt generator, the model is trained by seeking to align the prompt and visual embeddings corresponding to the same state (normal abnormal). The cross-modas alignment is further conducted at the token level to strengthen the model's fine-grained understanding ability. In addition to the prompt-visual alignment, we also introduce a set of general textual descriptions about normal and abnormal states, which can be viewed as a kind of expert knowledge, and use them to guide the training process by encouraging prompts to align with them. Furthermore, the memory bank is adopted within the one- for-all paradigm by endowing it with category-awe capabilities. Extensive experimental results on MVTec and VisA datasets demonstrate that our proposed one-for-all few-shot anomaly detection method shows significantly better performance than baselines under the one-for-all setting, and can achieve similar or even better performance than the SOTA one-for-one few-shot method.
2 RELATED WORK
Few-Shot Anomaly Detection Few-shot anomaly detection is designed for scenarios where only a limited amount of normal data is available for training. TDG (Sheynin et al.] 2021) proposed to augment images in the support set through various transformations and leverage a hierarchi- cal generative model to learn the multi-scale patch distribution. Instead of image augmentation, RegAD (Huang al., 2022) leverages an auxiliary dataset to learn the matching mechanism for anomaly detection on target dataset. DifferNet (Rudolph et al. (02T]) leverages a normalizing flow to estimate the distribution of descriptive features extracted by a pre-trained model. FastRecon (Fang
et al. 2023) and GraphCore (Xie et al.) 2023) build on PatchCore (Roth et al. (022).. FastRecon learns a projection matrix to reconstruct features as normal, while GraphCore enlarges the normal feature bank through data augmentation and trains a graph neural network to identify anomalies. However, all these methods follow a aenemmodel-per-categoyy paradigm.
Leveraging VLMs for Zero-/Few-shot Anomaly Detection Since CLIP (Radford et al., 2021) demonstrates remarkable performance in zero-shot and few-shot classification, it has gained sig- nificant attention and is widely studied for its potential application in anomaly detection. Win-
2

---

Published as a conference paper at ICLR 2025
* Frozen
Trainable
to Original Vil
Bf VIT Laser Vésatn
VT Lover Wiwtrnn
V Layar Westin
VT Lave Weatts
CA-Visoo
CA-Prompt
Synthetize Visual Features
Projection Network
#Q-Former
Prompt Generator
Construct collective visual embedding
st Text Encoder
MAggO! @ o
Patch feature
p Concatorate
Synthetic
Minuaal normal prompts
9
Manual abnormal prompts
00:00
Semantic- level Text
Guidance Prompt Learning.
The framework of IIPAD. The Q-Former is leveraged to extract object tokens for prompt and category-aware memory bank construction. The prompt generator trained using contrastive learning, guided by both visual and textual modalities, to produce instance-specific prompts for anomaly detection. The VIT used extract patch features is transformed with V-V attention.
Figure 1:
CLIP (Jeong et al. 2023) leverages manual text prompts to detect anomalies across predefined win- dows. AnoVL (Deng et al. 2023) uses an adapter to integrate text prompts with visual patches ex- visual encoder. Getting rid of abundant manual text prompts, 2024b) proposes to learn normal and anomalous prompts via semantic con-
tracted through V-V attention-basse
PromptAD (Li et al. catenation. But these methods still follow the one-for-one paradigm, learning tailored prompts for each class. AnomalyCLIP (Zhou et al. 2024) utilizes an auxiliary dataset from another domain to learn object-agnostic text prompls that capture generic normality and abnormality across categories, while AdaCLIP (Cao et al. 2024) uses the auxiliary dataset learn hybrid prompts. InCTRL (Zhu & Pang 2024) proposes leveraging few normal images prompts conduct residual learning, which captures general abnormality for anomaly detection. However, AnomalyCLIP and AdaCLIP require thousands of normal and anomalous images with pixel-level annotations, while InCTRI struggles with the anomaly localization task.
METHODS
In the task of few-shot anomaly detection, it is assumed the availability of a training dataset, Nerain = [21,X2,... ...... xN h. consisting of small number of normal images from multiple classes. Each class contains at most K examples, where K typically ranges from to this paper, our goal is to leverage the small dataset Xirain to learn a set of prompts to help detect and localize anomalies across various categories. To this end, we first develop class-shared prompt genera- tor, which can generate instance-specific prompt for every instance. Then, train the prompt generator by encouraging the alignment between the prompt and visual space as well as exploit- ing the guidance from general textual descriptions on normality and abnormality. Furthermore, cawee memory bank is further introduced to incorporate the test-time knowledge to detect anomalies. The framework of method is shown in Figure
3.1 INSTANCE-SPECIFE PROMPT GENERATOR
To avoid manually crafting the prompts, inspired by CoOp (Zhou et al. 2022b), existing prompt- based anomaly detection methods like AnomalyCLIP and PromptAD can be described introduc- ing set of learnable vectors for each category of data and then learning them separately. The prompts for the e-th category in these methods can be generally described
(1) where S and S denote the normal and abnormal prompts for the c-th category; [Pas] and [Nal denote the é-th token and T is the total length; and [CLS2] represents the tokens of e-th class name. Obviously, this prompt design makes the method fall into the 'one-for-one' paradigm, which could cause significant computation burden for training and ineur inconvenience for practical use.
ST = PP [Pe][CLLS]], S: Ne) [Nr]]CLLS))
3

---

Published as a conference paper at ICLR 2025
To adapt these methods to the 'one-for-all' paradigm, very simple way is to make the tokens [Pal and [aa not reliant on the class c, leading to the prompts S. = [P [Pr] CLS;] and S N N ... ....INN[[CLLS. Now, since the tokens [P,] and [N,] are shared among all instances, the only difference of prompts comes from the class name token [CT.S.]- However, the class name can only describe instances coarsely, resulting in these prompts not being expressive enough to capture the subtle difference between normal and abnormal instances. With the development of VLMs, an image can be described by several tokens that are extracted from VLMs (Gal et al. 2023 Li et al., 2023a 2024a). Since the Q-Former in BLIP-Diffusio (Li et al. 2024a) is trained together with CLIP to output a set of tokens that not only describe the image content but also can be interpreted by the CLIP textual encoder, we here propose to use it to extract set of object tokens Z; E Rd for the image xs. However, we note that directly using the object tokens Z, to replace the class name [CL.S] does not work very well, which is probably due to the fact that VLMS are not trained to capture the subtle difference between normality and abnormality of images. Thus, propose to first use two MLPs, fa and fa RM xd + R (m < M). to transform Z, into two distinct tokens O and O E Rmmd that are responsible for describing the normality and abnormality of images, respectively Then, we use OF and Of to design the normal and abnormal prompts as
= [P ... [Pr][O]].... [ S = [N.[aal [N [0 [0
(2)
Since the prefixss [P.] and [N.) are still shared among all instances, these prompts are still restrictive in capturing the subtle normality and abnormality in different images. To address this issue, inspired by CoCoOp (Zhou et al. 2022a), we further propose to inject instance-specific visual information into the tokens P.. and [N.] J. Specifically, for instance x,. we first propose a way to extract instance- specific visual normal and abnormal tokens C₁ E Txx and Cj E R from the output tokens of xd
CLIP and Q-Former Then, inspired by CoCoOp (Zhou et al. 2022a), we add the visual normal and abnormal tokens into the prompts [P:] and [N.]. leading to the final form
S PP + C + C%2] ... [P + C/ |
O im]
(3)
S N + Cū [N +
".. + 10 ... O im
where Cj denotes the 1-th token of CT. Here, the learnable prefixes [P. and [N,] are responsible to capture general descriptions of normality and abnormality, while CT and C; aim to provide normal and anomalous details tailored for a specific image. In this way, instance-specific prompts can be generated to detect anomalies, rather than relying prompt shared across all instances.
To obtain CT and C₁, we propose to simultaneously make use of the class token extracted from CLIP's visual encoder F.,(() and the category tokens Z; from Q-Former. The class token E, (x;) is not only projected into the textual token space via an MLP fp IRd + R as done in CoCoOp, but is also leveraged as query in cross-attention to retrieve information from category tokens Z, as
Zx 2)),, (4) where CA(-) denotes the cross-attention operation, while fn. fa : RRd - Txd denote two MI.s;; and Z and Z are obtained from Z, via two linear projection. Utilizing the two sources of visual information encourages C and C to capture both global visual information from the class token and fine-grained details from the category tokens.
C (xx(((((()))
)), C - (x,))
By passing the prompts S and S into the CLIP textual encoder E-(-), we obtain
c, Pa. ......., P.r = (ST), where T' = T + m is the length of input sequence; e, and e.f- E RR denote the prompt embeddings of ST and S., respectively; P.j and n., denote token embeddings of j-th token S. and SY:
cf n,,, D2y. Dy = (S,),
(5)
3.2 MULTI-MODAL PROMPT TRAINING
To enable the prompt generator to produce tailored normal and anomalous prompts for the corre- sponding input image, we propose guiding the prompt learning in both visual and textual modalities.
3.2.1 VISUAL GUIDANCE
Since can only access few normal samples, we apply contrastive learning to pull the normal prompt embedding closer to normal visual features, while pushing the anomalous prompt embed- ding away. This approach is consistent with the goal of Cl.P--based methods to detect anomalies.
4

---

Published as a conference paper at ICLR 2025
During training, considering the trade-off between utilizing multi-level semantic information and computational cost, we select the outputs from certain intermediate layers of the CLIP visual en- coder, with the set of indices of selected layers denoted as J. Then, we use the selected features to construct the set F; = (iill E J), where F, E R x+xWXx xd means the output of j-th layer of image X;. Notably, the visual encoder, which is used to extract internal normal features, is modified with V-V attention mechanism to enhance its capability to preserve local information (Li et al. 2024b
Zhou et al. 2024).
Semantic-Leves Alignment For each training image x,, the corresponding normal prompt embed- ding e is pulled closer the normal patch features in Fi, while the anomalous one ef is pushed away from them. Formally, the semantic-level alignment loss is defined as
W log exp(< F Fij,hw e," >) + exp(< << F Fij.hws e, >) (6)
=
1 J
Σ
exp( < F kij hun, o
H
JETT
where <. > denotes cosine similarity while F Fij.uw denotes the feature of (h, w)-th patch of image x; at the j-th layer. By minimizing this loss, the semantics of normal prompts become more aligned with the normal patches, while the semantics of the anomalous prompts diverge from them.
Token-Level Aligument Solely aligning the embedding of the entire prompt with patch features may weaken the prompt's ability to capture details, which, however, is crucial for the success of fine-grained anomaly detection tasks. Inspired by the observation that multiple image patches often correspond to a single word in the caption (Bica et al. (2024), we propose to encourage the alignment between a group of patches and the embedding of one prompt token. This approach encourages prompts to capture detailed, highly relevant local information provided by the V-V attention in CLIP. Specifically, we first compute the similarity matrix Mij E TR. T' between normal prompt token embeddings and patch embeddings from the j-th layer of CLIP visual encoder for image x;, where T" equals to T + m and R equals to H x W. The element Mer.kk of M,j represents the similarity between prompt token embedding pi. and patch feature Vij.k- To obtain the token-level alignment weight, we then normalize the elements of M.j to the range [0, 1] by min-max normalization across patch dimension Mij.t reduce irrelevant patches and encourage each prompt token embedding to be aligned with several patch features by setting Mas.ex = Meg.e if M.j,.x 2 a or otherwise, where o denotes the sparsity Mij.uk With threshold. The token-level alignment weight is finally obtained as Wij. Σn Mcj...
max, er min, Mig.er Then we sparsify the normalized matrix to
Ma.tk - min, Magar
the token-level alignment weight Waj. we can obtain the representation of patch features that are most relevant to a prompt token as Vij. = Σm= Wij.tr That is, the embedding Vij.a can be understood as the embedding of collective visual patches that are relevant to the j-th token. Thus, we propose to pull closer the prompt token embedding pi, and the collective visual embedding Vij. by minimizing the following loss
T
]
exp(< (i,1, Vijae >)
exp(< Vij.e. Pr.e >) E exp(< PI,k
Cj =
log
+log
(7)
J
T
E exp(< Ps. Vay.k >)
JEJ
Due to the lack anomalous samples, we can only push the anomalous prompt embedding away from normal patch features. However, since anomalies are often visually similar to normal images, blindly pushing the anomalous prompt away can result in the prompt losing meaningful semantic information, rendering it ineffective for anomaly detection. SimpleNet (Liu et al.) 2023) proposed to add noise to normal features. Since Gaussian noise is too mild and does not provide any meaningful information guide the anomalous prompt, we propose to further distort the feature F., from j-th layer with feature from the j'-th layer, where j' is the index immediately before j in J, that is,
Synthetic Visual Guidance for Anomalous Prompt
Fa Normalize + F.y + +c),
where Normalize()) denotes the operation that normalizes Fū; to have norn of 1, and ~ (O 1) denotes the Gaussian noise. By adding features from different layers, information at different scales can be fused, resulting in synthetic visual features Fž that include both low-level visual details and
5

---

Published as a conference paper at ICLR 2025
high-level semantic concepts, thereby introducing some inconsistencies unnaturalness. Then, can perform the semantic-lever and token-level alignment for the anomalous prompt:
1
1
w
cxp( < kj. c >)
C
log
(9)
exp(= << F
ij.hw of >) + expl < Fij ij.hw e
jEJ
exp(< ni
Vij. exp(< ni vg
cxpl< Vo
Di,t >)
2
log
+log T
13,23
(10)
JEJ
Ek=1
T
-k=1
expl<
Vigee
Di.s
where ni, denotes the 1-th anomalous prompt token embedding and Vija denotes the grouping patch features corresponding to n
3.2.2 TEXTUAL GUIDANCE
In the visual guidance above, we can only access normal images and the abnormal visual data are synthetic, which would introduce bias into the prompt learning process. To address this issue, propose to leverage the manually crafted prompts, which can be understood knowledge, to train the model. But we follow the one-for-all paradigm and only use general descrip- tions for normality and abnormality, e.g., replacing any specific category name with the common word 'object'. In particular, we generate normal manually-crafted prompts 7 from a template list that describes general normality, such as "A photo of a perfect object, A photo of flawless object", elc. While the anomalous manual prompts T are generated from the anomaly labels of the datasets as done PromptAD, e.g., "A photo of the object with color stain". Completed manually-crafie prompts 7 and T refer to Appdenix A.2 After generating a set of general normality and abnor- mality descriptions, instead of directly using them for anomaly detection, which lacks specificity and adaptability across different categories, we use them to guide the prompt learning. Consistent with the visual guidance of prompts, contrastive learning is also applied to align the learnable prompts with manual prompts by minimizing the loss
kind of expert
exp(<<e",
expl<eett exp(<ee texp(<e"
c;=Ee E
log
+R -log
(11)
where t", 1. denote embeddings of normal manual prompts and anomalous manual prompts, re- spectively. With the guidance of manual prompts, the learned prompts are endowed with the ability to identify general normality and real abnormality via the use of expert knowledge.
The overall objective of prompt learning is weighted sum of visual guidance and textual guidance:
L (C + ( + (C3 + LT) + af.
(12)
where a and B are hyperparameters.
3.3 CATEGORY-AWARE MEMORY BANK
Due to the rich normality information embedded in the features extracted by CLIP's visual encoder, the memory bank has been introduced in previous works (Jeong al. 2023 Li et al. 2024b) to enhance anomaly detection. However, under the one-for-all paradigm, the memory bank contains visual features from different classes and anomalous patch features may resemble and have high sim- ilarity with normal features from totally different categories, leading to undesired feature matching. This is called a mismatch problem and impairs the effectiveness of the memory bank in distinguish- ing between normal and anomalous features. To overcome this challenge, a category- memory bank is proposed. Instead of naively storing the image patch tokens output by the j-th layer of the CLIP visual encoder, we make full of category tokens Z extracted from the Q-Former. Specifically, we store the image patch tokens F,, of image x; and corresponding Q-Former (i.e. category) tokens Z, The category-awars visual memory bank can be represented as
visual
B" [[z F F1y1. [Zz; F2yl. ........ [ZN: F FNy1]
(13)
In the testing phase, we first retrieve a group of elements from B, using the similarity between the category tokens Z's of the test image and stored ones. We then evaluate the similarity between patch

---

Published as a conference paper at ICLR 2025
token features Fú of the test image and the ones that retrieved at the first stage. In practice, the intermediate features of two layers are selected as memory, denoted as Bjj B where j1. E J. Besides constructing visual memory bank, also build the category-aware prompt embedding bank B which stores various prompt embeddings training instances from Mtrain as
B e [Zz; e e ee)...,,,,,nnn ejy eN
(14)
During testing, we retrieve the top-k prompts most similar to the test image, following strategy similar that used in the category- visual memory bank.
3.4 ANOMALY DETECTION
Unlike prior works that rely solely on fixed prompts learned during the training phase (Zhou et al., 2024 Li al. 2024b), we can additionally leverage information from the test image by our prompt generator. With the groups of prompt embeddings E" and Ed retrieved from B" as stated above, the prompt embeddings ex and ef of the test image for detecting anomalies are then computed as
1 K
= 2 K EEE
e+e
ex =
E+te
(15)
CEE
where ef' and ei denote normal and anomalous fusion prompt embeddings of the test image, respec- tively. Given the j-th layer patch tokens F., E H image, the (h, w-tth element of
xxxWxx xd test image with representing the test
anomaly map A E [0,1) [0,1]"xW is obtained as
exp(<< < F -j. @a
A j.bw
(16)
exp( < F. F,,,ow, @ >) + exp( < F. F.j.ww e" >)
As there exist selected features from J, the final prompt-guided anomaly map is calculated as the average of that in all [T] layers, that is, AP 1J Σ5EJ A.. In addition, we also make use of the category-aware visual memory bank. With the group of patch tokens V1, V/2 retrieved from B By,, the layer patch tokens F, F. F of test image is compared with V1, V/2 to obtain visual anomaly map A" E [0, 1] HxW as follows:
A = min _ (1-
(1-<
< FF F.j, v >) + min (1 < F F.j2.ww V >), vEV. 2
(17)
vE 2
where ji and je denote the two layer indices that are used in construction B and B respec- tively. The final pixel-level anomaly score map and image-level score are computed as
Apizel
A + A",
Ammage
max A h,
max A"
(18)
h,ar
where the image-level score is sum of the maxima of the prompt-guided and visual anomaly maps.
4 EXPERIMENTS
4.1
EXPERIMENT SETTING AND BASELINE
Dataset In this paper, we conduct extensive experiments on benchmarks MVTec (Bergmann al., 2019) and VisA (Zou et al., 2022). The MvTec dataset contains 10 object categories and 5 texture categories while VisA dataset contains 12 different categories. Both benchmarks comprise high- resolution images and various anomalous types.
Experiment Setup We conduct the comparison experiments between our method and the latest methods under the setting that the training dataset contains all categories of the benchmark and only shot normal image for each class. Additionally, the memory bank used across all methods, well as the prompts in CLiP-based methods, follow the one-for-all paradigm. This means that the memory bank contains features from various categories, and the prompts are not initialized or specifically tailored to each individual class. The experimental results reported in this section averaged across their respective sub-datasets.
Evaluation Referring to previous work, the performance of anomaly detection and localization is evaluated using the Area Under the Receiver Operating Characteristic Curve (AUROC). Addition- ally, the Area Under Precision Recall (AUPR) for anomaly detection and the Per-Region Overlap
7

---

Published as a conference paper at ICLR 2025
Comparison of anomaly detection and localization performance on MVTec and VisA datasets across different few-shot settings under the onr-for-all paradigm. The best result is marked in bold and the runner-up result is marked underlined.
Table 1:
MVTec
VisA
Setup
Method
Image-level
Pixel-level
Image-level
Pixel-level
AUROC
AUPR
AUROC
PRO
AUROC
AUPR
AUROC
PRO
O-sbot
AnomalyCLIP
WinCLIP
89.1
91.5
96.2
95.0
81.1
91.1
62.3
81.4
73.3
82.1
76.3
85.4
79.8
57.9
SPADE
58.8
63.7
60.4
53.1
61.3
68.2
69.0
PatchCCore
63.7
81.2
83,9
72.7
58.9
62.8
76.7
64.3
FastRecon
$1.2
72.6
62.1
60.3
55.0
72.8
70.7
58.2
WinCLIP
96.5
92.4
PromptA
86.3
93.4
91.8
83.6
80.8
83.2
96.3
82.2
InCTRL.
IPAA
94.2
97.2
96.4
89.8
85.4
87.5
96.9
87.3
SPADE
68.4
84.2
61.2
54.7
66.8
72.0
71.3
59.6
PatchCCore
72.4
86,2
89.6
74.2
60.2
64.3
82.4
68.
FastRccon
$1.7
62.4
59.9
74.6
79.6
63.5
2-sbot
WinCLIP
92.7
96.3
92.4
83.9
83.7
84.9
95,1
81.8
PromptAD
89.2
94.8
92.2
84.3
84.3
87.8
96.9
InCTRL
94.0
96,9
85.88
87.7
IPAD
95.7
97.9
96.7
90.3
86.7
88.6
97.2
87.9
SPADE
76.6
88.8
62.8
55.6
73.0
76.6
72.1
60.9
PatchCore
74,9
FastRecon
50.8
73,1
65.0
62.8
57.6
73.7
78.8
62.9
4-sbot
WinCLIP
94.0
96.9
92.9
84.4
84.1
86.1
95.2
82.1
PromptAD
90.6
96.5
92.4
84.6
85.7
88.8
97.2
84.7
InCTRL
04.5
97.2
87.7
90.2
IPPAD
96.1
98.1
97.0
(PRO) for anomaly localization are employed to provide more comprehensive analysis of the model's performance.
Implementation details We build our model based on publicly available CLIP (Ilharco et al. 2021 (V14- The Q-Former used to extract category tokens is taken from BIIP-Diffusin (Li al,, 2024a). All parameters of pre-trained models are frozen. The length of learnable prompt tokens is set as 24. All experiments are conducted with a single NVIDIA RTX 3090 24GB GPU. For more implementation details refer to Appendix A.I
Baselines To demonstrate the superiority of our method, we select the lateset training-fres or CLIP- based anomaly detection methods as baselines: SPADE (Cohen & Hoshen 2020). PatchCore (Roth
ct al. 2022), FastRecon (Fang et al. (203)), WinCLIP (Jeong et al.] 2023) ). PromptAD (Li et al. 2024b), InCTRL (Zhu & Pang 2024) and AnomalyCL.IP (Zhou al.] 2024). Then, further
compare with the full-shot for-for-al anomaly detection methods: UniAD D (You et al.] [2022). Om-
niAL (Zhaol (023), HVQ-Trans (Lu et al. 2023) and DiAD (He et al. 2024). For more baselines details refer to Appendix A.3
4.2 EXPERIMENT RESULTS
The experiment results on MVTec and VisA datasets in different few-shot setting demonstrated in Table where SPADE (Cohen & Hoshen 2020), PatchCore (Roth al., 2022) are reformulated in the few-shot setting. To further demonstrate the competitiveness of our method, as shown in Table with baselines in the 'one-Tor-one" paradigm and the results of them are referred to their original papers. As shown in Table D 2 the performance of zero-shot methods remains weak compared to other baselines, even though they utilize auxiliary datasets or additional expert knowledge. This indicates the importance of incorporating information from the target domain for effective anomaly detection. Comparing Table and Table it is evident that the performance of all baselines signifi- cantly drops when they are trained or adapted to the one-for-all paradigm. This indicates that these few-shot anomaly detection methods struggle to effectively detect anomalies across different cate- gories when using a unified model. In contrast, our method, which trains an instance-specio speciifc prompt generator, significantly outperforms other baselines in both pixel-level and image-level anomaly de- tection under the one-for-all paradigm. Notably, even the fone-for-one paradigm, our method surpasses all baselines at the pixel level and achieves the second-best performance at the image level
Overall Performance
also compare our method, which remains the one-for-all paradigm,
8

---

Published as a conference paper at ICLR 2025
Table Comparison of anomaly detection and localization performance on MVTec and VisA across different few-shot settings under the one-for one-for-one paradigm. The results of baselines are taken from the original paper. Our method remains the one-for-all paradigm. The best result is marked bold and the runner-up result is marked underlined.
MVTec
Sctup
Method
Image-level
Pixel-level
Image-lovel
Pixcl-level
AUROC
AUPR
AUROC
PRO
AUROC
AUPR
AUROC
PRO
O-shot
WinCLIP
91.8
96.5
85.1
64.6
78.1
81.2
79.6
56.8
SPADE
91.2
PatchCure
83.4
92.2
92.0
79.7
79.9
82.8
95.4
80.5
FastRecon
I-shot
WinCLIP
93.1
96.5
95.2
87.1
85,
96.4
85.1
PromptAD
94.6
97.1
95.9
87.9
86.9
88.4
96.7
85.1
HPAD(OOne-or Alr-All)
94.2
87.3
SPADE
82.9
91.7
92.0
85.7
80.7
82.3
96.2
85.7
PatchCure
86.3
93.8
93.3
82.3
81.6
84.8
96.1
82.6
FastRecon
91.0
95.9
2-shot
WinCLIP
PromptAD
95.7
97.9
96.2
88.5
88.3
94.0
97.1
85.8
IIPADOOne-fo-AAll
96.7
SPADE
84.8
92.5
92.77
87.0
81.7
83.4
96.6
87.3
PatchCore
88.8
94.5
94.3
84.3
85.3
87.5
96.8
84.9
FastRecon
94.2
97.0
4-shot
WinCLIP
PromptAD
96,6
98.5
96.5
90.5
89.1
90.8
97.4
86.2
MBPADoOne-for-Al
96.1
98.1
97.0
91.2
88.3
89.6
97.4
88.3
6T
PromptAD
WinCUP
Ours
WinCUP
bottle
Cansii
Cable
Capules
MVRRc
Vis
Figure Visualization result comparison of 1-shot anomaly detection on MVTec and VisA.
across various few-shot settings the MVTec and VisA datasets, further demonstrating superiority of our method. More comparative analysis of our method refer to Appendix A..4
Table 3: Comparison of anomaly detection and lo- calization performance on MV'Tec and VisA with full-shot one-for-all anomaly detection methods.
Comparison with One-For-All Full-Shot Methods The results, presented Table 3 show that method achieves performance close to state-of-the--at full-shot one-for-all anomaly detection approaches, despite using only a few normal images. Remarkably, with the 4-shot setting, our method outperforms UniAD and DiAD at the pixel level on the MVTec dataset and surpasses OmniAL and DiAD at both image and pixel levels on the VisA dataset. This demonstrates that our method can effectively leverage limited data.
MVTec
Method
Sctup
AUROC,
AUROC, AUROC,
AUROC,
I-shot
94.2
96.4
85.4
96.9
IPPAD
2-sbot
95.6
86.7
4-shot
96,1
97.0
88.3
97.4
UniAD OmniAL.
96.5
96.8
91.9
98.6
Full-shot
97.2
98.3
87.8
96.6
DiAD
97.2
86.8
HVQ-Irans
98.0
97.3
93.2
98.7
Visualization Results We conducted qualitative experiments on MVTèe and VisA. The partial visualization results are shown in Figure 2 It can be observed that method provides more distinguishing and precise anomaly maps compared to other methods. Notably, as shown in Figure 2. our method is more sensitive to anomaly compared PromptAD while avoiding misclassification of normal regions like WinCLIP does. Additionally, our method shows the efficiency localizing small anomalies like those occur in candles and capsules. For more qualitative results refer to A.7

---

Published as a conference paper at ICLR 2025
4.3 ABLATION STUDY
Prompt Generator Ablation To validate the effective- ness of our prompt generator design, we conducted an ablation study by removing M1 and M2 components of generated prompts. M1 denotes the instance-specific vi- sual tokens C C while M2 denotes the normal and anomalous object tokens O", Oa appended at the end of prompts. As demonstrated in Table 4 the performance of our method declines when M2 is removed, indicat- ing that detailed and projected category descriptions are beneficial for anomaly detection. Additionally, the introduction of instance-speccffc component, which incorporate richer image information to prompts, improves the prompt's ability to detect and localize anomalies. More ablation and detailed experiment results refer to Appendix A.5
Table 4: Prompt generator ablation on MVTec and VisA under 1-shot setting.
MVTee AUROC, AUROCp AUROC,
VisA AUROC,
Mi M2
x
x
90.6
95.8
83.7
96.2
V
x
93.2
96.2
83.8
96.7
x
92.5
96.2
84.1
96.4
94.2
96.4
85.3
96.9
Loss Ablation To study the effectiveness of each guid- ance used in our method, we conducted loss ablation study by sequentially removing different loss compo- nents to assess their individual contributions. As il- lustrated in the Table 5 where L., L, and C refer to the semantic alignment Loss, textual guidance loss and token-level alignment loss respectively, both image- level and pixel-level performances of our method con- sistently degrade as we remove these loss components. A significant performance drop can be observed when the semantic-level alignment is removed. This is because semantic-level alignment serves as the pri- mary objective for training the prompt generator, aligning with the detection behavior during testing. The token-level alignment loss encourages the prompt tokens to focus on specific patches, enhanc- ing the anomaly localization capability of the generated prompts. Moreover, the improvement in performance brought by text guidance is significant. This indicates that expert knowledge, even when represented in text form rather than as specific anomaly maps, is very helpful for anomaly detection. These losses cooperatively guide the prompt generator to produce prompts that capture the normality and abnormality of images.
Table 5:
Loss ablation on MVTec and VisA under 1-shot setting.
R.
Le Cy
MVTec AUROC;
A AUROC, AUROC P
ViSA
V
x
91.1
95.8
81.5
95.9
/
x
92.3
81.8
V
<
91.4
96.2
82.6
96.4
x
92.6
95.6
83.1
96.2
94.2
96.4
85.4
96.9
To study the effec- tiveness of our category-aware memory bank design, conducted an ablation study by either removing or
Category-Aware Memory Bank
Table 6: Visual memory bank ablation on MVTec and VisA under 1-shot setting.
degenerating the category-aware memory bank. The results are presented in Table 6 where OVB denotes using the original visual bank without utilizing the Q- Former tokens Z; and CAVB denotes using the pro- posed category-awarc visual bank. The first row shows the performance the prompt generator in isolation from visual memory bank. significant perfor- mance drop occurs when the visual memory bank is removed, indicating that the visual memory bank plays a critical role in our proposed one-for-all anomaly detection method. Notably, the category- aware visual memory bank outperforms the original version, indicating that it can accurately trieve the corresponding sub-bank based on Q-Former token similarity, effectively addressing the mismatch issues present in the original visual memory bank.
MVTse AUROC, AUROC, AUROC, AUROC,
VisA
OVB CAVB
91,7
89,5
82.3
93.6
93.1
95.9
84.5
96.7
94,2
96.4
85.4
96.9
5 CONCLUSION
In this paper, we propose novel few-shot anomaly detection method that enables to detect anoma- lies across different categories with a unified model. We design class-shared prompt generator, which can adaptively generate instance-specific prompts for detecting anomalies across different classes. Then, the prompt generator is trained by aligning the prompt with both visual and textual feature guidance. Furthermore, we address the mismatch issue in the memory bank to adopt it in one-for-all paradigm. The incorporation of test-time knowledge further enhances the adaptability and flexibility of the prompts. Extensive experimental results on MVTec and VisA demonstrate the superiority of our method in the few-normal-sho setting within the one-for-all paradigm.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGEMENTS
This work is supported by the National Natural Science Foundation of China (No. 62276280, 62276279), Guangzhou Science and Technology Planning Project (No. 2024A0419967).
REFERENCES
Paul Bergmann, Michael Fauser, David Sattlegger, and Carsten Steger. Mvtec ad-a comprehen- sive real-world dataset for unsupervised anomaly detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9592-9600, 2019.
Ioana Bica, Anastasija Ilie, Matthias Bauer, Goker Erdogan, Matko Bosnjak, Christos Kaplanis, Alexey A. Gritsenko, Matthias Minderer, Charles Blundell, Razvan Pascanu, and Jovana Mitro- vic. Improving fine ane-grained understanding image-text pre-training. In Forty-first International Conference on Machine Learning. 2024.
Yunkang Cao, Jiangning Zhang, Luca Frittoli, Yuqi Cheng, Weiming Shen, and Giacomo Borac- chi. Adaclip: Adapting clip with hybrid learnable prompts for zero-shot anomaly detection. In European Conference on Computer Vision, pp. 55-72. Springer, 2024.
Niv Cohen and Yedid Hoshen. Sub-image anomaly detection with deep pyramid correspondences.
arXiv preprint urXhv:2005.02357, 2020.
Hanqiu Deng and Xingyu Li. Anomaly detection via reverse distillation from one-class embedding. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp- 9737-9746, 2022.
Hanqiu Deng, Zhaoxiang Zhang, Jinan Bao, and Xingyu Li. Anovl: Adapting vision-language models for unified zero-shot anomaly localization. arXiv preprint arXiv:2308. 15939, 2023.
Qingqing Fang, Qinliang Su, Wenxi Lv, Wenchao Xu, and Jianxing Yu. Boosting fine- ane-grained visual anomaly detection with coarse-knowledge--warre adversarial learning. arXiv preprint arXiv:2412.128550, 2024.
Zheng Fang, Xiaoyang Wang, Haocheng Li, Jiejie Liu, Qiugui Hu, and Jimin Xiao. Fastrecon: Few- shot industrial anomaly detection via fast feature reconstruction. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 17481-17490, 2023.
Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit Haim Bermano, Gal Chechik, and Daniel Cohen-Or. An image is worth one word: Personalizing text-to-image generation using textual inversion. In The Eleventh International Conference on Learning Representations, 2023.
Haoyang He, Jiangning Zhang, Hongxu Chen, Xuhai Chen, Zhishan Li, Xu Chen, Yabiao Wang, Chengjie Wang. and Lei Xie. A diftusion-based framework for multi-class anomaly detection. In Proceedings of the AAAI Conference Artificial Intelligence, volume 38, pp. 8472-8480, 2024.
Chaoqin Huang, Haoyan Guan, Aofan Jiang, Ya Zhang, Michael Spratling, and Yan-Feng Wang. Registration based few-shot anomaly detection. In European Conference Computer Vision, pp. 309.319. Springer, 2022.
Gabriel Ilharco, Mitchell Wortsman, Ross Wightman, Cade Gordon, Nicholas Carlini, Rohan Taori, Achal Dave, Vaishaal Shankar, Hongseok Namkoong, John Miller, Hannaneh Hajishirzi, Ali Farhadi. and Ludwig Schmidt. Openclip, July 2021. URL bttpp:://cc..rrs zenodo. 51 5143773 If you use this software, please cite it as below.
5281/
Jongheon Jeong, Yang Zou, Taewan Kim, Dongqing Zhang. Avinash Ravichandran. and Onkar Dabeer. Winelip: Zero-ffew-shot anomaly classification and segmentation. In Proceedings of the IEFF/CVF Conference Computer Vision and Pattern Recognition, pp. 19606-19618 2023.
Diederik Kingma and Jimmy Ba. Adam: method for stochastic optimization. In Yoshua Bengio and Yann LeCun (eds.), 3rd International Conference on Learning Representations, 2015.
11

---

Published as a conference paper at ICLR 2025
Dongxu Li, Jumnan Li, and Steven Hoi. Blip-diffusion: Pre-trained subject representation for con- trollable sokatto-imag generation and editing. Advances in Neural Information Processing Sys- tems, 36, 2024a.
Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning. pp. 19730-19742. PMLR, 2023a.
Xiaofan Li, Zhizhong Zhang, Xin Tan, Chengwei Chen, Yanyun Qu, Yuan Xie, and Lizhuang Ma. Promptad: Learning prompts with only normal samples for few-shot anomaly detection. Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16838- 16848, 2024b.
Yi Li, Hualiang Wang. Yiqun Duan, and Xiaomeng Li. Clip surgery for better explainability with enhancement in open-vocabulary tasks. arXiv preprint arXiv:2304.05663, 2023b.
Yuxin Li, Yaoxuan Feng, Bo Chen, Wenchao Chen. Yubiao Wang, Xinyue Hu, Baolin Sun, Chunhui Qu, and Mingyuan Zhou. Vague prototype-orieted diffusion model for multi-class anomaly detection. In Proceedings of the s International Conference on Machine Learning, volume 235, pp. 27771-27790, 21-27 Jul 2024c.
Zhikang Liu, Yiming Zhou, Yuansheng Xu, and Zilei Wang. Simplenet: A simple network for image anomaly detection and localization. In Proceedings of the IÉEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 20402-20411, 2023.
Ruiying Lu, YuJie Wu, Long Tian, Dongsheng Wang, Bo Chen, Xiyang Liu, and Ruimin Hu. Hier- archical vector quantized transformer for multi-class unsupervised anomaly detection. Advances in Neural Information Processing Systems, 36:8487-8500, 2023.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp- 8748-8763. PMLR, 2021.
Karsten Roth, Latha Pemula, Joaquin Zepeda, Bernhard Schölkopf, Thomas Brox, and Peter Gehler. Towards total recall in industrial anomaly detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 14318 14328, 2022.
Marco Rudolph, Bastian Wandt, and Bodo Rosenhahn. Same same but differnet: Semi-supervised defect detection with normalizing flows. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp. 1907-1916, 2021.
Hannah M Schlüter, Jeremy Tan, Benjamin Hou, and Bernhard Kainz. Natural synthetic anomalies for self-supervised anomaly detection and localization. In European Conference on Computer Vision, pp. 474 489. Springer, 2022.
Shelly Sheynin, Sagie Benaim, and Lior Wolf. A hierarchical tranfomatoonddiccimiaatigg gen- erative model for few shot anomaly detection. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 8495-8504, 2021.
Tran Dinh Tien, Anh Tuan Nguyen, Nguyen Hoang Tran, Ta Duc Huy, Soan Duong, Chanh D Tr Nguyen, and Steven QH Truong. Revisiting reverse distillation for anomaly detection. In Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 2023.
Guoyang Xie, Jinbao Wang. Jiaqi Liu, Yaochu Jin, and Feng Zheng. Pushing the limits of fewshot anomaly detection in industry vision: Graphcore. In The Eleventh International Conference on Learning Representations, 2023.
Zhiyuan You, Lei Cui, Yujun Shen, Kai Yang, Xin Lu. Yu Zheng, and Xinyi Le. A unified model for multi-class anomaly detection. Advances in Neural Information Processing Systems, 35:4571- 4584, 2022.
12

---

Published as a conference paper at ICLR 2025
Jianpeng Zhang, Yutong Xie, Guansong Pang, Zhibin Liao, Johan Verjans, Wenxing Li, Zongji Sun, Jian He, Li, Chunhua Shen, al. Viral pneumonia screening on chest x-rays using confidence- aware anomaly detection. IEEE transactions medical imaging. 40(3):879-838 890, 2020.
Ying Zhao. Omnial: A unified framework for unsupervised anomaly localization. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3924-3933, 2023.
Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Conditional prompt learning for vision-language models. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 16816-16825, 2022a.
Kaiyang Zhou, Jingkang Yang. Chen Change Loy, and Ziwei Liu. Learning to prompt for vision- language models. International Journal of Computer Vision, 130(9):2337-2344 2022b.
Qihang Zhou, Guansong Pang, Yu Tian, Shibo He, and Jiming Chen. Anomalyclip: Object-agnostic prompt learning for zero-shot anomaly detection. In The Twelfth International Conference on Learning Representations, 2024.
Jiawen Zhu and Guansong Pang. Toward generalist anomaly detection via in-context residual learn- ing with few-shot sample prompts. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 17826-178366 2024.
Yang Zou, Jongheon Jeong, Latha Pemula, Dongqing Zhang, and Onkar Dabeer. Spot-the-difffrence self-supervised pre-training for anomaly detection and segmentation. In European Conference Computer Vision, pp. 392-408. Springer, 2022.
13

---

Published as a conference paper at ICLR 2025
A APPENDIX
A.1 IMPLEMENT DETAILS
Training Details We build our model based on publicly available CLIP (VIT-L-144.. The Q- Former used to extract category tokens is taken from BLIP-Diffusinn, which is already align to CLIP's text encoder. The parameters of CLIP and Q-Former are all frozen during training. The length of learnable prompt T is projection networks used to incorporate visual information are two-layer neural network. The weight factors a, 33 that balance loss are both fixed as 8. To provide adequate visual details, we select patch visual embeddings F,, from the 6-th, 12-th, 18-th, and 24-th layers of the visual encoder to guide prompt learning. The visual encoder is reformed into V-V attention mechanism. Patch features from the 6-th and 18-th layers of the original visual encoder are selected as memory bank. The threshold for sparsify the normalized matrix set as 256 We use the Adam optimizer (Kingma & Ba 2015) with learning rate of 0.001 to update model parameters. The epoch is 20 for all experiments, which are performed in Py Torch with a single NVIDIA RTX 3090 24GB GPU.
24. The length of compressed category tokens m is 4. The
I_
A.2 MANUAL PROMPTS DETAILS
We introduce manual text prompt as expert knowledge to guide learnable prompts in capturing normality and abnormality. The manual normal and anomalous prompts are given as follows:
Normal manual prompts: {'a image of the normal object', 'a image of the flawless object', 'a image of the perfect object', 'a image of the unblemished object', 'a image of the object without flaw', , 'a image of the object without defect', 'a image of the object without damage' }
Anomalous manual prompts (MVTec): { la image of the damaged object', 'a image of the abnormal object', 'a image of the imperfect object', 'a image of the blemished object', 'a image of the object with flaw', 'a image of the object with defect', 'a image of the object with damage', 'a image of the object with large breakage', , 'a image of the object with small breakage', , 'a image of the object with contamination", 'a image of the object with defect', 'a image of the object with anomaly', 'a image of the object with hole', 'a image of the object with color stain', 'a image of the object with metal contamination', residue', 'a image of the object with thread', 'a image of the object with cut', 'a image of the object with crack', , 'a image of the object with cut', 'a image of the object with hole', 'a image of the object with print', 'a image of the object with color stain', 'a image of the object with cut', 'a image of the object with fold', 'a image of the object with glue', 'a image of the object with poke', 'a image of the object with bent wire', 'a image of the object with missing part', 'a image of the object with missing wire', 'a image of the object with cut', 'a image of the object with poke', , 'a image of the object with crack', , 'a image of the object with faulty imprint', 'a image of the object with poke', 'a image of the object with scratch', 'a image of the object squeezed with compression', with breakage', 'a image of the object with thread residue', 'a image of the object with thread', 'a image of the object with metal contamination' }
'a image of the object with thread
'a image of the object
Anomalous manual prompts (VisA): { 'a image of the damaged object', aa image of the abnormal object', 'a image of the imperfect object', , 'a image of the blemi shed object', 'a image of the
14

---

Published as a conference paper at ICLR 2025
object with flaw', 'a image of the object with defect', 'a image of the object with damage', 'a image of the object with melded wax', 'a image of the object with foreign particals', 'a image of the object with extra wax', 'a image of the object with chunk of wax missing', 'a image of the object with weird candle wick', 'a image of the object with damaged corner of packaging', 'a image of the object with different colour spot', 'a image of the object with scratch', image of the object with discolor', 'a image of the object with misshape', 'a image of the object with leak', 'a image of the object with bubble', 'a image of the object with breakage', 'a image of the object with small scratches', 'a image of the object with burnt', 'a image of the object with stuck together', 'a image of the object with spot', 'a image of the object with corner missing', 'a image of the object with scratches', 'a image of the object with chunk of gum missing', 'a image of the object with colour spot', 'a image of the object with cracks', 'a image of the object with color spot', 'a image of the object with fryum stuck together', 'a image of the object with small chip around edge', 'a image of the object with bent', 'a image of the object with missing', 'a image of the object with melt', 'a image of the object with extra', 'a image of the object with wrong place', 'a image of the object with damage', 'a image of the object with dirt')
A.3 BASELINES DETAILS
We compare our method against a wide range of baselines to demonstrate its superiority. The im- plementation details of the baselines are provided below:
SPADE (Cohen & Hoshen 2020). SPADE uses correspondences based on multi- resolution feature pyramid, which is inspired by KNN and training-free. SPADE is suitable for detecting anomalies in now-normal-soe setting.
PatchCore (Roth et al. 2022). PatchCore is also a training- anomaly detection method which is proposed to construct coreset of normal feature. PatchCore has demonstrate the ability that handle few-shot anomaly detection anomaly detection.
FastRecon (Fang et al. 2023). FastRecon is proposed to improve the performance of Patch- Core in few-shot settings. This method learns a transform matrix from a few normal sam- ples to reconstruct feature as normal. FastRecon is reproduced from open source code. All parameters are kept the same in their paper.
WinCLIP (Jeong et al. 2023). WinCLIP is pioneering work that adopts CLIP in anomaly detection. It utilize manual text prompts to detect anomalies across predefined multi-scale windows. Simultaneously, the multi- scale memory bank is constructed for feature matching in sew-shot setting. WinCLIP is reproduced from open source code. All parameters are kept the same as in their paper.
AnomalyCLIP (Zhou et al., 2024). AnomalyCLIP is the first work to take category in- formation into consideration. AnomalyCLIP is proposed to learn the category-agnostic prompts to capture general normality and abnormality by leveraging auxiliary dataset. AnoM is reproduced from its official open source code. All parameters are kept the same as in their paper.
PromptAD Li et al.) 2024b). PromptAD is SOTA few-shot anomaly detection method. They proposed semantic concatenation reverse prompt's 's semantic and directly optimize a set of learnable context vectors. PromptAD is reproduced from its official open source code. All parameters kept the same in their paper.
15

---

Published as a conference paper at ICLR 2025
InCTRL (Li et al. 2024b). InCTRL is SOTA few-shot anomaly detection method. They propose to learn the ability to compare on an auxiliary dataset through in-context residual learning to detect anomalies. The performance of InCTRL is referred to its original paper.
A.4 COMPARISON ANALYSIS
Our method have superiority in the scenarios which requires the anomaly detection methods follow the one-for-all paradigm. For example, we may only access to the raw dataset without category information of it in practice. In this scenario, to, which prohibit WinCLIP and PromptAD to construct different prompt for different category. Our method, on the other hand, follows the one-for-all paradigm and extracts category informa- tion directly from the image using Q-Former, thus avoiding the limitations faced by WinCLIP and PromptAD. The superiority our method in such scenario can be demonstrate by the comparison with other baselines in one-for-all paradigm as shown in Table 1
not know the exact category the image belong
A.5 ADDITIONAL RESULTS AND ABLATIONS
A.5.1 SYNTHETIC VISUAL FEATURES
To further investigate the impact of synthetic visual guidance, conducted an ablation study on the synthetic visual feature. As shown in the Table 7 the performance of our method with synthetic visual feature outperforms the version without them. This be attributed to the synthetic visual feature acting as an anchor with informative context for anomalous prompts to pull closer, rather than blindly pushing away from normal features.
A.5.2 NEURAL NETWORKS PROMPT GENERATOR
We further investigate the impact of modules in instance-specific prompt generator. As shown in the Table S every module in the prompt generator has positive influence. The Projection Network projects the token from the Q-Former and the class token from the VIT into representations that more suitable for the anomaly detection task. The Cross-Attention module demonstrates improvement in anomaly localization, indicating that the interaction between the Q-Former and VIT enhances the fine-grained description of prompts.
A.5.3 GAUSSIAN NOISE IN SYNTHETIC VISUAL FEATURES
To verify whether the Gaussian noise added synthetic visual features is required or not, we conduct the ablation study and the experiment result is shown in Table 9 As we can observe, significant performance drop occurs when removing the Gaussian noise in synthetic visual feature. This can be attributed to the role of Gaussian noise in smoothing the synthetic visual feature and pushing it into low-density areas. Consequently, the Gaussian noise is required and plays an important role in synthetic visual feature.
A.5.4 INTRODUCE MANUAL PROMPT ABLATION
We further investigate the impact of manual prompts in guiding prompt learning by conducting ablation study comparing the performance of prompts guided by specific manual text descriptions (e.g., T²) versus those guided by more general manual text descriptions. The results, shown in Table 10 reveal that prompts guided by specific manual text descriptions outperform those guided by general descriptions. This because specific manual text guidance incorporates knowledge of real anomalies, thereby enhancing the prompts' ability to effectively detect anomalies.
A.5.5 MULTI-LAYER PATCH FEATURE ABLATION
In the visual guidance phase, we use selected feature set F, In particular, patch visual embeddings Fy from the 6-th, 12-th, 18-th, and 24-th layers of the visual encoder are selected to guide prompt learning. We conduct the ablation experiment to study the performance of prompt guided by multi- layer features and guided by last layer output. The ablation result presented in Table IТ. It can be observed that prompts guided by multi-layer visual features outperform prompts guided by last
16

---

Published as a conference paper at ICLR 2025
Table 7: Synthetic visual features ablation.
Synthetic
MVTec AUROC₁
MVTee AUROCp
VisA AUROC;
VisA
Visual Features
AUROCp
93.6
95.9
84.3
96.5
94.2
96.4
85.4
96,9
Table 8:
Neural networks in prompt generator ablation.
Projection Network Relevant to C and C
Cross Attention
Projection Network
MVTec AUROC,
MVTec AUROCp
VisA AUROC,
VisA AUROCp
Relevant O and O
93.7
95.9
84.4
96.3
X
93.7
96.1
84.6
96.5
94.1
96.1
84.8
96.6
93.8
96.2
84.4
96.7
94.2
96.4
85.4
96.9
layer output. This indicates that multi-layer visual features contain richer visual information range from lew-level details high level semantic, which can better guide prompts in capturing normality and abnormality.
A.6 HYPER- PARAMETER ANALYSIS
We conduct hyper-parameter analysis experiments a, B, and T, where and 3 represent the weights of the text guidance loss and fine-grained visual guidance loss, respectively, and T is the number of learnable prompt tokens. The experimental results ble 12a a has significant influence to our method. Setting a too small results in the prompts cap- turing insufficient expert knowledge, while setting too large leads to loss of visual information. Our method is not sensitive to 8 as shown in Table [2b] For the number of learnable prompt tokens, we find few tokens fail capture the normality and abnormality while too many tokens cause semantic redundancy, we set T as 24 to achieve the best performance as shown in Table [2c
shown in Table 12 From Ta-
A.6.1 CLASS-WISE RESULTS
In this section, report the detailed subset-wise result of our method. The results of MVTec
presented in Table [3 16
The results of VisA are presented in Table 17
A.7
MORE VISUALIZATION RESULTS
A.7.1 VISUALIZATION OF ANOMALY LOCALIZATION
Additional qualitative results, which are tested on MVTec and VisA and obtained from method I-shot setting, is shown in Figure 4 From Figure we can observed that our method can detect and locate both large anomaly area and small scratch dot the surface. Moreover, method is very restrained locating anomalies to avoid identifying normal areas as anomalies.
A.7.2 FAILURE CASES STUDY
In this section, we analysis the failure cares of anomaly localization in MVTec and VisA dataset. In the "Capsules" case, bubbles within the capsules are annotated as anomalies. However, the presence of an uncommon shadow in the image led our method to mistakenly identify this area, resulting in mislocalization. In logical anomaly cases such as the misplacement of a transistor or the sticking of two cashews, as well as in the "Pill" case, the anomalous areas exhibit patterns similar to nor- mal ones. Since we can only access to normal sample and the anomalous labels do not specifically describe these abnormalities, lies from the text prompts based on such labels, leading to the failure in accurately locating these anomalous areas. The "Zipper" case in MVTec and the "PCB1" case in VisA, may be caused by the
method struggles to learn the distinguishing features of the anoma-
17

---

Published as a conference paper at ICLR 2025
Table Gaussian noise in synthetic visual features ablation.
Gaussian Noise
MVTec AUROC₁
MVTec AUROCp
VisA AUROC;
VisA
AUROCp
92.5
95.4 96,4
83.1 85.4
96.4
94.2
96.9
Table 10: Manual prompt design ablation.
Generic Prompt
Designed Prompt
MVTec AUROC,
MVTec AUROCp
VisA AUROC,
VisA AUROCp
93.9
96.1
96.4 85.4
85.1
94.2
96.4
96.9
Q-Former's failure to effectively capture and describe the foreground object, leading to an inability detect small anomalies present on the object.
Original
GT
Heatmap
Original
GT
Heatmap
Pill (color)
Capsules (bubble)
Transistor
Cashew (stuck)
(misplace)
Zipper (broken teeth)
PCB1 (bent wire)
Figure 3: Failure case on MVTec-AD and VisA. The left figure demonstrate the sample from MV'Tec while the right figure demonstrate the sample from VisA.
18

---

Published as a conference paper at ICLR 2025
Table 11: Layer selection of visual guidance ablation.
MVTec AUROC,
MVTec AUROCp
VisA AUROC;
VisA
Last Layer
Multi-layer
A
92.3
96.2 96.4
85.3
X
94.2
85.4
96.9
Table 12: Hyper-parameter analysis result.
(a) Analysis of a.
(b) Analysis
(c) Analysis of token number.
Mytec Pixel
VisA
Mvtec
VisA
Mvtec Image
VisA
Image
Image
Pixel
Image
Pixel
Image
Pixel
Pixel
Image
Pixel
1
92.1
96.0
79.9
96.1
1
93.1
96.3
84.1
96.7
93.6
95.9
83.8
96.8
4
93.5
96.3
82.4
96.4
4
93.7
96.3
84.3
96.6
16
93.9
95.9
83.9
96.7
94.2
96.4
85.4
96.9
85.4
94.2
96.4
85.4
96.9
12
91.4
96.2
84.5
96.8
12
93.8
96.4
84.7
96.8
32
93.8
96.0
84.2
96.7
Table 13: Comparison of pixel-level anomaly detection in terms of subset-wise AUROC MVTec.
L-her PrrmgrAD
2oher PrimprD
4-dvit PrepgAD
SPARI:
WirtTLB
Asomaty(11PP
SPAIH
WirC3.1"
AnrmatyC111P
WirC1/P
AnematyCLiP
Botts
65.9
15.1
$4.5
10..
95.4
63.3
15.5
$5.9
90L4
95.5
67.5
95.1
96.9
S0L4
95.6
Cable
51.2
74.55
76.8
73.9
54.3
56.1
16.4
10..
72.9
95.3
09.2
17.]
$3.1
12.0
95.4
Capualc
Gnd
64.7
05.1
96.8
07.3
9565
06.9
16.3
96.0
07.3
95.3
07.6
56.0
96.5
97.3
95.1
56.0
98.5
96.9
97.1
08.4
18.1
96.1
58.0
97.1
06.7
00.3
98.1
58.1
07.1
08.3
95.8
991
96.
01.4
994
96.5
V1.9
56.3
95.9
Scaw
11.0
W.2
05.7
97.5
88.0
53.7
92.
36.3
97.5
93.5
939
98.3
073
97.
95.9
02.5
Thabbeshh
91.9
68.5
59.1
Whod
51.0
04.7
05.8
06.5
810
32.3
95.3
06.5
81.3
33.6
98.5
55.6
06.5
Zigner
62.9
918
91.5
98.0
02.4
97.3
91.4
95.1
46.7
62.9
050
91.4
90.4
91.1
928
91.1
91.0
Table 14: Comparison of pixel-level anomaly detection terms of subset- wise PRO on MVTec.
MVTee
SPARI:
Whrt71P
L-her TrergrAD AD
AsoratyC11PP
WirC7.1P
2olher TremyrA
AarmatyC111P
WhrT1JP
4-dee PreqpAD
AnrmutyC1
Drc
56.5
59.1
50.9
55.5
$1.5
$0.9
05.2
56.5
53.4
50.5
95.5
Cable
30.7
55.9
T1L1
64.4
15.7
31.6
62.1
11.3
64.6
83.5
15.3
61.1
15.1
66.1
91.3
Caposoo
39.0
$1.3
57.3
$1.3
K8.2
F2.0
$7.0
93.6
51.7
18.5
$0.1
87.3
Grd
$9.3
$1.5
Rodiun
47.6
07.5
56.5
07.4
53.3
51.6
95.1
53.6
02.8
91.0
51.8
02.0
63.1
02.1
01.8
082
00.
61
98.1
02.3
1955
Scew
.0.8
$7.0
K30
10.0
85.3
90.8
18.
K6.
1000
35.3
11.3
16.
191
10.0
87.5
75.7
87.
791
87.
18.3
914
87.
Torcaurr
41.9
67.5
15.1
10.1
45.5
694
15.1
Wovel
56.7
96.9
02.6
91.2
61.3
58.7
62.6
93.4
91.2
94.0
92.5
10.1
03.3
91.5
91.
Zipper
90.2
61.9
63.
51.7
61.8
91.7
63.
96.6
51.2
31
91.3
65.3
63.9
Table 15: Comparison of image-level anomaly detection in terms of subset-wise AUROC on MVTec.
Iobet
Zobet
Bortte
$1.m
M6.9
18.5
$2.5
90.7
96.3
94.2
100:1)
$2.3
90.5
185
96.3
500
$2.1
901
65.6
65.3
$3.3
94.5
Cage
72.3
99.9
100
100.0
100.0
57.0
99.9
100.0
108.0
101.0
02.5
09.9
100.0
100.0
10LD
Gal
91.0
9916
35.8
97.0
930
66.6
9912
97.4
93.0
9500
07.3
09.5
1000
97.0
950
1000
95.8
Mena
6222
DED
97.6
93.6
994
T0O
07.8
962
03.6
997
75.5
02.0
1030
$3.0
969
$1.35
$1.35
2044
$1.5
100.
983
11819
982
28.8
982
11819
99.5
562
118111
909
Tretbrohh
46.0
14.2
944
$417
913
46.8
913
1030
$477
92.5
0910
95.1
18.3
$47
92.5
732
91.5
91.4
35.8
8112
91.35
9968
15.0
94-4
91.8
Zipper
07.3
RED
95.3
48.5
99.4
88.7
07.3
93.5
98.5
95.
02.
07.0
95.8
98.5
9600
853
91.
89.2
91.
906
915
19

---

Published as conference paper at ICLR 2025
Table 16: Comparison of image-level anomaly detection in terms of subset-wise AUPR on MVTec.
SPADE
WirCT.
ArerelyCTJP
SPADE:
WirCT.P
ArurulyyT.JP
SPADE:
WorCl.P
AnmrllyCTJ
Buil:
96.0
52.5
01.0
053
06.1
09.3
100.0
01.8
110.0
0v.0
09.3
50.1
513
95.7
T6.1
85.7
97.9
95.4
85.5
97.9
98.3
05.
$6.5
91.3
$5.0
Cape
87.2
1080
1810
11810
1181
96.0
118LO
1930
11810
1181)
97-4
118L0
103.0
11810)
[181)
993
9611
9911
9611
1931
991
Lesber
16.4
1080
1930
999
188L
1930
999
188
1041
1931
993
1880
Nani nat
19.6
9933
04.5
993
91.9
91.
98-1
91.3
993
66.1
98.8
1920
99.3
1180
9538
91.1
86.5
912
93.
Tie
$5.1
1000
595
100.0
938
58.6
100.
59.7
100.
9938
92.
100.
594
1000
958
Tootbesh
72.2
67.7
973
93.7
07.3
7016
97.6
1900
93.7
97.5
72.
59.5
837
97.
Wood
993
9938
75.
Zipper 09.3
08.6
98.8
996
05.6
96.9
99.3
96.7
99.6
95.7
98.0
98.3
98.5
996
98.1
Table 17: Comparison pixel-level anomaly detection in terms of subset-wing AUROC on VisA.
Viek
SEPADE:
WasCLIP
L-dae PherpeD
AssmaayLiP
Ouz
SUADE
WinkCUiP
Zohet PremppAD
ArsmaayyCU
Gan
SPALE
WisCLIP
4abbtt PrrepiAL)
AsseralyLIIP
Uai
eapaaies
52.1
113.2
96.7
95.0
97.
56.3
05.0
93.2
95.3
97.3
56.7
95.2
07.4
05.0
97.3
uallew
19..5
97.9
03.8
07.6
81.5
05.3
07.3
03.3
97.3
83.3
54.1
07.0
03.3
01.5
deceinggss
05.6
94.6
83.3
55.4
71.7
03.6
97.
96.3
97.5
11.1
96.7
98.8
98.3
97.5
71.
07
93.6
01.3
07.8
95.6
97.6
96.6
62.
96.1
37.6
96.3
6366
93.8
05.1
07.6
958
90.
92.4
96.6
93.3
14b3
63.9
91.6
953
$5.4
83
67.1
02.4
05.5
18.4
93.1
76.1
93.3
05.4
88.6
94
job4
78.1
95.
95.7
952
10.4
66.9
96.3
35.7
96.6
90.3
95.6
05.
Mion
06.0
91A
96.
95.5
969
71.
95.1
98-9
95.1
97.3
77.1
95.1
97.2
95.5
91.1
Table 18: Comparison of pixel-level anomaly detection in terms of subset-wise PRO on VisA.
I-dae
4-btt
candic
65.5
27.5
10.3
16.2
95.0
65.4
1012
22.3
96.2
55.3
65:7
50L5
92.8
95.2
95.4
capuaies
91.5
02.9
56.7
65.9
12.7
94.9
91.2
92.5
65.7
63.0
34.1
91.2
93.4
68.9
52.3
83.2
01.2
93.7
Gyszi
56.5
97.5
$1.9
86.8
82.1
543
878
8033
86.3
[8.1
999
58.1
8L.
85.
89.2
$4.2
18.5
pobl
43.2
05.8
$7.9
$1.7
96.3
47.5
70.2
86.5
11.3
55.3
53.1
70L
87.1
81.7
85.6
Job2
46.
730
13.4
75.9
72.3
48.0
74.0
16.3
18.9
17.5
$1.9
76
77.
7309
09.9
16.7
91.3
13.77
36.1
82.6
33.8
PEN Joym
72.6
95.7
96.2
96.8
972
753
95.7
98-4
98.8
97.1
76.3
95.8
96.7
96.8
91.1
Table 19: Comparison of image-level anomaly detection in terms of subset-wise AUROC on VisA.
L-dae ProeCAD
2ol64 ProeitDD
4-bet
AuntsiyCLIP
Ous
AusmaayyC.
Assual:CLIP
canfl
37.3
06.3
51.
70.3
91.
57.3
06.6
51.0
75.3
55.5
65.0
96.3
91.9
70.3
$1.5
caporks
51.8
79.3
85.1
$1.5
$8.9
56.6
81.6
16.7
11.5
00.1
83
81.2
97.
97.4
977
16.
96.1
96.1
91.1
Ryun
72.3
07.5
$8.0
91.0
30.9
13.
96
157
93.0
92.1
118
91.
90.6
9100
61.1
ancemell
69.1
873
87.3
85.
594
86
17.4
17.
14.1
869
811
873
83.0
88.4
83.5
85.5
13.3
16
70b3
71.1
13.8
77.
82.3
72.6
79-1
96.8
11.4
63.3
15.3
17.3
19-1
31
67.
75.5
Joh1
79-
83.7
81.3
67.7
05.5
37.1
41.7
98.2
96.9
92.4
90.8
53.1
98.5
99.6
92.4
99.9
56.6
98.4
99.3
92.4
99.8
Niza
30L8
12.1
13.4
053
$4.3
82.1
1637
730
14.1
Table 20: Comparison of image-level anomaly detection in terms of subset-wise AUPR on VisA.
Yis
SPADE:
WivCLIPP
I-dxe ProrpeAD
AnorssyCCLIP
Ctr:
SPADE WhaCT11P
2-chee PreejrAD
AnomalyCT.I
Gar
SPARE:
WHiTHI
Exikit ProrijiA
AsurrayCC1
cend
41.3
16.7
ME1
$1.1
96.1
51.
06.9
41.7
57.7
97.1
91.5
81.1
05.5
caperic
65.5
X7:11
900
13.7
96.7
08.3
191
15.7
15.7
96.9
71.9
10
19.01
1611
95.0
95.0
85-4
05.7
8113
06.1
15.4
05.1
trym
M6.1
96.
95.8
06.3
1638
05.4
58.1
95.9
05.8
musmeell
12.3
103
15.0
19.
381
87.2
1900
18.0
680
133
954
011
19.11
812
61.
72.1
193
313
$1.2
20h2
7ML3
$4.3
11.2
78.4
56.6
51.1
04.5
2053
58.5
66.2
$1.6
70L0
15.2
82.
07.3
73.5
7000
16.3
13.1
54.5
85.0
7001
140
59.6
96.3
66.2
45.1
852
85.4
$7.5
72.
$4.9
37.8
35.4
18.5
76.6
96.1
85.8
8544
89
20

---

Published as a conference paper at ICLR 2025
VisA
Normal
Anomaly
GT
Heatmap (1-shot)
Bottle
Cable
Capsule
Carpet
Candle
Capsules
Cashew
Chewinggum
Normal
Anomaly
GT
Heatmap (1-shot)
Hazelnut
Leather
Metal nut
Pill
Screw
Fryum
Macaroni1
Normal
enjo
Anomaly
GT
Heatmap (1-shot)
Tile
Toothbrush Transistor
Wood
Zipper
PCB2
Pipe fryum
Figure 4: Additional qualitative results from our method (1-shot), tested on MVTec and VisA.
21


Reviews:
[{"review_id": "PtqszCYuEu", "reviewer": "Reviewer_epHH", "summary": "This paper introduces a novel anomaly detection challenge: multi-class few-shot anomaly detection (AD). The authors examine the current few-shot AD methods using image-text models, such as WinCLIP and PromptAD, and identify the key limitations when applied to multi-class few-shot AD tasks. In response, they propose an instance-specific prompt generator that enhances the prompt's capacity to identify both anomalous and normal regions while mitigating issues that arise from shared prompts. Additionally, the paper introduces a multi-model prompt training strategy to strengthen model training and modifies the memory bank approach for multi-class tasks by proposing a class-aware memory bank.", "questions": "Please kindly refer to the weakness.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.The authors have innovatively proposed a new detection task that meets the practical industrial needs. They tested numerous outstanding anomaly detection algorithms on this task.\n2.The proposed anomaly detection algorithm performs well in multi-class anomaly detection within few-shot scenarios. This method incorporates the capabilities of several currently popular large models. The proposed prompt learning strategy is innovative.", "weaknesses": "1. The paper contains several issues that affect its clarity and focus. The introduction covers too broad a range of topics and fails to highlight the core theme of the article. Additionally, Figure 1 does not fully and clearly illustrate the methodology of this paper, especially the Guidance of Prompt Learning section. It also fails to adequately represent the complex visual guidance process described in Section 3.2.1. Furthermore, there is a grammatical error with the second 'S's in line 203 of the paper.\n2. The method contains too many modules, and the ablation experiments are insufficient to demonstrate whether the certain modules  contribute to the final experimental results, especially in the Prompt Generator Ablation part. The existing ablation experiments do not clearly show whether the projection network and cross-attention are truly useful. The method uses too many loss functions, and the analysis in the ablation experiments is not accurate enough. For example, the role of Synthetize Visual Features is not clearly stated.\n3. AnomalyClip and InCTRL are representative works in the same field but this paper does not compare their performance in the experiments section.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GtN619Ak8F", "reviewer": "Reviewer_Nv81", "summary": "This study proposes an instance-specific prompt generator and a category-aware memory bank aligned with a new one-for-all paradigm in few-shot anomaly detection.", "questions": "1. Given that both normal and anomalous object tokens 𝑂 are output from the same MLP, please clarify whether they utilize the same learnable token. Additionally, if the same token is used, it would be helpful to explain how the MLP architecture enables the generation of distinct normal and anomalous tokens.\n2. To validate the claim that Gaussian noise is not required, consider including an experiment comparing the proposed method to a version that incorporates Gaussian noise. Such a comparison could illustrate the impact of Gaussian noise on performance and provide evidence for this design choice.\n3. In Table 4, the shift from OVB to CAMV in the Memory Bank module when adding M2 could explain some performance improvements. To better understand each module's role, an additional ablation study that isolates the effects of the Prompt Generator and the Class-aware Memory Bank would clarify their individual contributions to performance in the one-for-all task.\n4. The identical performance results in Tables 1 and 2 between the class-wise training in the one-for-one setting and the one-for-all setting are intriguing. A more detailed explanation of how the method differs across these settings, as well as a discussion on why the performance remains the same, would provide valuable insights into the implications for the method's contributions.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This study proposes a novel prompt generation method, utilizing a class-aware memory bank to store visual features by class and extract normal and abnormal features tailored to each instance. As a result, it achieved the highest performance in the new one-for-all task.", "weaknesses": "- In the overall performance comparison table supporting the proposed methodology, the performance of \"one-for-all\" and \"one-for-one\" is identical, yet no interpretation is provided for this outcome.\n- In Figure 1, the representation of P is omitted.\n- The meaning of the output values of the newly applied Q-Former in this study is not explained.\n- In Equation 12, there is an undefined loss term.\n- In Table 4, it is essential to confirm whether variables outside the experimental modules were well controlled. Given that the highest performance was achieved by adding the M1 and M2 modules, it seems possible that other modules were incorporated at intermediate stages.\n- Table 5 lacks definitions and contains errors in the loss expressions, making comparison and evaluation challenging.\n- In Table 6, the AUROC performance for VisA at the image level differs, indicating a need to verify whether the experiments were conducted accurately.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BjokLIi3iq", "reviewer": "Reviewer_9Lmw", "summary": "This paper proposes a novel anomaly detection methodology for industrial applications in a one-for-all categories paradigm. It uses prompt tuning and contrastive learning to pull training images and normal prompts closer together and multi-level fusion to create pseudo-anomalies which are pulled together with the anomaly prompts.", "questions": "1. On line 218, how are the selected subset of F for prompt learning chosen? \n\n2. It is not clear to me why building the category-aware memory bank using image patch tokens as well as category tokens is necessarily better than the memory banks used in previous methods. Is it the case that query samples from one category were often being matched to memory bank items from other categories? This seems unlikely. \n\n3. How does the additional training and prompt tuning affect computation complexity / runtime compared with the base methods?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The methodology appears to be sound and introduces several steps.\n\nThe experimentation is comprehensive and the results show consistent improvement over baselines.\n\nThe paper is mostly well written and clear.", "weaknesses": "Please see questions", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xncqmxErEH", "reviewer": "Reviewer_AM88", "summary": "In this paper, the authors propose a novel \"One-for-All Few-Shot\" anomaly detection method, aimed at addressing the challenge of few-shot anomaly detection. Compared to traditional \"One-for-One\" approaches, the authors design a class-shared prompt generator that utilizes vision-language models (VLMs) to generate instance-specific prompts, improving the model's adaptability in few-shot scenarios. The method is trained to capture normality and abnormality in both visual and textual modality and introduces a category-aware memory bank to resolve the memory mismatch issue within the \"One-for-All\" paradigm.", "questions": "1. **More Experimental Details**:\n\nThe impact of semantic-level alignment loss has not been extensively discussed in **Table 5**. The authors may add more experimental results to demonstrate the effectiveness of this specific loss.\n\n2. **More Quantitative Analysis**:\n\nThe performance impact of adopting Q-Former and instance-specific prompt generator is still unexplored. For example, an extra experiment or performance analysis can be added to demonstrate the specific contribution of Q-Former to the method. My concern is that the original ViT and Q-Former are unaligned and frozen during the training, without pre-training, e.g. representation learning stage in BLIP2, they may not own the ability to align the text feature with the visual feature. Additionally, due to the CLIP having been pre-trained to align the two modality features, why the authors did not use it? In other words, the advantages of the proposed architecture are not obvious in the paper when compared with other frameworks.\n\n3. **More theoretical analysis**:\n\nAlthough the experimental results demonstrate the effectiveness of the method, the theoretical analysis is somewhat insufficient. For example, in the Prompt Generator Ablation and Loss Ablation, the authors can add one sentence to analyze and generalize why the proposed module is effective.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. This paper designs a novel task.\n2. The proposed method is innovative and practical.\n3. The experimental results show that the proposed method has achieved excellent performance on the two datasets.", "weaknesses": "This paper has some drawbacks and the authors can consider improving the paper's quality from the following perspectives:\n\n1. **Grammar Mistakes and Spelling Errors**, such as:\n\n(1) In the Abstract, it has \"...fix prompts for each class, our method **learn** a class-shared prompt generator...\". Here, \"learn\" should be in the third person singular form and changed to \"...fix prompts for each class, our method **learns** a class-shared prompt generator...\".\n\n(2) In line 236, it has \"we propose to encourage the alignment between a group of patches **an** the embedding of one prompt token\". It seems that \"an\" should be replaced with \"and\" to ensure the correctness of the sentence.\n\n2. **Confused Content**, such as:\n\nIn the abstract, the expression that 'We address the **mismatch problem** of the memory bank within one-for-all paradigm' is relatively ambiguous, and the specific definition of 'mismatch problem' is not clearly explained in **Sec.3 Methods**. The authors may provide more explanations about this term.\n\n3. **Differences from others**:\n\nAlthough the comparisons with PromptAD and other methods have been mentioned, there is a lack of detailed comprehensive analysis. More comparative content can be added to illustrate the specific advantages of the proposed method. For example, the author can add an independent section in the Appendix to illustrate WinCLIP/PromptAD has limitations in some specific scenarios, and how the proposed method outperforms them under these scenarios.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qxadTTHr3e", "reviewer": "Reviewer_6UaR", "summary": "This paper presents a new anomaly detection problem: multi-class few-shot anomaly detection (AD). The authors analyze current image-text model based few-shot anomaly detection methods such as WinCLIP, PromptAD et al. have problems in multi-class few-shot AD tasks and propose an instance-specific prompt generator, which not only improves the ability of prompt to capture abnormal regions and normal regions, but also prevents the problems caused by prompt sharing. \nIn addition, this paper proposes a multi-model prompt training strategy for model training, and improves the memory bank strategy for multi-category tasks to propose a class-aware memory bank.", "questions": "major\n1. I would like the authors to briefly introduce the core insights of the paper.\n2. There are many modules in the method part, and more ablation experiments are needed to demonstrate the contribution of each module, especially the instance-specific prompt generator module, in which the Projection Network and Cross Attention need to be ablated. In addition, the two losses of Synthetic Visual Guidance for Anomalous Prompt also require separate ablation experiments.\nminor\n3. 236 “group of patches an the embedding” “an” seems to be a misspelling\n4. Table 5, I couldn't find “L_c”, if the author meant “L_s”.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. This paper discusses a new and practical task.\n2. The method has some novelty.\n3. The experimental results show that the performance of this paper is very superior.", "weaknesses": "There are some problems in the organization of the paper, 1) the introduction is too redundant and not concise enough, and it is difficult to get the core views of the authors. 2) Some details of the method section are not clear enough and some parts are not reflected in Figure 1", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)