One-to-Normal: Anomaly Personalization for Few-shot Anomaly Detection
Authors: Yiyue Li
Shaoting Zhang
Kang Li
Qicheng Lao
Venue: NeurIPS 2024 poster (2024)
Keywords: Few-shot anomaly detection
Diffusion models
Image personalization
Primary area: machine_vision

Abstract: Traditional Anomaly Detection (AD) methods have predominantly relied on unsupervised learning from extensive normal data. Recent AD methods have evolved with the advent of large pre-trained vision-language models, enhancing few-shot anomaly detection capabilities. However, these latest AD methods still exhibit limitations in accuracy improvement. One contributing factor is their direct comparison of a query image's features with those of few-shot normal images. This direct comparison often leads to a loss of precision and complicates the extension of these techniques to more complex domains—an area that remains underexplored in a more refined and comprehensive manner. To address these limitations, we introduce the anomaly personalization method, which performs a personalized one-to-normal transformation of query images using an anomaly-free customized generation model, ensuring close alignment with the normal manifold. Moreover, to further enhance the stability and robustness of prediction results, we propose a triplet contrastive anomaly inference strategy, which incorporates a comprehensive comparison between the query and generated anomaly-free data pool and prompt information. Extensive evaluations across eleven datasets in three domains demonstrate our model's effectiveness compared to the latest AD methods. Additionally, our method has been proven to transfer flexibly to other AD methods, with the generated image data effectively improving the performance of other AD methods.

One-to-Normal: Anomaly Personalization for Few-shot Anomaly Detection
Yiyue Li! Shaoting Zhang"
Kang Li4
Qicheng Lao
West China Biomedical Big Data Center, West China Hospital, Sichuan University
School of Artificial Intelligence, Beijing University of Posts and Telecommunications Sichuan University Pittsburgh Institute, Sichuan University
*Shanghai Artificial Intelligence Laboratory
kang. 1i rwwearchhgeaml...........a... com, qicheng lao@bupt...... edu. en
Abstract
Traditional Anomaly Detection (AD) methods have predominantly relied on unsu- pervised learning from extensive normal data. Recent AD methods have evolved with the advent of large pre-trained vision-language models, enhancing few-shot anomaly detection capabilities. However, these latest AD methods still exhibit limitations in accuracy improvement. One contributing factor is their direct com- parison of a query image's features with those of few-shot normal images. This direct comparison often leads to a loss precision and complicates the extension of these techniques to more complex domains-an area that remains underexplored in more refined and comprehensive manner. To address these limitations, we introduce the anomaly personalization method, which performs personalized one-to-normal transformation query images using an anomaly customized generation model, ensuring close alignment with the normal manifold. Moreover, to further enhance the stability and robustness of prediction results, we propose a triplet contrastive anomaly inference strategy, which incorporates comprehensive comparison between the query and generated anomaly-free data pool and prompt information. Extensive evaluations across eleven datasets in three domains demon- strate our model's effectiveness compared to the latest AD methods. Additionally, method has been proven to transfer flexibly other AD methods, with the generated image data effectively improving the performance of other AD methods.
Introduction
Anomaly Detection (AD) has garnered considerable attention due to its wide applicability across
various domains, such industrial defect detection 220 30 14 27 medical diagnostics [37
21 43), video surveillance [22] 33 manufacturing inspection (39 9 This heightened interest is
primarily attributed its only reliance on normal samples and its adoption of unsupervised learning paradigm, where it typically learns from the distribution of normal samples to identify anomalies
by detecting outliers [46] 39 Traditional AD methods, including auto-encoder based 48 40).
GAN-based, and knowledge-basee
approaches 33, g 35), and others. Moreover, some diffusion- based methods [24] [1] have also emerged recently. Although most of these methods do not require annotated data, they do necessitate a substantial number of normal samples during the training phase to capture the distribution of normal samples effectively. However, this requirement has also severely constrained its advancement in various fields.
* Corresponding author
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Recent studies 33 14 38 in few-shot scenarios have improved AD tasks. Currently, state-of- the-art (SOTA) advancements [45] T6
TO 44 are primarily due the development of large pre-
trained Visual-Languagge Models (VI.Ms). For example, WinCLIP [16] first uses pre-trained CLIP, employing carefully designed text prompts and image feature comparisons to perform few-shot anomaly detection. AnomalyGPT [TO] eliminates the need for manually setting thresholds and supports multi-round dialogues. InCTRL [45] achieves general anomaly detection through in- context residual learning. Despite these innovations, the latest SOTA methods often rely direct feature matching between few-shot normal images (i.e., reference images) and the query sample. However, without deeply exploring their subtle features, it is often difficult to achieve precise feature comparisons, which can easily lead to unstable results, since the non-anomalous differences between query and references severely impact the prediction accuracy. Furthermore, the limited number of normal images serving as independent references also makes their prediction insufficient.
In this work, we hypothesize that: 1) To achieve more accurate prediction results, it is essential to compare the query image with its corresponding or most similar normal image. Ideally, this comparison should involve one-to-one transformation of the query image into its normal counterpart; 2) Additionally, one should also aim for robust and stable results, necessitating a comprehensive approach to accurately predicting results from multiple perspectives. To this end, we propose an anomaly personalization method for few-shot anomaly detection. To achieve more accurate, personalized comparative predictions, mentioned in 1), method employs a diffusion model to create an anomaly- free customized model inspired by BI. This model uses pairs of object text prompts and few-shot normal images (i.e., reference images) to explore the distribution of normal samples. Previous works [23 4 41 have demonstrated the effectiveness of such customized models. Furthermore, the suitability of diffusion models stems from their powerful customization capabilities, allowing for flexible control over intermediate steps and the generation process compared to other methods [26]. Additionally, for enhanced stability and robustness as mentioned in 2). we propose the use of triplet contrastive anomaly inference, i.e., in addition to comparing the query image with normal samples, we conduct comprehensive comparisons with personalized samples and text prompts. Furthermore, we use comprehensive state words and templates as text prompts inspired by [15] [6]. Ultimately, we synthesize predictions from diverse perspectives to yield the final anomaly score.
To summarize, this paper makes the following main contributions:
We introduce novel anomaly personalization method for few-shot anomaly detection, unlike other state-of-the-an (SOTA) approaches that directly compare query images with reference images, our method enables finer-grained comparison through one-to-normal personalization of query images, leading to enhanced prediction accuracy.
To achieve more stable and robust results, we propose triplet contrastive anomaly inference strategy. This approach facilitates a more comprehensive comparison by incorporating not only customized comparisons but also comparisons with anomaly-fiee samples and text prompts. The anomaly- free samples augment normal images sampled by an anomaly-free customized model.
Our proposed method demonstrates strong generalizability. We conduct comprehensive experiments across 11 datasets spanning three distinct domains: industrial, medical, and semantic. Moreover, the anomaly-frr free samples generated by our method can be used to augment the normal samples in most few-shot anomaly detection methods, enhancing the performance of some existing methods and demonstrating adaptability and flexibility.
2
Related Work
2.1 Anomaly Detection
Anomaly detection (AD) is a critical task in computer vision aimed at identifying samples that deviate significantly from the norm. Traditional AD methods can be categorized into several types: auto-encoder based [43 40). GAN-based, and knowledge- based approaches B & 35), and others.
Recent advancements in AD research focus on few-shot and zero-shot learning to overcome data limitations. Few-shot AD methods like RegAD 14 utilize pre-trained models and a few normal samples from the target domain to detect anomalies without extensive re-training. WinCLIP (16) meticulously designs a variety of prompts to ensure the model comprehensively covers all possible normal and abnormal scenarios. AnomalyCLIP [4] uses CLIP for zero-shot anomaly detection.

---

sample
Normal [object]
Anomaly-free
Anornal--fre samples
Generated nulti-leve features
Normal [objoct] Abnormal [object]
Knormal
Text encodder
Customization
nhormal
S
nommal images
Query image
emmples
Query image
Spect
P Pquery
Xy
Personalized image
Sp
Prediction
A low resolution close-up ... good
photo of normal [object]
persomatized
cersonalized multi-leval features
Localization
One-to-normal personalization
Triplet contrastive anomaly inference
Figure Overview of our proposed anomaly personalization approach. First, we use few-shot normal images to customize anomaly-free diffusion model (top left), similar to Dreambooth. Next, perform one-to-normal personalization of the query image, obtaining the personalized image (bottom left). Finally, adopt triplet contrastive anomaly inference method, which integrates anomaly scores from three aspects to obtain the final results. Sx. Sp and Stezt represent different contrastive anomaly inference processes.
They have also meticulously designed prompts but have overlooked the aspect of image comparison. In summary, many of the current few-shot AD approaches have primarily focused on text prompt refinement without detailed exploration in visual feature comparisons and sample generation.
2.2 Image Personalization
Personalized text-to-image generation has emerged as a pivotal area, particularly for creating highly personalized and contextually accurate images. DreamBooth [3] is the first to focus on fine-tuning diffusion models with specific subject images to generate high-fidelity renditions of those subjects, typically using 3 to 5 images for customization and incorporating prior preservation to prevent language draft. SuTi [4] leverages single model to learn from a multitude of expert models, each fine-tuned to specific subjects, allowing for instant personalization using in in-context learning with only a few examples. Another innovative approach 23 involves "concept neurons" in diffusion models, identifying clusters of neurons that correspond to specific subjects within pre-trained model. These methods have achieved significant results in image generation. However, most of these methods have been applied to image editing and synthesis, and have yet to be explored for anomaly detection.
Method
3.1 Overview
We propose novel anomaly personalization method to improve few-shot anomaly detection. First, to obtain more precise results, we aim for one-to-one customized comparison, where the query image compared with the image derived from itself, transformed towards an anomaly-free distribution to align the anomalous distribution with the normal state. This is achieved by employing an anomaly-free customized model (Sec. 3.2) followed by one-to-normal personalization of the query image (Sec.) (.3). Second, to enhance the detection stability and robustness, we further propose triplet contrastive anomaly inference process (Sec. 3.4). where the query image is thoroughly compared with its personalized version, augmented anomaly-free frre samples, and text prompts for comprehensive analysis.

---

3.2 Anomaly-Fres Customized Model
To develop a customized model for normal objects, we choose diffusion model 12 as the framework, building on the work based on [i]] which has proven effective for specific object customization. To enhance the diversity of normal object samples given limited data, we first apply data augmentation to normal reference images. Subsequently, to better align objects with their textual descriptions, we provide the diffusion model with series of demonstrations Co. using augmented reference images paired with their corresponding texts for specific objects. Specifically, for each object (e.g, cable), we prepare a series of augmented reference images (i.e., few-shot normal samples) Toormall E Draan and their matching text c, i.e., a photo of normal [object], for tailored training of the diffusion model. Here, the specific category name (e.g.. 'cable') is used to replace 'object', Additionally, another set of demonstrations C, is employed design class-specific prior preservation loss, aimed at promoting diversity and preventing language drift, similar to [31 4 To obtain an anomaly-free customized model (y c) parameterized by 0 on an object o, we constrain a pre-trained diffusion model on the image cluster Co with the denoising loss as:
0 = arg min E (x.,e),C.UE, Do c) [0]]
where e is a short description of normal images Tnormal, and Znommll. latent version of the input noised by i steps. Note that the dimensionality of both the image and the latent is the same throughout the entire process. Therefore, the anomaly-free customized model learns the distribution of normal images Paemaa conditioned on the text prompt where unomaly-free samples can be generated.
3.3 One-to-Normal Personalization
Our proposed one noremal strategy is designed to transform the query image of object toward the distribution of its normal samples. It adaptively retains the normal characteristics while the anomaly parts gradually transformed with no defects. This is similar to previous work (3) 26 for image editing; however, our strategy specifically focuses on utilizing it to align with the normal manifold.
Specifically, we first design text prompts to maximize the retention of information in the normal regions of the query image while transforming anomalous areas towards a normal state. To mitigate the influence of other factors (c.g., contrast, image quality), simulate all potential normal states using these prompts. Inspired by [16] [44), we curate a template list for various potential image physical states, e.g., low-resolution photo of a normal object. Additionally, we include descriptors for common states shared by most normal objects, such as a photo of normal object without flaw. We provide these descriptors for normal state prompts in Appendix A.1
Given a series of normal state prompts {c) described above, we aim to select the generated image that most closely resembles the normal state for the one-to-normal personalization of the query image Tg. Specifically, we reconstruct a set of reconstructed images { where each text prompt e; guides the diffusion process of the customized anomaly-free model. We select the optimal text prompt ey that most closely resembles the normal state from the prompt set:
set:
Cg arg min C(x ..)) = arg min L(zyg Do(Eg,e, 6s
(2)
where De represents the anomaly-free customized diffusion model derived from the Section 3.2 and L represents SSIM. Finally, we obtain the personalized images ázi
fg = DD( + v1 - ane, Cgy t),
where a is noise schedule, (O( represents Gaussian noise, and f is the diffusion step.
Following (25). during this noising phase, the hyperparameter is meticulously adjusted dictate the extent of the one-to one-to-normal personalization, which determines the similarity between query image x, and personalized image i. A relatively lower / means insufficient personalization where a substantial portion of the original query is retained. As approaches 0, the generated image becomes identical to the input query image, and the text prompt's condition has no effect. Conversely, setting to T initiates a complete forward diffusion process, consequently erasing all information in 2, and

---

allowing the generation to be fully guided by text prompts cy toward the normal manifold. Ultimately, we bridge the distributions of Pquery and nommal for obtaining P persomaizedd from which we can sample personalized images for subsequent anomaly detection tasks.
3.4 Triplet Contrastive Anomaly Inference
To achieve comprehensive predictions and enhance the precision and robustness, introduce triplet contrastive anomaly inference, i.e., in addition to comparing the query image with anomaly-free samples and text prompts, also compare with our personalized images. Finally, we integrate predictions from three comparison aspects to mutually complement each other.
One-to-one personalized comparison. We divide the CLIP image encoder into multi-feature extraction blocks, each designed to capture the intermediate features of the input image at different levels. For query image xi and its corresponding personalized image Eig, the model extracts features F. E hgxxwxd from Fg and Fg hxxxx from E. The comparison score Sp between the query image and its personalized image is then computed by evaluating their similarity in the extracted features each level:
1 Σ Sp
g (Fo.s Fqi)),
where denotes the 1-th level feature extraction block, n represents the total number of blocks, and (.) signifies the cosine similarity function, represents the grid number.
Anomaly-free sample comparison. Our anomaly-free sample pool comprises set of normal reference images and generated normal images, better representing the distribution of normal samples. We first generate normal samples from the customized anomaly-free model Dg in Section 3.2 and incorporate them into the anomaly-free sample pool for subsequent prediction tasks, as previous studies have demonstrated the diffusion model's capability to synthesize high-fidelity images, which have been proven effective in many works [ 25). To further expand the anomaly- -rre sample pool, we use the same process described in Section 3.3 but with settings that more closely align with the distribution of normal reference image instead of the query image. Then, we employ the same CLIP image encoder to extract multi-level features from anomaly- samples (e.g.. 30) and store these features in memory bank M. The prediction score SN between the query image and anomaly-free samples can be expressed as:
1 n Σ
SN
max min (1 (Fg,t, mi)))
mEM
Text prompt comparison. To calculate the anomaly score between the query image features and the text prompt features, we categorize text prompts into two types: normal and abnormal objects, similar to [16, 5 15). We aim to cover more possible states for these objects to better simulate the various potential conditions of the images, similar to [16]. Specifically, we obtain text features by applying the CLIP text encoder to set of predefined normal and abnormal prompts and then calculating the average of these features, denoted as Ftezt [8 xd To facilitate comparison, the global image feature for anomaly detection obtained from the CLIP image encoder is represented Fy E TR Lxd These two features from text prompts and the query image are then used to calculate the anomaly score follows:
Stent = softmar(FF) F.T
(6)
Here, the score Stezt represents the probability corresponding to the anomaly.
The final prediction result generated by combining outputs from three branches: the personalized image, anomaly-free samples, and text prompts. These three sets of output scores provide comple- mentary information for collaboration. The final anomaly score can be obtained by combining the three branches:
5

---

Ascore Sp aSx SStett
where B are the hyper-parameters.
4 Experiments
4.1 Experiment Setup
Dataset and Evaluation Metries. We validate the effectiveness of our method across 11 datasets spanning three distinct domains: industrial, medical, and semantic domains. In the industrial domain, utilize multiple datasets including MVTec-AD (2). Visa [46], KSDD [34,, AFID [3), and ELPV 77. For the medical domain, we incorporate datasets covering various modalities such as magnetic resonance imaging (MRT), computed tomography (CT), and optical coherence tomography (OCT). Specifically, the medical datasets include OCT2017 [1], BrainMRI, HeadCT [3], and employ datasets: MNIST [19] and CIFAR-
RESC 13. For semantic anomaly detection,
10 [8] These semantic datasets are utilized following the one-vs-all strategy. where one class is designated normal, and all other classes are treated as anomalous. We follow [45) for the dataset partitions. We only use data from the MVTec-AD dataset as auxiliary training data. When testing on the MVTec-AD dataset, we use auxiliary data from the Visa dataset. The anomaly detection performance is evaluated using the Area Under the Receiver Operating Characteristic Curve (AUROC) and the Area Under the Precision-Recat Curve (AUPRC).
Competing Methods and Baselines. We compare our proposed method with various state-of-the-art anomaly detection (AD) methods, including traditional AD approaches (full-shot) such PaDiM (6] and PatchCore 30). both of which are adapted to the few-normal-shor setting. Additionally, we compare traditional few-shot learning method, RegAD [4 and prompt learning method CoOp [2]
along with the latest AD methods based on vision-language models, including WinCLIP [16] and
InCTRL 45). To further validate the flexibility and applicability of our method, evaluate our generated anomaly-free samples using four state-of-the-art anomaly detection methods: PaDiM, PatchCore, WinCLIP and InCTRL.
Implementation Details. We select Stable Diffusion V1.5 29 and utilize the CLIP with ViT-L/14 ar- chitecture (28 for all tasks. Our anomaly- mirre customized model is fine-tuned using Dreambooth BI] and all model parameters are frozen subsequent tasks. All the experiments are trained by use of PyTorch NVIDIA GeForce RTX 4090 GPU. Our anomaly detection task is too- setting including 2-shot, 4-shot, and 8-shot scenarios using only normal images. The ratio of the 1-step is 0.3. We set the parameters and / for Ascore to and 0.5, respectively, across all datasets. The image-level memory bank is set to 30, and the inference resolution is 240 240. During inference, use the same text prompts as WinCLIP [16). To validate the improvement of other anomaly detection methods with our generated data, we use our generated anomaly-five free samples to augment the datasets for these methods. For the MVTec dataset, we generate 100 anomaly- samples per subclass to expand the reference dataset, while keeping all other settings identical to previous works.
4.2 Results
Comparative Analysis with Industrial Datasets. In Table 1 we comprehensively compare five different datasets from the industrial domain, with the MVTec-AD and Visa datasets containing 15 and 12 subclasses, respectively. We also conduct an in-depth analysis across various few-shot scenarios (i.c., 2-shot, 4-shot, 8-shot) verify the robustness of our method. The experimental results demonstrate that our method outperforms other approaches, showing superior performance, especially in the 4-shot and 8-shot scenarios, where it achieves the best results across all datasets. In the 8-shot setting, our method achieves higher AUCs by 3.4%, 4.1%. 0.6%, 1.2%, and 0.9% on the KSDD, ELPV. AFID, KSDD, Visa, and MVTec datasets, respectively, compared the second-bes baseline. Notably, when compared with the latest InCTRL method on the AFID dataset, our method achieves higher AUCs by 2.2%, 3.6%, and 4. 1% in the 2-shot, 4-shot, and 8-shot settings, respectively. Additionally, the results indicate that our method is less affected by different runs, with variance significantly lower than other methods, supporting its superior stability and robustness. Finally, as shown in 2 the anomaly map indicates that our method accurately identifies abnormal regions

---

Table 1: A quantitative comparison of our proposed method and other methods using (AUROC (%), AUPRC (%)) as the evaluation metric on industrial datasets. We show the average performance and standard deviation across five runs, with the top value highlighted in bold for each comparison.
Methods
PADIM
Puschcoes
ResAD
CoOp
WICCLIP
MVTee
(55.4334, 939:122
(6406447 53.72344
(85 92.250.7)
(83.1219 56.520.7)
(940515,
9692044)
(95.1:9.5, 973::44)
ViAA
68.0244, 71.9227)
(81.422, 841:233
(55.7.53, 61443.7)
(806423,
53.5119
(842224, (5.922.)
(572211, (158:22,877:116)) 13)
(858:22,
87.7216)
2-shot
KSDD
72.111.5 33.760.)
(902406
(49.9.0.8 17.3819)
(99.7.06
54330.4
(94.206
56.5204)
917:09)
91660.6
(78422.8 52.9334)
(73.951.7, 37.3:03) (56.427.2 27.5335)
(68.7662
44330.55 (7266455 50.0:43)
(76.1:25, 519:23)
(783 536:1.2
ELPV
(99,428.3,
70.725.)
(71.633.), $40:2.1) (57.) 67.920.5)
(76.221)),
54.100.2) (72.62200 34.0-10)
(8399:0.3, 91.3:03)
(85.6:0.5 928::0.5)
MVTx
(90.511.8
90.951.3)
(88.5126 (50:13) (66.1:3.2 $464266
(87.466.7,
92.40.5) (94.0221) (48:221,96::0.8)
(84.5115 97 67.2:0.)))
(966:1.2, 97.8-06)
(73.563.1.
75.8.18)
(6434255
(5744442, 628534)
(813611 54241.6 (85.862.5 67.5x2.3)
(87.7&19, 90.2:277
90.7:00))
4-bbo
(74266.4 35.1
(9233:65,
70.3:133 (52.54277 176503) (002406 99.4144 (943504 568:0.3
(9755406 92.4=115)
(97.8:0.8 92.4-07)
ARID
(787743.8
54.045.)
(73.3:6.2, 37.7-01) (996474 194:3.1) (7200))7, 454:14) (7644255 51.3:177
(7900115 (48:116
(826::0.7, 57.9:1333
ELPV
66.2200 72.4567)
(75647.3, (7.1 (996140 S88:15) (78.1+02, M67:0.3) (7544009, 5644:04)
(84641.1
91.11,1116600 91.609)
(87.3:0.5, 928--.6)
MVTec
882061.6 92.721.))
(922419 962211) (67.4133, 15552.) (MODLA 9333607) (947225, 91320.9
(953813, 97.7.06)
(96.2:0.88
VisA
(768833.2 72.12224)
(560826 87.322) (589140, 643232) (822222), 5488200 (808220 800:2.1)
(88.722.1
90.4225)
(89.9:1.3 91.0:005)
8-bbo
KSnD
(76.9+377. 88.444.5
(90.54633 703+09) (99442.9, 34649.1 (8980055 578-6.1) (94.H001) 16.5-01)
(978:06 12
(98.-0.77 92.7-1.8)
ARI
(7.2442.5 55.5+3.1)
(14.56.2 189+0.3) (903162,, 314466) (768H08 S14H0,3 (76611.5 562--211
(806+36 56.1-44)
(54.7 63.1-1.5)
(73449177 708+144) (837416 91.5-.) (63342.7, 596+1.5) (81.7+1.2 90.+0%) (814+)
Abnormal images
Our results
Abnormal
images
Our results
Figure 2: Visualization of representative results for pixel-level anomaly localization of our proposed method on different datasets.
while being less likely to falsely recognize normal regions as anomalous. This further validates the accuracy of our approach. More anomaly maps and localization results are provided in Appendix A.2
Comparative Analysis with Medical Datasets. In Table 2 we compare four medical image datasets from three different modalities (CT, MRI, OCT). Our method achieves superior few-shot anomaly detection performance across these diverse modalities, outperforming other methods in most datasets, particularly on the OCT2017 and RESC datasets. Notably, on the RESC dataset, our method surpasses the second-best method by 3.9%, 5.9%, and 4.6% in the 2-shot, 4-shot, and 8-shot settings, respectively. The suboptimal performance of other baseline methods on these datasets may be due to the unique characteristics of the medical domain, where much of the knowledge is not well-explored in VLMs pre-trained mostly on non-medical data. These results demonstrate that proposed anomaly personalization strategy alleviates these domain-specific challenges more effectively. Additionally, because our method is highly effective in specific domains, it demonstrates greater stability compared to other methods, further proving its robustness.
Comparative Analysis with Semantic Datasets. Here, we compare two semantic datasets using the one-vs-rest strategy, where one class is considered normal, and all other classes are treated as anomalous. The results show that our method achieves the best performance across three different few-shot settings. In the 8-shot scenario, our method reaches AUROC scores of 93.6(%) and 94.9(%) on the MNIST and CIFAR- 10 datasets, respectively. These experiments demonstrate that our method is also more effective on semantic datasets compared to other basclines.
7

---

Table 2: A quantitative comparison of our proposed method and other methods using (AUROC (%). AUPRC (%)) as the evaluation metric on medical datasets. We show the average performance and standard deviation across five runs, with the top value highlighted in bold for each comparison.
Methods
Datssets
PaDiM
Prtcheore
RegAD
Te
WinCIIP
[sCTRI
Ours
OCT2017
(730+36 86.+211)
(701452 84.--.9)
(94.7+22, 98.3+07)
(94.+26 98.5+224)
(967-099 990-14
BrainMRI
(65.74122 90.2466
92.1+11)
(440+120, 87.2-65)
(9144112, 98.40.3)
197.1+14
2-shox
HeadCT 8766122
(73.6196 91.3002)
(002211 954
(9152155 97.54122)
(92.92255 98.12113)
(942211 98.6:113)
RFSC
(60.4+54, 66.2+3.4)
(60.2439,6) 65.8-366)
(85.46+2.1.
79.5+0.4)
(88.3434,114) 33,434481,482255
(92.2-05, 84.3+1.2)
OCT2017
(7681288 88.1220)
(72.6813.1. 36.141338)
(96.22277 98.210.)
(96.81244
98.91224)
RrainMRI
(792444, (56+111)
(794440, 94.5+1.7)
(57.1+149, 90.0+4.1)
(941+-022 99.0+0.1)
(97.5+16, 99.4+1.3)
(982+1.3, 99.6+0.55
4-shot
HeadCT
(622411.88010)
(805106, 94.1409)
(52.215.0 81.0228)
(9122003, 97.40.2)
09.31113 98.41.1)
(94.7106 99.010.7)
(695+34 66.++21)
(68542.7, 65.3-2.5
(87.942.1 80.8+13)
(88.742.3, 81.1+24)
(94.6 93.2+0.9)
OCT2017
(80.612.1, 90.41166
(74,81299 88.61477
(9700299, 99.0.005)
(91,422.1,9911))
(99.3115, 99.7.0.8)
BrainMRI
04.4477)
(81.2+14 96.740.)
(63.2479 90.8-1.3)
(94440.1, (9.1+00)
99.-+0.)
198.6+0.7, 99.6-0.5)
8-sbox
HexdCT
(66.1 89.6.99)
(81.7134, 93.11006)
(62.8126. 88.121.4)
(91.5008, 97.510.3)
(93.6.0.8
98.5.0.5)
(94.810.6 99.1100.)
RESC
(71.2224, 67.9129)
(68.713.2 65.521.9)
(88.92.299 83.121.6)
(90.612.7, 83.4118)
142211..77...11
Table A quantitative comparison of our proposed method and other methods using (AUROC (%), AUPRC (%)) as the evaluation metric on semantic datasets. We show the average performance and standard deviation across five runs, with the top value highlighted in bold for each comparison.
Datssets
Patcheore
30
ReyAD 14
CoOp
WiCCIIP L6
IoCTRL K5
Oars
MNIST
6..4604,,000 6+0.11
(52.5=300 36.36))))
(55.7406
(S1.0=0 96.+0L))
(89.200) 97, 97.5+l14)
(92660055 98.9:0.4
2 2-bbut
CIRAR-10
2-0.9 92.6+12))
(53.4005,
900+0.3)
(52.7111, (1.1002)
(92.5=0.1, 99.0+01)
(93.5+0.2, 99.2+00)
(94.1=0.2, 99.3-0011
MNIST
(833=009 33.30099992220...
(54.8=5.3, 91.6655,,91.6133. 1.)
(56.3+044 02.9m022)
(85.1=10. +7.1
(X02+1.6 98.0+0.77
(92.900.7, 99.0=)1.6)
CIFAR-10
63.9..0,99....000
(53,4002, 90..0.11
(33.710.5, 91.5.0.3
(92.710.1) 99.01.00)
99.21044
(94.6.0.2, 99.3.0.2)
MNIST
(87.600 97.90..1)
(54.7=663, 91 91.911.))
(56.7#017 93.7=0.4
(86770.77 97 97.4+001)
(92.080.3, 98.9+0.1)
(93.6=0.2, 99.5+0.3)
8-bbt
CIFAR-10
(67 0.666 94.2.10.)
(55.5108, 91.110.1)
(54.2.005, 92.0.033)
(92.8.101. 99.0110.0
(94.510.2. 99.41001)
(94.9.0.1, 99.510.1)
Normal
Query
Personalized
Normal
Query
Personalized
Figure 3: Visualizations of anomaly personalization. The red box of the query image indicates anomalous regions.
Visualizations of Anomaly Personalization. Figure 3 compares the normal images (i.e., reference images), query images, and our resulting personalized images. As depicted, there are some differences between the normal image and the query image in terms of non-anomalous features, such as position, shape, and texture. Our personalized image retains most of the normal regions from the query image, with the anomalous regions largely converted to normal regions. This visualization further substantiates the precision our method compared to existing few-shot AD approaches.
4.3 Ablation Study
The effectiveness of generated anomaly-free samples. To validate the effectiveness of our method in generating normal samples, we apply
to multiple AD methods, including Patchcore, RegAD,
WinCLIP and InCTRL. As shown in the first row of Figure 4 our method demonstrates improvements

---

Paichrare -anemally fre wenple VS Ours RegAD US fanamaly fiee samples VX Ders WICCIPP VS "ateendyy free comple Dans INCTRE, VX Faseonally free cample Ourc
Average improvement AUROC
I-het
4-bbtt
Sohid
2-tbot
4-Qht
Sobut
2-bot
4-bet
I-shec
2whktt
4-hve
Average improvenent AUROC18))
1914
1335
1994
144
+35
103
VKA KXND
BedrT
VbA
Figure 4: The effectiveness of our generated anomaly-free samples for other AD methods. The red numbers highlight performance increases.
in the 2-shot, 4-shot, and 8-shot settings across all three methods. Notably, for the RegAD method, the AUC is improved by 6.8(%), 8.7(%)) and 10.2(%)) respectively. Additionally, we compare our generated strategy with five other AD datasets. After incorporating our anomaly-free samples, the four AD methods show varying degrees of improvement on industrial and medical datasets (Visa, KSDD, AFID, OCT2017, and headCT). particular, the Patchcore and RegAD methods exhibit significant improvements the OCT2017 dataset, with increases of 10.9(%) and 8.6(%), respectively. While InCTRL has already achieved high performance, our method still provides certain level of enhancement. This experiment demonstrates that the data generated by method can enhance existing AD methods, proving the flcxibility and adaptability of our approach.
The effectiveness of the triplet contrastive anomaly inference. Here, we discuss the impact of each branch in our triplet contrastive anomaly inference strategy, analyzing datasets across three different domains. Table validates the effectiveness of our strategy, showing that it is optimal in most datasets. When only text prompts are used (Stezt), meaning there no reference image input, particularly low results are observed in the medical datasets OCT2017 and RESC. However, when reference images are supplied (either Sp or 5x)). ), a significant improvement observed. This might be because the model lacks inherent understanding of specific medical domains, thus requiring normal images as references. When each strategy is used individually, we find that most results are not as good as when the strategies are combined, indicating that these strategies complement each other. When all three strategies are employed together, most datasets demonstrate optimal performance. However, in certain cases, such as the KSDD dataset-wiie all scores does not consistently yield the best results. This inconsistency may stem from the high diversity in the distribution of normal images, which presents challenges for the AD model's learning process. Nevertheless, across the majority of datasets, the combined use of three strategies leads to relatively strong performance, showcasing that our triplet contrastive anomaly inference strategy effectively enhances accuracy.
focuses surface defect inspection -utilzzing
The effectiveness of text prompts in anomaly personalization. We also conduct ablation periments on the settings of the text prompts 6 used in the one-to-normal personalization stage (Section 3.3 In this work. employed three prompts to generate three images (one for each prompt). The results ddemonssttt that our designed text prompt strategy effective for normmll personalization, where most datasets show improvement.
Inference time. Regarding inference time, our proposed method slightly higher (+200-300ms per query image) than that of WinCLIP (389ms) and InCTRL (276ms). If necessary, we can further increase the inference speed by reducing the number of generated samples or decreasing the memory bank size. When using single prompt corresponds to generating only personalized image, the

---

Table 4: Ablation study on the impact of three different contrastive anomaly inference branches in 8-shot setting. We show the average performance (AUROC (%)) across five runs, with the top value highlighted in bold.
Strutegies
Industrial
Medical
Somantic
Sp
Sture
MV
VisA
KSDD
AFID
ELPV
OCT2017
BrainMRI
HesdCT
RESC
MINIST
CIFAR-10
91.8
78.2
94.5
72.8
73.1
45.3
92.4
89.6
39.6
69.2
91.3
V
93.1
86.3
93.1
80.5
83.3
87.1
89.6
86.0
89.6
91.6
93.6
93.7
88.6
96.3
85.2
86.1
94.3
90.3
92.6
93.9
84.3
/
98.7
96.0
89.6
98.6
84.3
88.3
99.2
94,6
92.6
94.2
92.6
88.2
96.2
89,9
98.4
84.7
90.6
99.3
98.6
94.8
95.2
93.6
94.9
Table 5: Ablation study on the text prompts used in one-to-normal personalization on all datasets in 8-shot setting. We show the average performance (AUROC (%)) across five runs, with the top value highlighted in bold.
Incustrial domain
Medical domain
Semantic
Stratgy
MVToc-AD
KSDD
ELPV
OCT2017
BrainMRI
HeadCT
RESC
MINIST CIFAR-10
89.8
93.9
94.6
refined 96.2
89.9
98.4
84.7
90.6
98.6
94.8
95.2
93.6
94,9
required inference time (326ms) is slightly lower than that of WinCLIP, while still demonstrating superior performance compared to other methods.
5
Conclusion
We introduce a novel personalized few-shot anomaly detection method that enhances prediction accuracy through one-to-normal personalization of query images. Unlike other sfate-of-the-ard approaches that directly compare query images with reference images, our method enables finer- grained comparison. Our triplet contrastive anomaly inference strategy further stabilizes results by incorporating personalized images, anomaly- samples and text prompts, facilitating more comprehensive comparison. Extensive experiments across 11 datasets in industrial, medical, and semantic domains demonstrate the method's generalizability and effectiveness. Moreover, the anomaly-free samples generated by our method can augment the normal samples in existing few-shot anomaly detection techniques. Experimental results also demonstrate that method improves the performance of several existing few-shot anomaly detection techniques.
I Limitation Achieving precise detection necessitates comprehensive exploration of the distribution of each category, which in turn requires few reference images from each category. Consequently, our current method is not applicable to zero-shot scenarios. Future research will focus on enhancing the capability to identify anomalies in zero-shot settings more accurately, as well as exploring real-time applications and open-vocabulary scenarios.
Broader Impacts This article focuses on customized few-shot anomaly detection method, which offers more precise and stable anomaly detection. It has the potential to enhance development in industrial and medical fields. There are no negative societal impacts involved in this work.
Acknowledgement
This work was supported in part by the National Key Research and Development Program of China under Grant 1020YFB1711500 and Grant 2020YFB1711503, in part by the 1.3.5 project for disciplines of excellence, West China Hospital, Sichuan University, under Grant ZYYC21004 and ZYAI24053; in part by the Aier Eye Hospital-Sichhi Sichuun University Research Grant 23JZH043.
10

---

References
[1] Mohamed Akrout, Bálint Gyepesi, Péter Holló, Adrienn Poór, Blága Kineső, Stephen Solis, Katrina Cirone, Jeremy Kawahara, Dekker Slade, Latif Abid, Diffusion-based data augmentation for skin disease classification; Impact across original medical datasets to fully synthetic images. In International Conference on Medical Image Computing and Computer-Assister Intervention, pages 99-109. Springer, 2023.
[2] Paul Bergmann, Michael Fauser, David Sattlegger, and Carsten Steger. Mvtec ad-a comprehensive real-world dataset for unsupervised anomaly detection. In Proceedings of the IEEE/CVF conference on computer vision and patiem recognition, pages 9592-9600, 2019.
[3] Tri Cao, Jiawen Zhu, and Guansong Pang. Anomaly detection under distribution shift. In Proceedings of 2023.
the IEEE/CVF International Conference Computer Vision, pages
[4] Wenhu Chen, Hexiang Hu, Yandong Li, Nataniel Ruiz, Xuhui Jia, Ming Wei Chang, and William W Cohen. Subject-driven text-to-image generation via apprenticeship learning. Advances in Neural Information Processing Systems, 36, 2024.
[5] Xuhai Chen, Yue Han, and Jiangning Zhang. A zero-ffew-shot anomaly classification and segmentation method for evpr 2023 vand workshop challenge tracks 1&2: 1st place zero-shot ad and 4th place few-shot ad. arXiv preprint arXiv:2305. 17382, 2023.
[6] Thomas Defard, Aleksandr Setkov, Angelique Loesch, and Romaric Audigier. Padim: a patch distribution modeling framework for anomaly detection and localization. In International Conference on Pattern Recognition, pages 475-489. Springer, 2021.
Sergiu Deitsch, Vincent Christlein, Stephan Berger, Claudia Buerhop-Lutz, Andreas Maier, Florian Gallwitz, and Christian Riess. Automatic classification of defective photovoltaic module cells in electrolu- minescence images. Solar Energy, 185:455-468, 2019.
[8] Hanqiu Deng and Xingyu Li. Anomaly detection via reverse distillation from one-class embedding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9737-9746, 2022.
[9] Dong Gong, Lingqiao Liu, Vuong Le, Budhaditya Saha, Moussa Reda Mansour, Svetha Venkatesh, and Anton van den Hengel. Memorizing normality to detect anomaly: Memory-augmented deep autoencoder for unsupervised anomaly detection. In Proceedings of the IEEF/CVF international conference computer vision, pages 1705-1714, 2019.
[10] Zhaopeng Gu, Bingke Zhu, Guibo Zhu, Yingying Chen, Ming Tang, and Jinqiao Wang. Anomalygpt: Detecting industrial anomalies using large lision-laguage models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 1932- 1940, 2024.
[11] Haoyang He, Jiangning Zhang, Hongxu Chen, Xuhai Chen, Zhishan Li, Xu Chen, Yabiao Wang, Chengjic Wang, and Lei Xie. Diad: diffusion- framework multi-class anomaly detection, arXiv preprint arXiv:2312.0660 06607, 2023.
[12] Jonathan Ho, Ajay Jain, and Pieter Abbecl. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-663 6851, 2020.
[13] Junjie Hu, Yuanyuan Chen, and Zhang Yi. Automated segmentation of macular edema in using deep neural networks. Medical image analysis, 55:216-227. 2019.
[14] Chaoqin Huang, Haoyan Guan, Aofan Jiang, Ya Zhang, Michael Spratling, and Yan-Feng Wang. Regis- tration based few-shot anomaly detection. In European Conference on Computer Vision, pages 303-311 319- Springer, 2022.
[15] Chaoqin Huang, Aofan Jiang, Jinghao Feng, Ya Zhang, Xinchao Wang, and Yanfeng Wang. Adapting visual- language models for generalizable anomaly detection in medical images. arXiv preprint arXiv:2403.12570 2024.
[16] Jongheon Jeong, Yang Zou, Taewan Kim, Dongqing Zhang, Avinash Ravichandran, and Onkar Daheer. Winelip: Zero-Ifew-shot anomaly classification and segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19606-19616. 2023.
[17] Daniel S Kermany, Michael Goldbaum, Wenjia Cai, Carolina CS Valentim, Huiying Liang, Sally L Baxter, Alex McKeown, Ge Yang, Xiaokang Wu, Fangbing Yan, et al. Identifying medical diagnoses and treatable discases image-bases based deep learning, cell, 172(5):1122 1131, 2018.

---

[18] Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. Cifar- 10 (canadian institute for advanced research). 2010.
[19] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied document recognition. Proceedings of the IEEE, 86(1 1):2778-2324, 1998.
[20] Chun-Liang Li, Kihyuk Sohn, Jinsung Yoon, and Tomas Pfister. Cutpaste: Self-supervised learning for anomaly detection and localization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9664-9674, 2021.
[21] Yiyue Li, Qicheng Lao, Qingho Kang, Zekun Jiang, Shiyi Du, Shaoting Zhang, and Kang Li. Self- supervised anomaly detection, staging and segmentation for retinal images. Medical Image Analysis, 87: 17:1020 2023.
[22] Yusha Liu, Chun-Liang Li, and Barnabás Póczos. Classifier two sample test for video anomaly detections. In BMVC, page 71, 2018.
[23] Zhiheng Liu, Ruili Feng, Kai Zhu, Yifei Zhang, Kecheng Zheng, Yu Liu, Deli Zhao, Jingren Zhou, and Yang Cao. Cones: Concept neurons in diffusion models for customized generation. In International Conference on Machine Learning, pages 21548-21566. PMLR, 2023.
[24] Victor Livernoche, Vincet Jain, Yashar Hezaveh, and Siamak Ravanbakhsh. On diffusion modeling for anomaly detection. arXiv preprint ar3:v22305. 18593, 2023.
[25] Lorenzo Luzi, Ali Siahkoohi, Paul M Mayer, Josue Casco- Macoo-Rorriguez. and Richard Baraniuk. Boomerang: Local sampling on image manifolds using diffusion models, arXiv preprint arXiv:2210.12100, 2022.
[26] Chenlin Meng. Yutong He, Yang Song, Jiaming Song. Jiajun Wu, Jun Yan Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equations. In International Conference on Learning Representations, 2021.
[27] Guansong Pang, Chunhua Shen, Longbing Cao, and Anton Van Den Hengel. Deep learning for anomaly detection: A review. ACM computing surveys (CSUR), 54(2);1-38, 2021.
[28] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 3748-8763. PMLR. 2021.
[29] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björ Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 00684-10695, 2022.
[30] Karsten Roth, Latha Pemula, Joaquin Zepeda, Bernhard Schölkopf, Thomas Brox, and Peter Gehler. Towards total recall in industrial anomaly detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14318-14328, 2022.
[31] Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dream- booth: Fine tuning eex-to-image diffusion models for subject-drive generation. In Proceedings the the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22500-203 22510, 2023.
[32] Javier Livvestre-Blanes Teresa Albero-Albero, Ignacio Miralles. Rubén Pérez-Llorens, and Jorge Moreno. A public fabric database for defect detection methods and results. Autex Revearch Journal, 19(4):363-774, 2019.
[33] Waqas Sultani, Chen Chen, and Mubarak Shah. Real-world anomaly detection in surveillance videos. In Proceedings the IEEE conference on computer vision and pattern recognition, pages 6479-6488 96899,66822228 2018.
[34] Domen Tabernik, Samo Sela, Jure Skvarě, and Danijel Skočaj. Segmentation--aased dee--eernng approach for surface-defect detection. Journal of Intelligent Manufacturing, 31(3):759-776, 2020.
[35] Tran Dinh Tien, Anh Tuan Nguyen, Nguyen Hoang Tran, Ta Duc Huy, Soan Duong, Chanh D Tr Nguyen, and Steven QH Truong. Revisiting reverse distillation for anomaly detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 24511 1-24220, 2023.
[36] Ze Wang, Yipin Zhou, Rui Wang, Tsung Yu Lin, Ashish Shah, and Ser Nam Lim. Few-shot fast- idst-adaetive anomaly detection. Advances in Neural Information Processing Systems, 35:4957-445 4970, 2022.
12

---

[37] Tiange Xiang, Yixiao Zhang, Yongyi Lu, Alan Yuille. Chaoyi Zhang, Weidong Cai, and Zongwei Zhou. Exploiting structural consistency of chest anatomy for unsupervised anomaly detection in radiography images. IEEE Trunsactions on Puttern Analysis and Machine Intelligence, 2024.
[38] Guoyang Xie, Jinbao Wang. Jiaqi Liu, Feng Zheng, and Yaochu Jin. Pushing the limits of fewshot anomaly detection industry vision: Graphcore. arXiv preprint arXiv:230.112082, 2023.
[39] Vitjan Zavrtanik, Matej Kristan, and Danijel Skočaj. Draem-a discriminatively trained reconstruction embedding for surface anomaly detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8330 8339, 2021.
[40] Vitjan Zavrtanik, Matej Kristan, and Danijel Skočaj. Reconstruction by inpainting for visual anomaly detection. Puttern Recognition, 112:107706, 2021.
[41] Yifan Zhang, Daquan Zhou, Bryan Hooi, Kai Wang, Jiashi Feng. Expanding small scale datasets with guided imagination. Advances in Neural Information Processing Systems, 36, 2024.
[42] Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Conditional prompt learning vision- language models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16816-168255 16825, 2022.
[43] Kang Zhou, Yuting Xiao, Jianlong Yang, Jun Cheng, Wen Liu, Weixin Luo, Zaiwang Gu, Jiang Liu, and Shenghua Gao. Encoding structure-texture relation with p-net for anomaly detection in retinal images. In Computer Ession-ECCy 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XX pages 360-377, Springer, 2020.
[44] Qihang Zhou, Guansong Pang. Yu Tian, Shibo He. and Jiming Chen. Anomalyclip: Object-agnostic prompt learning for zero-shot anomaly detection. arXiv preprint arXiv:2310.189),, 2023.
[45] Jiawen Zhu and Guansong Pang. Toward generalist anomaly detection via in-context residual learning with few-shot sample prompts, arXiv preprint arXiv:2403.0649 2024.
[46] Yang Zou, Jongheon Jeong. Latha Pemula, Dongqing Zhang, and Onkar Dabeer. ggot-the-difference self-supervised pre-training for anomaly detection and segmentation. European Conference on Computer Vision, pages 392-408. Springer, 2022.
13

---

A Appendix supplemental material
A.1 Text Prompt Setting
The text prompts are provided in Figure
(a) State-level (-mormall
(b) Physical-level
"[o] without flaw"
"a photo of a/the small [cl."
"a bad photo of althe
= "[o] without defect"
[c]."
"[o] without dam-
"a photo of a/the large [e]."
"a good photo of a/the [c]."
age"
"a bright photo of a/the [c]."
"a cropped photo of a/the [c]."
"a dark photo of a/thc [c]."
close-up photo of a/the [c]."
"a blurry photo of a/the [c]."
"a low resolution photo a/the [c]."
Figure 5: Lists of state and template level prompts employed in this paper to construct text features.
Our experimental approach initially focused on the number prompts, investigating scenarios with 3, 5, and 10 prompts. We found that performance at its lowest with single prompt, whereas using all ten prompts resulted the highest performance.
A.2 Abnormal Localization Results
Original Image
Ground
Truth
Result
Figure 6: The anomaly map localization results of our proposed method for subclass chewinggum. The first row shows the original images with anomalies, the second row displays the ground truth of the anomalies, and the third row shows the localization results obtained by our method.
Original Image
Ground Truth
Result
Figure 7: The anomaly map localization results of our proposed method for subclass fryum. The first row shows the original images with anomalies. the second row displays the ground truth of the anomalies, and the third row shows the localization results obtained by method.
14

---

Original Image
Ground
Truth
Result
Figure 8: The anomaly map localization results of our proposed method for subclass candle. The first shows the original images with anomalies, the second displays the ground truth of the anomalies, and the third row shows the localization results obtained by our method.
Original Image
THE
TF
FE
Ground
Truth
Result
Figure 9: The anomaly map localization results of our proposed method for subclass peb2. The first row shows the original images with anomalies, the second displays the ground truth of the anomalies, and the third row shows the localization results obtained by our method.
Original Image
Ground Truth
Result
Figure 10: The anomaly map localization results of our proposed method for subclass grid. The first row shows the original images with anomalies, the second displays the ground truth of the anomalies, and the third row shows the localization results obtained by our method.
Original Image
Ground Truth
Result
The anomaly map localization results of our proposed method for subclass hazelnut. The first row shows the original images with anomalies, the second displays the ground truth of the anomalies, and the third row shows the localization results obtained by our method.
Figure 11:
15

---

Original Image
Ground
Truth
Result
Figure 12: The anomaly map localization results of our proposed method for subclass metal nut. The first row shows the original images with anomalies, the second displays the ground truth of the anomalies, and the third row shows the localization results obtained by our method.
Original Image
Ground Truth
Result
Figure 13: The anomaly map localization results of our proposed method for subelass tile. The first row shows the original images with anomalies, the second row displays the ground truth of the anomalies, and the third row shows the localization results obtained by our method.
Original Image
Result
Figure 14: The anomaly map localization results of our proposed method for subclass BrainCT, The first row shows the original images with anomalies, the second displays the ground truth of the anomalies, and the third row shows the localization results obtained by our method.
Original Image
Result
Figure 15: The anomaly map localization results of our proposed method for subclass HeadCT, The first row shows the original images with anomalies, the second row displays the ground truth of the anomalies, and the third row shows the localization results obtained by our method.
16

---

Original Image
Result
Figure 16: The anomaly map localization results of our proposed method for subclass OCT. The first row shows the original images with anomalies, the second row displays the ground truth of the anomalies, and the third row shows the localization results obtained by our method.
17

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Yes, the main claims presented in our abstract and introduction accurately reflect the contribution and scope of our paper
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: Yes, our paper discusses the limitations of the work in the discussion section.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
18

---

Justification: For each theoretical result, we have presented the assumptions and provided comprehensive experiments.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification:
We fully disclose all the information.
Guidelines:
The answer NA means that the paper does include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
19

---

Answer: [Yes]
Justification: Yes, the data in our paper is publicly available, and we will also make the code public in the future.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to to reproduce the results. See the NeurlPS code and data submission guidelines (https
wttgggidddd/ooSSuuunnnnn ssionpolicy) for more details. hetpolliiiggiddee/ooooooo
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Yes, our paper provides fairly comprehensive training and test details in the experimental setup section.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: es, our main experiments all provide standard deviation information.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
20

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπхо rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: we provide sufficient information.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform. in every respect, with the NeurIPS Code of Ethics https://eearrppp..pppooo//hhhaca
Answer: [Yes]
Justification: Yes, the research conducted in the paper conforms, in every respect, with the
NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- cration due to laws or regulations their jurisdiction..
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Yes, we discuss both potential positive societal impacts and negative societal impacts of the work.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
21

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our experiments do not involve this aspect.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Yes, the assets used in the paper (e.g., code, data, models) are all properly credited.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the Datasets.papersstt paperswi thcode. cww/ datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
22

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
Our experiments do not involve this aspect.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
Our experiments do not involve this aspect.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
Our experiments do not involve this aspect.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
23


Reviews:
[{"review_id": "Xy9KmUJX37", "reviewer": "Reviewer_Ljby", "summary": "The paper focuses on a practical yet challenging anomaly detection in a few-shot-normal-image setting. Instead of directly matching features between the query image and a few normal reference images, the core insight is to replace the reference image with a personalized normal image generated by an anomaly-free custom model. The authors also propose a triplet contrastive anomaly inference strategy, incorporating the original/generated anomaly-free samples and text prompts. Extensive experiments are extensively conducted across 11 datasets.", "questions": "1. Which data augmentations are used to augment few-shot normal images? Do these data augmentations vary across different categories?\n2. It would be interesting to investigate how text prompts affect the anomaly-free customized model. For example, what if a specific category name (e.g., “cable”) is used to replace \"object\"?\n3. How are \\alpha and \\beta determined in the final prediction? How about the ratio of the t-step?\n4. It would be favorable to report F1-max, AP and PRO along with AUROC.\n5. What about the one-shot setting?", "limitations": "The limitations are adequately addressed.", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "1. The paper is well-writing and easy to follow.\n2. State-of-the-art results are achieved across 11 datasets.", "weaknesses": "1. It is essentially a memory-augmented reconstruction-based anomaly detection (AD) method [1], which attempts to reconstruct the query image to its most similar anomaly-free counterpart. However, the reconstruction-based AD method also explores the Stable Diffusion (SD) denoising network [1]. Could you clarify the differences? If I understand correctly, the core difference is the inputs to SD, which are pairs of object text prompts and few-shot normal images.\n2. Though Figure 4 demonstrates the effectiveness of the generated anomaly-free samples on three AD methods, how do these generated samples enhance the authors’ method? Can we use more or fewer generated normal samples instead of 100? An ablation study is required here.\n\n[1] Gong et al. Memorizing normality to detect anomaly: memory-augmented deep autoencoder for unsupervised anomaly detection. In ICCV, 2019.\n[2] He et al. Diad: A diffusion-based framework for multi-class anomaly detection. In AAAI, 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the careful reviews and constructive suggestions. We answer the questions as follows.\n***\n>W#1: It is essentially a memory-augmented reconstruction-based anomaly detection (AD) method [1], which attempts to reconstruct the query image to its most similar anomaly-free counterpart. However, the reconstruction-based AD method also explores the Stable Diffusion (SD) denoising network [1]. Could you clarify the differences? If I understand correctly, the core difference is the inputs to SD, which are pairs of object text prompts and few-shot normal images.\n\n**A#1:** Thanks for your constructive question. Indeed, our approach and memory-augmented reconstruction-based anomaly detection both attempt to reconstruct the query image to its most similar anomaly-free counterpart. There are some core differences: (1) Our reconstruction process does not utilize a memory bank. As the core differences you mentioned, It relies on Stable Diffusion (SD) using pairs of object text prompts and few-shot normal images. (2) Their memory-augmented reconstruction-based anomaly detection is entirely based on memory bank features for reconstruction, without any feature information from the original input image. In such a scenario, the most similar features found in the memory bank may still have low similarity because no information from the original input image is used for reconstruction, and the features stored in the memory bank may not fully match the input image, which may lead to increased reconstruction errors or noise. (3) Instead, We explore a stable diffusion process that allows control over the generative process and retains most of the original image information by converting anomalous regions of the query image to normal, thereby reconstructing the query image to its most similar anomaly-free counterpart. This method preserves most of the normal regions of the original image in the reconstructed image, reducing errors. (4) The reconstruction-based anomaly detection methods in [1] and [2] detect anomalies entirely by computing the difference between the original and reconstructed images, whereas our personalized image reconstruction is only one of the three branches used to compute the anomaly score.\n***\n***\n>W#2: Figure 4 demonstrates the effectiveness of the generated anomaly-free samples on three AD methods, how do these enhance the authors’ method? Can we use more or fewer generated normal samples instead of 100?\n\n**A#2:** Thank you for the detailed question. 1) The generated samples are involved in calculating the anomaly score, which is discussed in Triplet Contrastive Anomaly Inference section. These normal samples are generated to compare multi-level features with query images, contributing to one aspect of the anomaly score, denoted as $S_{N}$. Additionally, the ablation study in Table 5 of the manuscript confirms that these generated samples enhance the performance.  2) Yes, we attempted to validate the effectiveness of the proposed method by generating samples ranging from 10 to 300. For your convenience, we have placed the results in Figure 1 of **the uploaded PDF**. Increasing generated samples continues to enhance the results, likely because a larger sample set better represents the distribution of normal samples. However, for computational efficiency, we do not recommend generating too many samples (e.g., exceeding 200).\n***\n***\n>Q#1: Which data augmentations are used to augment few-shot normal images?Do these data augmentations vary across different categories?\n\n**A#3:** Thank you for the detailed question. We performed only simple data augmentation, such as flipping and Gaussian blur, on medical datasets as well as the KSDD dataset. This approach was adopted primarily to preserve the distribution of normal samples. Additionally, our method has a comprehensive text prompts design which, unlike data augmentation, can more effectively simulate various states of normal images.\n***\n***\n>Q#2: It would be interesting to investigate how text prompts affect the anomaly-free customized model. For example, what if a specific category name (e.g., “cable”) is used to replace \"object\"?\n\n**A#4:** Thanks for your good suggestion. In the experiments, we indeed used the specific category name, e.g., “cable”  to replace the variable “object”. This is to be consistent with recent related work using textual prompts. To avoid confusion, we will clarify this in our revised manuscript.\n***\n***\n>Q#3: How are alpha and beta determined in the final prediction? How about the ratio of the t-step?\n\n**A#5:** Thank you for the detailed question.1) We set the parameters $\\alpha$ and $\\beta$ for $A_{score}$ to 1 and 0.5, respectively, this configuration remains consistent across all datasets. We provide Table 1 in **the uploaded PDF**, demonstrating the robustness of our method to different choices of hyperparameters.\n2）The ratio for the t-step is set at 0.3. In experiments, we observed that performance fluctuations remain minimal within the range of 0.2 to 0.5. A ratio of 0.3 not only performs well across the majority of datasets but also offers enhanced computational efficiency due to its smaller value.\n***\n***\n>Q#4: It would be favorable to report F1-max, AP and PRO along with AUROC？\n\n**A#6:** Indeed, presenting metrics such as maximum F1-max,  AP, and probability of PRO, along with AUROC, would be the most favorable and comprehensive approach. However, given the breadth of our data—comprising 11 datasets across three domains—and the limited page space, it is not feasible to display all these indicators. Therefore, similar to previous methods like InCTRL and RegAD, we prioritize presenting AUROC.\n***\n***\n>Q#5: one-shot setting\n\n**A#7:** Thanks for your constructive suggestions. We have calculated the results of our method and compared them with other methods in a one-shot setting. For your convenience, these results are presented in Table 2 in **the uploaded PDF**.\n***", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yX3zPS15XV", "reviewer": "Reviewer_tpYi", "summary": "This paper addresses the issue of few-shot anomaly detection, which introduces an anomaly personalization method by using an anomaly-free customized generation model and performing a triplet contrastive anomaly inference strategy. Experiment evaluations across eleven datasets in three domains demonstrate its superior performance compared to the latest AD methods.", "questions": "1. What are the exact components of $C_0$ and $\\overline{C_0}$? As for the normal state prompts, is the number of $n$ below Line125 13? And what is the number of the memory bank $M$?\n\n2. The \"Triplet Contrastive Anomaly Inference\" seems to work as a weighted prediction from three comparison aspects during testing. Where can it reflect the concept of \"contrastive\"? What is the full training objective?\n\n3. Does $S_{text}$ reflect the degree of anomaly score? It seems that there are two types (both normal and abnormal) objects in the text prompts. How do they work in the same way as the equations of $S_{text}$ and $A_{score}$?\n\n4. Is there any limitation for the proposed method to handle the open-vocabulary scenarios? And what is the computation cost and inference cost of the proposed method compared with other methods since there are many images for generation?", "limitations": "The authors provide a brief discussion of the limitations on zero-shot anomaly detection.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is generally well-organized and well-written.\n\n2. The ideas of anomaly personalization and triplet contrastive anomaly inference are well-motivated with solid theoretical support.\n\n3. The superior performances on various evaluations demonstrate the effectiveness of the proposed method.", "weaknesses": "- There is a lack of discussion of the computation cost and inference speed of the proposed method since there are many generation steps. And the explicit description and exact number of prompts and images for generation are not clearly stated.\n\n- The method introduces several hyperparameters (e.g., α, β) which require careful tuning. It's unclear how sensitive the results are to these hyperparameters and whether the paper provides sufficient guidance on setting them. Regarding the process of multi-level feature comparison, it would be better to clarify the effect of the number of multi-feature extraction blocks.\n\n- The equations are not numbered and the line numbers are incomplete which makes it hard to reference.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the careful reviews and constructive suggestions. We answer the questions as follows.\n***\n\n>W#1: There is a lack of discussion of the computation cost and inference speed of the proposed method since there are many generation steps. And the explicit description and exact number of prompts and images for generation are not clearly stated.\n\n\n**A#1:** We thank the reviewer for the constructive question. In this work, we employed three prompts to generate three images (one for each prompt), and the inference time of our proposed method is slightly higher (+200-300ms per query image) than that of WinCLIP (389ms) and InCTRL (276ms). If necessary, we can further increase the inference speed by reducing the number of generated samples or decreasing the memory bank size. When using a single prompt corresponds to generating only one personalized image, the required inference time (326ms) is slightly lower than that of WinCLIP, while still demonstrating superior performance across three domains compared to other methods.\n\n\n***\n\n***\n\n>W#2: The method introduces several hyperparameters (e.g., α, β) which require careful tuning. It's unclear how sensitive the results are to these hyperparameters and whether the paper provides sufficient guidance on setting them. Regarding the process of multi-level feature comparison, it would be better to clarify the effect of the number of multi-feature extraction blocks\n\n**A#2:** Thank you for your detailed question. The hyperparameters α and β are set to 1 and 0.5, respectively, across all experiments and datasets. This setting was determined based on our preliminary experiments. For your convenience, we have presented these results in Table 1 of **the uploaded PDF**. As shown in the table, our method is quite robust to different values of these hyperparameters (e.g., α, β). We have updated the manuscript with the settings and specific values of these parameters. \n\n2) The use of multiple multi-feature extraction blocks has proven to be effective in enhancing performance [1]. The results in the following table also indicate that increasing the number of these blocks leads to further performance improvements. However, considering computational costs, we have decided to utilize four blocks in our final implementation to achieve an optimal balance between performance and efficiency.\n\n| Number |   2   |   3   |   4   |   5   |\n|:------:|:-----:|:-----:|:-----:|:-----:|\n| AUROC  | 95.7  | 96.0  | 96.2  | 96.4  |\n***\n\n***\n\n>W#3: The equations are not numbered and the line numbers are incomplete which makes it hard to reference.\n\n**A#3:** Thank you for your helpful suggestion. we will incorporate the modifications in the version.\n\n***\n\n***\n\n>Q#1: What are the exact components of C0 and C0? As for the normal state prompts, is the number of n below Line125 13? And what is the number of the memory bank ?\n\n\n**A#4:** 1)   We follow the notation used in the DreamBooth method to customize a diffusion model. ${C_0}$ consists of a set of text-image pairs, $ C_0 = \\{(x_k, c_k)\\} $, centered around the target object. The components of ${C_0}$ are reference images (i.e., few-shot normal samples) of the same object (e.g., cable, candle, brain, etc.) and their corresponding prompts. Additionally, $\\overline{C_o}$ contains different images (i.e., regularization images) within the same object for prior preservation and regularization purposes. The regularization images are generated using the Stable Diffusion (SD) model, as is common in most methods. These images use a prompt that is a coarse class descriptor to prevent language drift and reduce output diversity. 2) The number \\( n \\) in our method is 3 and the number of memory bank is 30.\n\n***\n***\n\n>Q#2: The \"Triplet Contrastive Anomaly Inference\" seems to work as a weighted prediction from three comparison aspects during testing. Where can it reflect the concept of \"contrastive\"? What is the full training objective?.\n\n**A#5:** Thank you for your detailed question on the \"Triplet Contrastive Anomaly Inference\" part. Yes, the \"Triplet Contrastive Anomaly Inference\" method we propose involves a weighted prediction from three comparative aspects during the testing phase. The term \"contrastive\" in our context refers to the dissimilarity comparisons among three branches: the query image in comparison with the personalized image, anomaly-free samples, and text prompts. This is designed not to interfere with the training process.\n\n***\n***\n\n>Q#3: Does reflect the degree of anomaly score? It seems that there are two types (both normal and abnormal) objects in the text prompts. How do they work in the same way as the equations of  and ?\n\n\n**A#6:** Indeed, in our method, text prompts contain two types of objects: normal and abnormal. During the computation of $S_{\\text{text}}$, the query image is compared with prompts of both types, yielding two probabilities: $p_{\\text{normal}}$ and $p_{\\text{abnormal}}$, which represent the likelihood of the image being normal and abnormal, respectively. Subsequently, the anomaly score $S_{\\text{text}}$ is calculated by summing $p_{\\text{abnormal}}$ and $1 - p_{\\text{normal}}$, which is then utilized for subsequent calculations of $A_{\\text{score}}$.\n\n\n***\n\n***\n\n>Q#4: Is there any limitation for the proposed method to handle the open-vocabulary scenarios? And what is the computation cost and inference cost of the proposed method compared with other methods since there are many images for generation?\n\n**A#7:** Thank you for your good suggestion. 1) Our method relies on learning a representative distribution of normal samples and therefore has to see few-shot normal examples.  We have not yet considered the direction of open-vocabulary scenarios. This could be a potential area for further exploration in future work. 2）Please refer to A#1. \n\n\n\n***\n\n[1] Toward Generalist Anomaly Detection via In-context Residual Learning with Few-shot Sample Prompts, CVPR 2024", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xQroxuktef", "reviewer": "Reviewer_oVNv", "summary": "This paper proposes a new few-shot anomaly detection method based on one-to-normal personalization of query images using a diffusion model and a triplet contrastive inference process. It leverages a diffusion-based generation model to transform a query image into a personalized image towards the distribution of normal images, for use in the triplet contrastive inference. Experimental results on datasets from industrial, medical, and semantic domains demonstrate that the proposed method outperforms existing models.", "questions": "- Can you provide visualizations or information about the distribution of the individual anomaly scores (S_n, S_p, S_text)?  It would be helpful to understand whether these scores focus on different parts or the same regions.\n- The sensitivity of the model to text prompts should be further explored. Specifically, I would like to know the number of selected text prompts when generating personalized image and the rationale or justification for setting physical-level text prompts in the chosen template.\n- In Table 4, using all the scores does not always correspond to the highest performance. A discussion on the possible reasons for this would be valuable. \n- Could you provide results from InCTRL in Figure 4, and also add lines or bars to compare each performance against the performance of the proposed model?\n- Why is the performance on MVTec-AD different in Table 5 (96.8) and Table 4 (96.2)?\n\nMinor issues:\n- In Table 4, the top value for AFID is 85.2, not 84.7 as bold-faced.\n- Tables 4 and 5 lack information on the few-shot setting.\n- Typo: \"semantice\" should be \"semantic\" in the caption of Table 3.", "limitations": "- As mentioned in the paper, the method is not applicable in zero-shot scenarios.\n- The theoretical exploration is lacking; there is only empirical evidence.\n- The computational cost is likely to be high, which may not be suitable for real-world scenarios\n- The approach depends on the performance of the generative model. And it is limited if the generated images are abnormal (potentially vulnerable to attacks).", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Unlike common augmentation-based methods that generate pseudo-anomalies for AD, this approach transforms a query image towards the normal distribution, which is interesting and effective. The proposed triplet contrastive inference is compelling.\n\n- Experiments conducted across various domains show its state-of-the-art performance in few-shot scenarios.", "weaknesses": "- Details about the experimental settings are not fully provided, which limits reproducibility. For instance, there is no information on the selection of hyperparameters such as alpha and beta for A_score in the implementation details. It raises concerns about the model's performance. If alpha and beta were optimized based on test data, the claim of state-of-the-art performance is questionable. Details about the memory bank M such as its size and sensitivity of the performance to its hyperparameters are also missing. \n- The paper lacks theoretical analysis, relying solely on empirical evidence.\n- Figure 1 does not provide sufficient information about the overall process. Particularly, it omits the training process of the diffusion model and the composition of the anomaly-free pool, which can cause confusion before reading the detailed explanations in the main text. \n- The definition of S_text in Sec 3.4 is unclear. It does not seem logical to apply the softmax function directly to the paired feature of F_q,l and F^T_text. \n- The lack of equation numbers in the Method section makes it difficult to follow. \n- There is no discussion of Table 3 in the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "***\n\n>W#1: Details about the experimental settings are not fully provided, which limits reproducibility. The hyperparameters such as alpha and beta for $A_score$ and Details about the memory bank M. \n\n**A#1:** Thank you for your constructive comments. 1) We set the parameters $\\alpha$ and $\\beta\\$ for $A_{\\text{score}}$ to 1 and 0.5, respectively. This configuration remains consistent across all datasets and is not optimized based on the test data of each dataset. To show the robustness of our method to different choices of hyperparameters, we present the detailed results in Table 1 of **the uploaded PDF**. We will also include more details in the revision.\n2) The memory bank size has been set to 30. Our preliminary experiments (Figure 1 in **the uploaded PDF**) indicate that larger memory bank values tend to improve results, but the improvement saturates after reaching a certain threshold. To balance model efficiency and performance, we selected \\(M = 30\\). For clarity, we have updated the manuscript to include the specific values of these parameters.\n\n\n\n\n***\n\n***\n\n>W#2: Figure 1 does not provide sufficient information about the overall process. Particularly, it omits the training process of the diffusion model and the composition of the anomaly-free pool, which can cause confusion before reading the detailed explanations in the main text.\n\n\n**A#2:** We apologize for any confusion caused. 1)The training process of the diffusion model follows the DreamBooth fine-tuning method to customize a diffusion model. We will provide more details in the revision.  2) Our anomaly-free sample pool comprises a set of normal reference images and generated normal images as mentioned in Section 3.4 of the manuscript. To avoid confusion, we will include a detailed explanation in the revision.\n\n\n***\n\n\n***\n\n>W#3: The definition of S_text in Sec 3.4 is unclear. It does not seem logical to apply the softmax function directly to the paired feature of F_q,l and F^T_text.\n\n\n\n**A#3:** Thank you for pointing this out and identifying the typo in our manuscript. $S_{text}$ determines the anomaly score by assessing the similarity between the text prompt and the query image. The image and text features are in fact combined using a dot product, and a softmax function calculates the probability, which serves as the anomaly score. Thank you for your correction; we have updated the manuscript accordingly. \n\n\n***\n\n***\n\n>Q#1: Can you provide visualizations or information about the distribution of the individual anomaly scores (S_n, S_p, S_text)? It would be helpful to understand whether these scores focus on different parts or the same regions.\n\n\n**A#4:** Thank you for the constructive questions. Following your suggestion, we have provided visualizations of the distribution of the individual anomaly scores. As illustrated in the figure of **the uploaded PDF**, combining both personalization and text prompts leads to the best performance, with each focusing on different regions.\n\n\n***\n***\n\n>Q#2: The sensitivity of the model to text prompts should be further explored. Specifically, I would like to know the number of selected text prompts when generating personalized image and the rationale or justification for setting physical-level text prompts in the chosen template.\n\n\n**A#5:** We thank you for your constructive suggestion. In this study, we utilized three text prompts, derived from our prior experiments. Our experimental approach initially focused on the number of prompts, investigating scenarios with 1, 3, 5, and 10 prompts. We found that performance was at its lowest with a single prompt, whereas using all ten prompts resulted in the highest performance. \n| Numbers of prompt | AUROC |\n|:-----------------:|:-----:|\n|         1         | 95.9  |\n|         3         | 96.2  |\n|         5         | 96.2  |\n|        10         | 96.4  |\n\nThe rationale for employing physical-level text prompts is predicated on the assumption that these prompts possess attributes directly related to the image, categorized into three main groups: global quality (e.g., 'a good photo of a/the [c]'), object size (e.g., 'a photo of a/the small [c]'), and image resolution (e.g., 'a low resolution photo of a/the [c]'). We assessed performance using prompts from one, two, or all three categories. Our results demonstrated that the prompts from all three categories yielded the best performance, leading us to adopt three prompts encompassing these categories for our analysis.\n\n \n***\n\n\n\n\n***\n\n>Q#3: In Table 4, using all the scores does not always correspond to the highest performance. A discussion on the possible reasons for this would be valuable.\n\n\n**A#6:** Yes, thank you for your valuable suggestion. Using all scores generally achieves the highest performance across most datasets, but it does not always correspond to the highest performance on a few datasets, for example, the KSDD dataset, a surface defect inspection dataset.  One possible explanation might be the high diversity in the distribution of normal images for the diffusion model to learn. In contrast, the BrainMRI medical dataset consists of grayscale images and features highly symmetrical normal samples. Such symmetry is not fully manifested in the personalized images, leading to better results when only using generated images and text.\n\n***\n\n***\n\n>Q#4: Could you provide results from InCTRL in Figure 4, and also add lines or bars to compare each performance against the performance of the proposed model?\n\n\n**A#7:** Thank you for your helpful suggestion. In Figure 2 of **the uploaded PDF**, we present results from InCTRL and include lines or bars to compare each performance against the performance of the proposed method.\n\n***\n***\n>Q#5: The lack of equation number, MVTec-AD performance typo, and other minor issues:\n\n\n**A#8:** We greatly appreciate your careful review to improve our manuscript. We will address all issues in the revision.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tQaLX1S1oH", "reviewer": "Reviewer_WVvk", "summary": "This paper provides a novel method to tackle the issue of precision loss in more complex domains. They introduce an anomaly personalization method with a diffusion model to utilize the diffusions to obtain the normal sample distribution and exchange the anomaly image into normal ones. Finally, a triplet contrastive strategy is designed to obtain the score. Their approach obtains a SOTA result compared to the recent methods.", "questions": "See Weakness.", "limitations": "The authors have address the limitations in zero-shot scenarios.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper provides novel approaches that utilize the diffusion model for both distribution modeling and recovering the anomaly images into the real one for comparison.\n2. The triplet contrastive strategy introduces a multi-level way to obtain the anomaly map from different views which is more robust than the previous methods.\n3. The experiments are comprehensive and the result is SOTA compared to the recent methods.", "weaknesses": "1. Will the utilize of the diffusion model bring a long inference time, which may not be effective for real-time application?\n2. Will the choice of the text influence the result? Is there any comparison result with different prompt choices?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the careful reviews and constructive suggestions. We answer the questions as follows.\n***\n\n>W#1: Will the utilize of the diffusion model bring a long inference time, which may not be effective for real-time application\n\n**A#1:** Thank you for the constructive question. 1) Our approach has a relatively short diffusion process and inference time compared to most diffusion-based anomaly detection methods, because our diffusion step ratio is only 0.3. 2) For very recent anomaly detection methods that do not utilize diffusion models (e.g., WinCLIP and InCTRL), the inference time of our proposed method is slightly higher (+200-300ms per query image) than that of WinCLIP (389ms) and InCTRL (276ms). 3) If necessary, we can further decrease the inference time by reducing the number of generated samples or decreasing the memory bank size. When using a single prompt corresponds to generating only one personalized image, the required inference time (326ms) is slightly lower than that of WinCLIP, while still demonstrating superior performance across three domains compared to other methods.\n\nTo improve the application in real-time scenarios, the following strategies may be considered: 1) Performing feature-level comparisons to reduce the steps involved in encoding and generating images. 2) Exploring the possibility of model pruning to lower computational complexity. 3) Employing more efficient diffusion model architectures (e.g., AT-EDM). Further exploration can be pursued in our future work.\n\n\n\n***\n\n***\n\n>W#2: Will the choice of the text influence the result? Is there any comparison result with different prompt choices?\n\n**A#2:** Thank you for your insightful question. Yes, the choice of text will influence the results. In our preliminary research, we conducted a comparison experiment. Our experiments explored both the category and quantity of prompts:\n\n1) Exploration by Category: We categorized text prompts based on physical-level attributes related to the image, such as global quality (e.g., \"a good photo of a/the [c]\"), object size (e.g., \"a photo of a/the small [c]\"), and image resolution (e.g., \"a low resolution photo of a/the [c]\"). We tested the performance using prompts from a single category, two categories, and all three categories. We found that using prompts from all three categories resulted in the best performance. For your convenience, we have provided some experimental results on the MVTec-AD dataset with three prompts as a reference below.\n\n| Datasets   | Global quality | Object size | Image resolution | AUROC |\n|------------|----------------|-------------|------------------|-------|\n|            |                | ✅           |                  | 95.8  |\n| **MVTec-AD** |                |             | ✅                | 95.9  |\n|            | ✅              |             |                  | 95.9  |\n|            |                | ✅           | ✅                | 96.0  |\n|            | ✅              | ✅           | ✅                | 96.2  |\n\n2）Exploration by quantity: We examined scenarios with 1, 3, 5, and 10 prompts. We observed that when there was only one prompt, the performance was the lowest, whereas using all ten prompts yielded the highest performance. Generally, we find there is no significant difference in performance between using three and five prompts.\n\n\n| Numbers of prompt | AUROC |\n|:-----------------:|:-----:|\n|         1         | 95.9  |\n|         3         | 96.2  |\n|         5         | 96.2  |\n|        10         | 96.4  |\n\nOur results suggest that the more comprehensive the inclusion of these three categories, the better the performance, with the optimal scenario using all 10 prompts. However, considering efficiency, we aimed to achieve the best results with the fewest prompts possible. We selected one prompt from each category and discovered through experiments that the combination of \"a good photo of a/the [c],\" \"a close-up photo of a/the [c],\" and \"a low resolution photo of a/the [c]\" yielded the best results in most cases. This combination of three prompts is what we present in the main figure describing our method in the manuscript.\n\n\n\n\n***", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We appreciate all reviewers for their careful reviews and constructive suggestions. In this rebuttal, Individual concerns have been carefully addressed in the response to each reviewer, with an uploaded PDF for more results suggested by reviewers. In the final version, we will revise the paper following these suggestions.

Meta-review: All reviewers are positive about this paper. They acknowledge the novelty of this paper and the achieved superior performance. They also have some questions about the ablation study and other analysis issues, which are well-addressed during the rebuttal. Therefore, the AC agrees with the positive opinion of the reviewers and recommends acceptance.
Decision: Accept (poster)