CLIP Meets Diffusion: A Synergistic Approach to Anomaly Detection
Authors: Byeongchan Lee
John Won
Seunghyun Lee
Jinwoo Shin
Venue: Accepted by TMLR ()

Abstract: Anomaly detection is a complex problem due to the ambiguity in defining anomalies, the diversity of anomaly types (e.g., local and global defect), and the scarcity of training data. As such, it necessitates a comprehensive model capable of capturing both low-level and high-level features, even with limited data. To address this, we propose CLIPFUSION, a method that leverages both discriminative and generative foundation models. Given the CLIP-based discriminative model's limited capacity to capture fine-grained local details, we incorporate a diffusion-based generative model to complement its features. This integration yields a synergistic solution for anomaly detection. To this end, we propose using diffusion models as feature extractors for anomaly detection, and introduce carefully designed strategies to extract informative cross-attention and feature maps. Experimental results on benchmark datasets (MVTec-AD, VisA) demonstrate that CLIPFUSION consistently outperforms baseline methods in both anomaly segmentation and classification under both zero-shot and few-shot settings. We believe that our method underscores the effectiveness of multi-modal and multi-model fusion in tackling the multifaceted challenges of anomaly detection, providing a scalable solution for real-world applications.

Published in Transactions on Machine Learning Research (10/2025)
CLIP Meets Diffusion: A Synergistic Approach to Anomaly Detection
Byeongchan Lee
prinsommer@kaist.cck
Korea Advanced Institute of Science and Technology (KAIST)
John Won
johmwon@kkaist.oc.k
Korea Advanced Institute of Science and Technology (KAIST)
Seunghyun Lee
shgun4839@karist.ocok
Korea Advanced Institute of Science and Technology (KAIST)
Jinwoo Shin
jinwoss@kaist.oco oc.
Korea Advanced Institute of Science and Technology (KAIST)
Reviewed on OpenReview: https: // openreview. net/ forun? id- WpFaZNuQng
Abstract
Anomaly detection complex problem due to the ambiguity in defining anomalies, the diversity of anomaly types (e.g., local and global defect), and the scarcity of training data. As such, it necessitates comprehensive model capable of capturing both low-level and high-level features, even with limited data. To address this, we propose CLIPFUSION, hethod that leverages both discriminative and generative foundation models. Given the CLIP- discriminative model's limited capacity to capture fine- fne-grained local details, we incorporate a diffusion- generative model to complement its fcatures. This integration yields synergistic solution for anomaly detection. To this end, we propose using diffu- sion models feature extractors for anomaly detection, and introduce carefully designed strategies to extract informative cross-attention and feature maps. Experimental results on benchmark datasets (MVTec-AD, VisA) demonstrate that CLIPFUSION consistently outperforms baseline methods in both anomaly segmentation and classification under both zero-shot and few-shot settings. We believe that our method underscores the effectiveness of multi-moodal and multi-model fusion in tackling the multifaceted challenges of anomaly detection, providing scalable solution for real-world applications.
1 Introduction
Anomaly detection is an important problem in real-world applications such as industry (Bergmann et al., 2019; Zou et al., 2022) and medicine (Setio et al., 2017; Menze et al., 2014). In this paper, address anomaly classification and segmentation. In anomaly classification (or segmentation), each image (or pixel) is classified as normal or anomalous.
In practice, the number of abnormal images small, while the potential types of abnormalities are highly diverse, making it challenging to obtain representative examples. Moreover, developing methods that can quickly adapt to environments (e.g., new processes or products), even with limited number of normal images, has become critical research topic (Rudolph et al., 2021; Sheynin et al., 2021; Zou et al., 2022; Huang el al., 2022). To address this, we focus on the problem within the zero-shot or lew-normal-shot? setup.
Itt is also referred anomaly detection and localization, respcctively, the litcrature. We use the terms anomaly classification and segmentation, and refer anomaly detection the overarching problem encompassing both. In this paper, "few-shot" specifically refers "tot

---

Published in Transactions on Machine Learning Research (10/2025)
Original Tmuage
Greund Truth Mask
WHCCLIP (1 aboa)
CLIPPESION dbat)
Cable
Carpet
Figure 1: Qualitative results of our model CLIPFUSION, It captures fino-grained anomalics more effectively than WinCLIP.
The recent advent of large-scale pretrained foundation models, e.g., GPT (Brown et al., 2020) and DALL-E (Ramesh et al., 2021; 2022), has opened new horizons for addressing such challenges. The generalizability and scalability of foundation models make them particularly effective for lackling zero-shot and few-shot tasks such as anomaly detection. Effectively utilizing off-the-shelf foundation models is an active research direction. Our design follows this trend without retraining or introducing shared modulc. Building on this, we present a novel methodology that combines the complementary strengths of discriminative and generative foundation models.
Anomaly detection is a multifaccted problem, and various approaches have been developed based on different perspectives. First, anomaly detection has the aspect of one-class classificatio.. This has led the community to mothods that learn normality from normal images and judge what deviates from it as anomalous. Here, many methods extract normal features from normal images using a vision foundation model (Cohen & Hoshen, 2020; Defard et al., 2021; Roth al., 2022). Second, anomaly detection has the aspect of two-class classification. The community has employed vision-language foundation models, such CLIP (Radford et al., 2021), to compare query image against normal and abnormal text prompts (Jeong al., 2023; Li et al., 2024a). Third, anomaly detection also has the aspect of semantic segmentation. Many anomalous features are fine-grained ones that appear in textures or edges. WinCLIP (window-based CLIP) attempted to solve this by sliding windows at multiple scales. Howevor, it is known that CLIP has limitations in learning fine-grained features (Wysoczáńska et al., 2025; Jiang et al., 2023), and such an approach might be suboptimal. Also, requires hundreds of forward passes due to the exhaustive sliding of multi-scale windows.
The multifaceted nature of anomaly detection makes it challenging to address using only discriminative models. So, we aim to create synergy by fusing discriminative vision-language model (VLM) with a generative text-to-image (T2I) model. These are two fundamentally distinct approaches to aligning images and text. Discriminative models, such as CLIP, directly align embeddings in the same space, while generative models, like diffusion models, align them indirectly by jointly performing a common task. Diffusion models align images and text through a generative process that prescrves fino-grained semantics. Consoquently, they may capture more detailed visual patterns. Therefore, it can be beneficial to leverage both high-level information extracted from the CLIP- model and low-level information extracted from the diffusion- based model for effective anomaly detection.
We propose CLIPFUSION (= CLIP + Diffusion), a novel method that combines the strengths of multi- modal (language and vision) and multi-model (CLIP and diffusion) approaches. In the zero-shot scenario, CLIPFUSION effectively identifics anomalies by integrating imago-text alignment from a CLIP-basod model and atossaattention maps (Chefer al., 2023; Zhang et al., 2023b; Liu et al., 2024) from a diffusion- model. In the few-shot scenario, it further enhances its performance by additionally leveraging feature
2

---

Published in Transactions on Machine Learning Research (10/2025)
maps from the CLIP image encoder and the diffusion denoiser, enabling more comprehensive comparison between the query image and reference images. Figure 1 illustrates qualitative results, showcasing the model's s ability to capture fine-grained anomalies. CLIPFUSION exemplifies a holistic framework that synergizes complementary foundation models, and has the potential to extend beyond anomaly detection and advance other recognition lasks.
Contributions of our work are summarized as follows:
We propose the use of diffusion models as feature extractors for anomaly detection. To this end, we exploit cross-attention and feature maps and carefully design extraction strategies. Since we extract intermediate features from an off-the-shelf diffusion model without any training or iterative denoising, our method is fast and efficient.
We introduce a novel approach to anomaly detection that jointly leverages features from CLIP and diffusion models. We demonstrate the complementary benefits of combining features from both models. Since diffusion models compensate for CLIP's limitations in capturing fine-grained local details, our method eliminates the need for time-consuming techniques such as multi-scale sliding windows.
In terms of performance, our method consistently outperforms baselines. Unlike most prior works, it is also versatile in scope: it can handle both segmentation and classification tasks under zero-shot and few-shot settings.
Our method requires additional training, which enables fast real-world deployment. Even in few-shot settings, the samples are used solely for feature extraction rather than training, making the method more robust. In contrast, training-based methods are sensitive to the choice of when to stop training, which is challenging in few-shot setups due to the absence of a validation set.
2 Related Work
Vision(-Language) Models for Anomaly Detection In works that utilize vision(-languus models in anomaly detection, features are extracted from the intermediate layers of model pretrained on a discrim- inative task (e-g., classification features of the query image with the features of reference images. Some works utilize ResNet-based models pretrained on ImageNet in the few-shot regime. In SPADE (Cohen & Hoshen, 2020), k-nearest neighbors algorithm is employed to identify the k reference images closest to the query image, which are then used as the basis for anomaly detection. In PaDiM (Defard et al., 2021), features extracted from reference images are modeled as Gaussian distributions, and the features of the query image are compared against these distributions. In PatchCore (Roth et al., 2022), memory bank is constructed using the features of reference images, and the features of query image are directly compared against this memory bank. WinCLIP (Jeong et al., 2023) applies local windows and combines the results through harmonic aggregation. PromptAD (Li et al., 2024a) learns prompt tokens through contrastive training.
contrastive learning)- A Anomaly detection is performed by comparing the
A separate line of work assumes access to additional data and leverages it. Some methods require an auxiliary datasot containing both normal and anomaly images, c.g., training on the MVTec-AD datasct and inference on the VisA dataset (Hu et al., 2024; Zhu & Pang, 2024; Yao el al., 2024; Li et al., 2024c; Cao et al., 2024; Zhou et al., 2023; Gu et al., 2024a; Qu et al., 2024). AnomalyGPT (Gu al., 2024b) relies on an auxiliary dataset with a large number of normal samples. In practice, acquiring such data with matching characteristics is challenging, and these methods often struggle when deployed on datasets with distribution shifts. Moreover, the need for additional training increases deployment time. Some methods also rely on ground-truth segmentation masks, which are difficult to obtain in real-world scenarios (Hu et al., 2024; Cao et al., 2024; Zhou et al., 2023). MuSe (Li et al., 2024b) leverages the test set by comparing the features of test images, and is only feasible for batch-processing of the test set.
Diffusion Models for Discriminative Tasks Recently, there has been growing interest in applying diffusion models to discriminative tasks (Fuest et al., 2024). In DDPM-Seg (Baranchuk et al., 2021), feature
3

---

Published in Transactions on Machine Learning Research (10/2025)
maps constructed using activations extracted from intermediate layers of the denoiser and are utilized for semantic segmentation. VPD (Zhao et. al., 2023) further incorporates cross-altention maps for semantic segmentation and depth estimation. These methods require fine-tuning for downstream tasks. On the other hand, DifsusionClassiferr (Li et al., 2023a) performs zero-shot classification by determining which text conditioning best predicts the noise. P2PCAC (Hertz et al., 2022) and USCSD (Hedlin et al., 2024) reveal that cross-attention maps contain rich information about spatial layouts through image editing and semantic correspondence tasks.
3 Preliminaries
CLIP CLIP (Radford et al., 2021) consists of an image encoder fr(x) and lext encoder fr(c). These encoders map images and texts into the same embedding space. The model is trained to match correct image- text pairs with high similarity while assigning low similarity to incorrect pairs. This process is optimized using the following contrastive loss (Chen et al., 2020):
exp(sim((1(x))
pT = - x) Eeec (r(()))(()))
where C is the set of candidate text prompts, T > 0 is the temperature hyperparameter, and sim(:,-) is the cosine similarity measure.
Diffusion A diffusion model (Ho et al., 2020) is trained through sequential process that involves forward diffusion and backward denoising. In the diffusion process, random noise is progressively added to a clean image over several steps, following the equation:
((=))=-)) /1- - 111111,,..0 1,
(2)
where 3r is the noise schedule hyperparameter, which determines the amount of noise added the image Øt-1 (Nichol & Dhariwal, 2021). In the denoising process, random noise is progressively removed over several steps using the following equation to restore the original image:
(1=)) = 17 (), Eo(x,, f)).
(3)
In this process, the denoiser eø (0t,, t) learns to predict the added noise from the noisy image 2; at a given time step (Song et al., 2020). During inference, random noise is sampled, and the denoiser iteratively removes it to generate a new image. By conditioning on a text prompt c, the denoiser to(xa,t,e) guides the generation process to align the resulting image with the provided text. Diffusion models demonstrate superior performance in understanding image details by generaling high-quality images compared to other generative models (Dhariwal & Nichol, 2021). They can also be utilized for impainting a specific region of an image. At cach step, the untouched region is preserved by replacing it with the corresponding part of the original image, ensuring that only the specified region is generated and seamlessly blended with the rest of the image (Rombach al., 2021).
4 Method
The final model, CLIPFUSION, is a fusion of our based and diffusion- models. For convenience, call the Casedbased model PatchCLIP (patch-based CLIP) and the diffusion- based model MapDiff (map- based diffusion). We first outline our feature extraction strategies from both models, tailored for anomaly detection.
PatchCLIP CLIP processes an image by dividing it into multiple patches and outputs both patch embed- dings and a class token embedding. While the class token embedding is aligned with text embeddings, patch embeddings can provide additional information, which we incorporate to enrich the spatial representation. Given that fine-grained details are handled by the diffusion- based model, PatchCLIP provides a simple yet effective spatial prior, thereby avoiding the cumbersome multi-scale window sliding in WInCI.IP.
4

---

Published in Transactions on Machine Learning Research (10/2025)
GLiP-based
Dfflution based
Aphets perfect hottel
CLIP loxt encoder
Panch-leet
Diffuso score map M/MMC
Gross- attention
score A aytLi
CLIP Image encodder
Feature map
Festurn map
Difazio Saatura untractor
Curry Image
CLPP witonn I
Eiftusoon vision soore map 1etire
Aphoaralitee parfest (ball
Tinesipp
CLIP encothe
Feature memory bank
Feature memory bant
teature extractor
Rel image
image
Figure 2: Overall framework for CLIPFUSION. The CLIP-based model and the diffusion-based model process query and reference images to generate anomaly maps. The final anomaly map obtained by fusing the outputs of the vision and language components of both models.
Additionally, utilize features extracted from CLIP's image encoder. To achieve this, make use of the VV-ateentio mechanism proposed in CLIP surgery (Li et al., 2023b; 2024a). This mechanism was introduced to complement CLIP's QK-attention mechanism, which is specialized in global feature extraction. It facilitates local feature extraction by enabling direct interactions between values (hence the name VV- attention) that carry information about the patches themselves.
MapDiff We extract and utilize cross-attention maps and feature maps from the diffusion denoiser. To impainting pipeline that takes clean image input, rather than generation
achieve this, we pipeline that starts with random noise. In the inpainting pipeline, mask is provided to specify the region where noise will be added. Here, we use the trick of applying an empty mask. This prevents noise from being added, but the denoiser still perceives the original image as noisy, thereby allowing us to control the timestep and extract features at the desired level of detail.
By passing the clean image through the denoiser once, we extract the maps from its intermediate layers. A large timestep makes the denoiser interpret the image highly noisy, capturing high-level pattern informa- tion, while small timestep emphasizes finer details.
4.1 Anomaly Segmentation
The final anomaly score map M H for segmentation obtained by fusing the score maps from Patch- CLIP and MapDiff. In the zero-shot setup, only the language-guided score maps MLIIP,L and Difff,L are used. In the few-shot setup, the vision-guided score maps V and M are additionally incorporated. The final anomaly score map is computed follows:
M: = CLI + CLI.P. + (1 aM + V
(4)
where a is the weight between the models. In the zero-shot setup, M = CLILL L + (1 - Ditt,
4.1.1 Zero-shot
In the zero-shot setup, given a query image, we leverage the relationships between images and texts learned multi-modal models to create anomaly score maps.
5

---

Published in Transactions on Machine Learning Research (10/2025)
CLIP Language-guided Score Map MCLIP, The query image is input to the CLIP image encoder (e.g.. ViT), which divides it into patches, processes them, and outputs patch embeddings. On the other hand, normal and abnormal text prompts are passed to the CLIP text encoder, producing normal and abnormal text emboddings. Then, the degree (Equation 1) to which each patch embodding is aligned to the abnormal text embedding is calculated for all patches and organized into a map. This is upsampled to the original image size through interpolation to obtain MCLIP,
Diff Language-guided Score Map Moire, The query image is passed to the diffusion inpainting pipeline along with an empty mask, a timestep, and a text prompt. The text prompt includes an object word and a state word, for example:
photo a [object] with [state].'
where [object] refers to the entity (e.g, "bottle") in the image, and [state] describes a possible abnormal condition (c.g. "crack") of it. The [state] token interacts with the image as passes through the denoiscr (e.g., U-Net), generating cross-allention map. The cross-attention map of diffusion model is a healmap showing where and how strongly text token is attended to across image locations. Because that denotes anomalous state, the attention weight at location serves as proxy for anomaly likelihood. That is, a higher value at a specific location in the map indicates that the location responds more strongly to the [state] token (i.e., the abnormal condition). Therefore, we can use this cross-attention map as an anomaly score map (Figure 3). Additionally, we calculate the average of cross-attention maps derived using various possible abnormal conditions (e.g.. "crack", "hole", "residue", "damage") to obtain Mostt.
token
This be interpreted as follows. In anomaly segmentation, given image I Ir we are interested in the probability that the (h, w)-th pixel is abnormal (A 1). This probability can be expressed as follows by conditioning on lext prompt T
= p(A = 1 = x) = Σp(AAn P(A = 1|
c)p(T c)2
(5)
oEC
where C represents the set of text prompts that describe abnormal conditions. Assuming that the text prompt T is uniform and interpreting p(An, = 1/1 = = = c) as the aross-attention map intensity at the corresponding location:
Σ
p( p(An,w 111 = =).
(6)
C
GEC
4.1.2 Few-shot
In the few-shot sctup, normal reference images are utilized. For the k-shot case, k images are used to construct feature memory banks, which are then compared with the features of the query image. In general, we extract normal features from normal reference images by passing them through the CLIP image encoder and the diffusion denoiser. Then, the query image is passed through and the extracted features are compared with the normal fcatures. The degree of deviation used as an anomaly score.
CLIP Vision-guided Score Map V The reference images are passed through the CLIP's image encoder extract reference features and stored in the memory bank. Features with low-level semantic information extracted from middle encoder block, while those with high-level semantic information come from later encoder block.
The set of reference features extracted from the encoder block b is denoted as R². For a given encoder block b, the feature map extracted from the query image is denoted as Fb E Hx Then, CLIP, is caleulated by comparing the feature F with the reference features E R², follows:
CLIP.V haw
Σ
min rER 2 (1 sim(Fh, r))
(7)
B
6

---

Published in Transactions on Machine Learning Research (10/2025)
rivin (A00))
Tert Prompt
Q
KV
Imaage
Crack
Hooe
RediQue
Damaga
Figure Examples of cross-attention maps extracted from diffusion denoiser.
where B the set of blocks That is, a score is computed based on the similarity, and the smallest among all reference features in Rè is conservatively used as the anomaly score for b. The final anomaly score the average over B.
Diff Vision-guided Score Map V Reference images are passed through the diffusion denoiser to extract features, which are stored the memory bank. The denoiser consists three parts: an encoder (down-blocks),, bottleneck (mid-block), and a decoder (up-blocks). Information in the encoder passed the decoder via skip connections. The bottleneck the most compressed stage, where much of the detailed information lost, making it unsuitable for feature extraction. Therefore, we extract features from the decoder. Specifically, focus the middle and later decoder blocks, as they provide the most semantically rich features (Baranchuk et al., 2021; Zhao et al., 2023).
When timestep is given, the set of reference features extracted from the decoder block b is denoted as R The feature map extracted from the query image at timestep t and decoder block b is denoted as pt,b H Then, M is calculated as follows:
M h,w
Σ
min (1 - sim ( r))
(8)
PERLE 2
(t.)jT
where T is the set of combinations 6).
4.2 Anomaly Classification
The final anomaly score S E R for classification is obtained by fusing four scores:
S a(SCLIP,L SOLIP.VV + (1 aa) + 5oitt.V).
(9)
In the zero-shot setup, only the languago-guided scores are used. Specifically, the CLIP language-guided score LCLIP,L defined as the degree (Equation 1) of alignment between the query image's class token embedding and the abnormal text embedding. The diffusion language-guided score goifi.L aggregates the diffusion language-guided score map Diff, and is defined as follows:
SDift.L
Mediann(MMitt..) ////mm/Xxxxi)))
(10)
This means that the score becomes larger when the degree of focus of the cross-attention map is high, that is, when the maximum is significantly greater than the median.

---

Published in Transactions on Machine Learning Research (10/2025)
In the few-shot setup, additionally aggregate the vision-guided score maps, defining the scores as follows in accordance with common practice (Jeong et al., 2023):
SCLIP.V = max(M)ar.V)) SoiEE, = max M
(11)
5 Comparisons
Datasets We use the MV (Bergmann et al., 2019) and VisA (Zou et al., 2022) datasets, which are mainly used as benchmark datasets in anomaly detection. Each dataset consists of several object categorics, and each object calegory is divided into a training set containing only normal images and a lest set containing mixture of normal and abnormal images. Query images are from the test set, and reference images are sampled from the training set.
Baseline methods We compare against methods with the same setup, that is, methods that use only the provided zero-shot or few-shot normal images and do not rely on any additional supervision. This sctup reflects a realistic and challenging scenario commonly encountered in practice.
Specifically, we compare our method, CLIPFUSION, with methods utilizing vision models (PaDiM, Patch- Core) and those utilizing vision-language models (CLIP-AC, Trans-MM, MaskCLIP, WinCLIP, PromptAD*). Note that, among these methods, only CLIPFUSION and WinCLIP are fully capable of handling both seg- mentation and classification tasks in both zero-shot and few-shot scenarios.
Experimental details For PatchCLIIP employ OpenCLIP (Ilaarco et al., 2021; Cherti et al., 2023; Schuhmann et al., 2022), which has been pretrained on LAION-400M (Schuhmann et al., 2021), using a Vision Transformer (ViT) (Dosovitskiy, 2020) as its image encoder. For MapDiff, we utilize the pretrained Stable Diffusion v2 model (Rombach et al., 2022), where the denoiser is based on U-Net (Ronneberger et al., 2015).
We set 0.25 in Equation for segmentation and a 0.75 in Equation 9 for classification to reflect the different importance of the models in each task. We report the average performance and standard deviations over five runs with different seeds. Further details are in Appendix A.2.
5.1 Anomaly Segmentation
Table presents comparison of our method's performance with baseline methods for anomaly segmenta- tion. The results show that CLIPFUSION consistently outperforms the bascline methods across all sctups, datasets, and metrics. Our method, in particular, demonstrates dramatic performance improvements in the zero-shot setup. This highlights the critical contribution of the diffusion-based model's cross-attention map to zero-shot anomaly segmentation. Additionally, the significant performance improvements the few-shot setup demonstrate how the diffusion-based model complements the CLIP- model.
5.2 Anomaly Classification
Table shows the performance comparison between our method and baseline methods for anomaly classifi- cation. CLIPPFSION consistently outperforms all baseline methods. This indicates that the diffusion-basod model enhances performance not only in the segmentation task but also in the classification task by rein- forcing local information.
In both segmentation and classification tasks, the consistent performance improvements as the number of shots increase highlight the scalability of our method with minimal additional data. Furthermos the smaller standard deviations observed in our results indicate the robustness of our approach. This robustness can attributed to the integration of multiple models, which mitigates the variability.
Thhe authors of PromptAD the test set validation during training record the best performance for each category separately and report the aggregated results. For fair comparison, treat the training epoch fixed hyperparameter across categories and report the best best-performing checkpoint reproduced using the official codebase. See Appendix A.5 for more details.
8

---

Published in Transactions on Machine Learning Research (10/2025)
Table 1: Quantitative results for anomaly segmentation.
Setup
Method
MVTec-AD
VisA
AUROC
AUPRO
AUROC
AUPRO
Trans-MM (Chefer ct al., 2021)
57.5+0.0
21.9+0.0
49.4+0.0
10.2+0.0
MaskCLIP (Zhou et al., 2022)
63.7.1..
40.540.0
60.9
27.310.0
0-shot
WinCLIP (Jeong al.; 2023)
85.120.0
64.620.0
79.6
56.820.0
CLIPFUSION (ours)
90.9+0.0
87.1+0.0
92.1=0.0
83.3+0.0
PaDiM (Delard al., 2021)
89.3
73.312.0
89.9
64.3442.4
PatchCore (Roth et al., 2022)
92.0#1.0
79.7=2.0
95,4+0.6
80.5
1-shot
WinCLIP (Jcong et al., 2023)
95.2+00.5
87.1+1.2
96.4+0.4
85.1+2.1
PromptAD (Li et al., 2024a)
94.910.88
87.7+:1.5
95.7+00.9
84.0+1.2
CLIPFUSION (ours)
95.810.1
90.4+0.11
97.3=1011
87.0=0.3
PaDiM (Defard et al., 2021)
91.3+0.7
78.2+1.8
92.0+0.77
70.1+2.6
PatchCore (Roth al., 2022)
93.310.6
82.311.3
96.1
82.6+1..3
2-shot
WinCLIP (Jeong al., 2023)
96.0+0.3
88.420.9
96.8
86.2+14
PromptAD (Li et al., 2024a)
95.0+0.7
88.3+1.0
96.3+0.9
84.8+00.8
CLIPFUSION (ours)
96.3+0.11
91.4+0.2
97.7=0.1
87.3+0.2
PaDiM (Defard et al., 2021)
92.620.7
81.311.9
93.240.5
72.6#1.9
PatchCore (Roth et al., 2022)
94.3++0.5
84.3+1.6
96.8+0.3
84.9+1.4
4-shot
WinCLIP (Jeong et al., 2023)
96.210.33
89.0
97.240.2
87.6+10.9
PromptAD (Li et al., 2024a)
95.5-+0.6
89.0+0.66
96.34
85.1+0.99
CLIPFUSION (ours)
96.8110.1
92.1:0.2
98.0.1.1
87.7
Table 2: Quantitative results for anomaly classification.
Setup
Method
MVTec-AD
VisA
AUROC
AUPR
AUROC
AUPR
CLIP-AC (Radford et al., 2021)
74.0++.00
89.1+0.00
59.3+0.0
67.0+..00
0-shot
WinCLIP (Jeong et al., 2023)
91.8+0.0
96.5+0..0
78.120.0
81.240.0
CLIPFUSION (ours)
93.6=0.0
97.020.0
79.5+0.0
83.6=0.0
PaDiM (Defard et al., 2021)
76.643.11
88.1+1.7
62.8+5.4
68.3+4.0
PatchCore (Roth et al., 2022)
83.443.0
92.211.5
79.912.9
82.8+2.3
1-shot
WinCLIP (Jeong et al., 2023)
93.142.0
96.5+1.99
83.8=4.0
85.144.0
PromptAD (Ti et al., 2024a)
92.6+1.0
95.7+0.9
83.1+2.1
85.7+1.2
CLIPFUSION (ours)
95.4-0.8
97.810.4
86.540.1
88.4-0.2
PaDiM (Defard et al., 2021)
78.9+3.1
89.3+1.7
67.4-5.1
71.6+3.8
PatchCore (Roth et al., 2022)
86.343.3
93.8+1.7
81.6+4.0
84.8+3.2
2-shot
WinCLIP (Jeong et al., 2023)
94.4#1..
97.0.10.7
84.612.4
85.8+2.7
PromptAD (Li et al., 2024a)
93.2+1.0
96.1#1.0
85.0=1.8
87.2+1.8
CLIPFUSION (ours)
96.3=0.3
98.210.2
87.110.5
88.8=0.5
PaDiM (Defard et al., 2021)
80.4+2.5
90.5+1..6
72.8+2.9
75.642.2
PatchCore (Roth et al., 2022)
88.8+2.6
94.5+1.5
85.3=2.1
87.5+2.1
4-shot
WinCLIP (Jeong et al., 2023)
95.2+11.3
97.3+0.6
87.3+1.8
88.8+1.8
PromptAD (Li et al., 2024a)
93.8++0.7
96.5++0.7
86.2+1.6
88.1+1.9
CLIPFUSION (ours)
96.9-0.2
98.6.10.1
88.110.2
89.7-0.1
6
Empirical Study
In this section, results reported under the one-shot setup.

---

Published in Transactions on Machine Learning Research (10/2025)
Table 3: Comparison of different models.
Table 4: Comparison of different modalities.
Task
Model
Dataset
Task
Modality
Datasel
CLIP
Diff
MVTec-AD
VisA
Language
Vision
MVTec-AD
VisA
94.7
93.7
x
90.9
92.1
Seg.
95.1
96.5
Seg.
95.4
96.1
95.8
97.3
/
95.8
97.3
93.5
82.2
93.2
79.5
Cls.
87.2
78.9
Cls.
V
90.5
82.8
V
95.4
86.5
V
95.4
86.5
Table 5: Timestep and block configurations.
Table 6: Decoder blocks for feature maps.
Task
Configuration
MVTec-AD
VisA
Task
Early Block
MVTec-AD
VisA
Step 1, Block f
94.3
95.1
94.9
93.7
Seg.
Seg.
Step 1, Block +
95.1
96.5
95.1
96.5
Step 7, Block
86.3
76.2
86.5
74.9
Cls.
Cls.
Step 1, Block 4
87.2
78.9
W
87.2
78.9
6.1 Ablation Study
Table presents the comparative performance of different models. As expected, the diffusion-based model, which captures local features effectively, performs better in the segmentation task, while the CT.PP-based model, which captures global features effectively, performs better in the classification task. The combination of different models consistently yields better results compared to using either model alone. This empirical evidence suggests that the two models capture complementary features, making their integration beneficial for both segmentation and classification tasks.
Table 4 presents the performance of different modalities, highlighting their complementary strengths through the combination of semantic and spatial information.
6.2 Extraction Strategies from Diffusion Denoisers
We explore strategies to effectively extract features from diffusion denoisers. Here, we conduct experiments using only MapDiff to see the effect more clcarly.
Table highlights the impact matching timestep and block numbers when calculating V (Equation 8). Note that the decoder consists of four blocks (Block Block 3). Matching higher timesteps with lower blocks yiclds better performance, and vice versa (Timestep 201 - Block 3, Timestep 401 - Block 2, Timestep 801 + Block 1). This patlern reflects the intrinsic behavior of the diffusion denoiser: at higher timesteps, the denoiser interprets the input image as highly noisy and prioritizes the extraction of global features. The lower blocks, which process coarso-grained information, are particularly well-suited for this task. Using higher timesteps can be seen as a regularization strategy, as they suppress local details and highlight broader patterns.
Table 6 shows that when calculating V (Equation 8), including the early block (Block 0) does enhance the model's performance. On the contrary, excluding this block consistently yields better results. This suggests that the early stages of the decoder act as preparatory steps for the later stages, without forming concrete features.
6.3 Inference speed
Our method can be viewed an extension of WinCLIP (Jeong et al., 2023). We report the inference latency measured on an NVIDIA A100 GPU (40 GB) in Table 7, with batch size of 1. As shown, our method is approximately 6-8 times faster than WInCI.IP
10

---

Published in Transactions on Machine Learning Research (10/2025)
Table 7: Comparison of latency (ms).
Latency [ms]
0-shot
1-shot
2-shot
4-shot
WinCLIP
4782
9514
9633
9710
CLIPFUSION
783
1240
1269
1278
To capture fine-grained details, WinCLIP slides multi-scale windows over the query image, resulting in hundreds of forward passes. In contrast, our CLIP-based model utilizes patch embeddings obtained from single forward pass. On top of this, we employ a diffusion-based model not denoiser for iterative generation, but as a feature extractor to capture fine-grained details. As a result, our method requires only a few forward passes in total.
6.4 One-for-all paradigm
IPPA (Lv et al.) recently proposes new setting for few-shot anomaly detection, termed the one-for-all paradigm. It aims to train single unified model that can detect anomalies across multiple object categories. In this paradigm, the one-shot setting refers to using N normal images (one for each category), where N is the number of categories. Apart from this, we follow the same experimental setting as described in Section 5. Table presents a comparison with IIPAD (the SOTA method under this paradigm) on the MVTec-AD dataset in the one-shot setting, where our method achieves higher performance. See Appendix A.4 for more results.
Table 8: One-for-all paradigm results.
Method
AUROC
AUPR
IIPAD
94.2
97.2
CLIPFUSION
94.5
97.3
7 Conclusion
We propose a methodology that synergistically combines discriminative and generative foundation models, leveraging their complementary strengths. As a concrete application, we introduce CLIPFUSION, which integrates CLIP and diffusion models for anomaly detection and consistently outperforms baselines in seg- mentation and classification across zero-shot and few-shot scenarios. This holistic use of features from discriminative and generative models may benefit the research community in tackling complex recognition tasks.
Acknowledgments
This work was conducted by Center for Applied Research in Artificial Intellgennce(AARA)] grant funded by Defense Acquisition Program AdministrationnDAAAA) and Agency for Defense Development(ADD) (UD230017TD).
References
Dmitry Baranchuk, Ivan Rubachev, Andrey Voynov, Valentin Khrulkov, and Artem Babenko. Label-efficient semantic segmentation with diffusion models. arXio preprint arXiv:2112.00 03126, 2021.
Paul Bergmann, Michael Fauser, David Sattlegger, and Carsten Steger. Mytoc ad comprehensive real-world dataset for unsupervised anomaly detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9592 9600, 2019.
11

---

Published in Transactions on Machine Learning Research (10/2025)
Paul Bergmann, Michael Fauser, David Sattlegger, and Carsten Steger. Uninformed students: Student- teacher anomaly detection with discriminative latent embeddings. In Proceedings of the IEEE/CVE con- ference on computer vision and pattern recognition, pp. 4183 4192, 2020.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neclakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877 1901, 2020.
Yunkang Cao, Jiangning Zhang, Luca Frittoli, Yuqi Cheng, Weiming Shen, and Giacomo Boracchi. Adaclip: Adapting clip with hybrid learnable prompls for zero-shol anomaly detection. In European Conference Computer Vision, pp. 55 72. Springer, 2024.
Hila Chefer, Shir Gur, and Lior Wolf. Generic attention-moder explainability for interpreting bi-modal and encoder-decoder transformers. In Proceedings of the TEEE/CVE International Conference on Computer Vision, pp. 397 406, 2021.
Hila Chefer, Yuval Alaluf, Yael Vinker, Lior Wolf, and Daniel Cohen-Or. Attend-and-excite: Attention-based semantic guidance for text-to-image diffusion models. ACM Transactions on Graphics (TOG), 42(4):1-10,
2023.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pp. 1607. PMLRR, 2020.
Mehdi Cherti, Romain Beaumont, Ross Wightman, Mitchell Wortsman, Gabriel Ilharco, Cade Gordon, Christoph Schuhmann, Ludwig Schmidt, and Jenia Jitsev. Reproducible scaling laws for contrastive language-image learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2818 2829, 2023.
Niv Cohen and Yedid Hoshen. Sub-image anomaly detection with deep pyramid correspondences. arXiv preprint Sttt::2005.02357 2020.
Thomas Defard, Aleksandr Setkov, Angelique Loesch, and Romaric Audigier. Padim: patch distribution modeling framework for anomaly detection and localization. In International Conference on Pattern Recognition, pp. 475 489. Springer, 2021.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255. Icce, 2009.
Prafulla Dhariwal and Alexander Nichol. Diffusson models beat gans on image synthesis. Advances in neural information processing systems, 34:8780-8794, 2021.
Alexcy Dosovitskiy. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXtt:2010.11929, 2020.
Michael Fuest, Pingchuan Ma, Ming Gui, Johannes S Fischer, Vincent Tao Hu, and Bjorn Ommer. Diffusion models and representation learning: A survey. arXie preprinl or 007.0083,, 2024.
Zhaopeng Gu, Bingke Zhu, Guíbo Zhu, Yingying Chen, Hao Li, Ming Tang, and Jinqiao Wang. Filo: Zero- shot anomaly detection by fine-grained description and high-quality localization. In Proceedings of the 32nd ACM International Conference on Multimedia, pp. 2041-2045 2024a.
Zhaopeng Gu, Bingke Zhu, Guibo Zhu, Yingying Chen, Ming Tang, and Jinqiao Wang. Anomalygpt: Detecting industrial anomalies using large vision-language models. In Proceedings of the AAAI conference on artificial intelligence, volume 38, pp. 1932 1940, 2024b.
Hewei Guo, Liping Ren, Jingjing Fu, Yuwang Wang, Zhizheng Zhang, Cuiling Lan, Haoqian Wang, and Xinwen Hou. Template-guided hicrarchical feature restoration for anomaly detection. In Proceedings of the TEEE/CVF International Conference on Computer Vision, pp. 6447-6455 2023.
12

---

Published in Transactions on Machine Learning Research (10/2025)
Eric Hedlin, Gopal Sharma, Shweta Mahajan, Hossam Isack, Abhishek Kar, Andrea Tagliasacchi, and Kwang Moo Yi. Unsupervised semantic correspondence using stable diffusion. Advances in Newral Infor- mation Processing Systems, 36, 2024.
Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yacl Pritch, and Daniel Cohen-Or. Prompt-to- prompt image editing with cross attention (ontrol.(022))) URL htlps://urziv. org/abs/ /ttp/2208.00000 2022.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, (16840068511, 2020.
Teng Hu, Jiangning Zhang, Ran Yi, Yuzhen Du, Xu Chen, Liang Liu, Yabiao Wang, and Chengjic Wang. Anomalydiffusion: Few-shot anomaly image generation with dilfusion model. In Proceedings of the AAAI conference on artificial intelligence, volume 38, pp. 8526-8538 2024.
Chaoqin Huang, Haoyan Guan, Aofan Jiang, Ya Zhang, Michael Spratling, and Yan-Feng Wang. Registration based few-shot anomaly detection. In European Conference on Computer Vision, pp. 303 319. Springer, 2022.
Gabriel Iaarco,, Mitchell Wortsman, Ross Wightman, Cade Gordon, Nicholas Carlini, Rohan Taori, Achal Dave, Vaishaal Shankar, Hongseok Namkoong, John Miller, Hannaneh Hajishirzi, Ali Farhadi, and Ludwig Schmidt. Openclip, July 2021. URL https://dof org/10 20..5221/2200 2221/zeeodo.oo 5143773.
Jongheon Jeong, Yang Zou, Taewan Kim, Dongqing Zhang, Avinash Ravichandran, and Onkar Dabeer. Win- clip: Zeero/few-shoo anomaly classification and segmentation. In Proceedings of the IEEE/CVE Conference on Compuler Vision and Pallern Recognition, pp. 19606 19616, 2023.
Dongsheng Jiang, Yuchen Liu, Songlin Liu, Jin'e Zhao, Hao Zhang, Zhen Gao, Xiaopeng Zhang, Jin Li, and Hongkai Xiong. From clip to dino: Visual encoders shout in multi-modal large language models. arXiv preprint ar Xr1:: 08825, 2023.
Alexander C Li, Mihir Prabhudesai, Shivam Duggal, Ellis Brown, and Deepak Pathak. Your diffusion model is secretly a zero-shot classifier. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2206 2217, 2023a.
Chun-Liang Li, Kihyuk Sohn, Jinsung Yoon, and Tomas Pfister. Cutpaste: Self-supervised learning for anomaly detection and localization. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 9664 9674, 2021.
Xiaofan Li, Zhizhong Zhang, Xin Tan, Chengwei Chen, Yanyun Qu, Yuan Xie, and Lizhuang Ma. Promp- tad: Learning prompts with only normal samples for few-shot anomaly detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pottern Recognition, pp.
2024a.
Xurui Li, Ziming Huang, Feng Xue, and Yu Zhou. Musc: Zero-shot industrial anomaly classification and segmentation with mutual scoring of the unlabcled images. orXiv preprint arXiv:2411.16553 2024b.
Yi Li, Hualiang Wang, Yiqun Duan, and Xiaomeng Li. Clip surgery for better explainability with enhance- ment in opea-vocabulary tasks. arXiv preprint arXiv:2304.056: 2023b.
Yiyue Li, Shaoting Zhang, Kang Li, and Qicheng Lao. Ono-to-normal: Anomaly personalization for few-shot anomaly detection. Advances in Neural Information Processing Systems, 37:78371 78393, 2024c.
Bingyan Liu, Chengyu Wang, Tingfeng Cao, Kui Jia, and Jun Huang. Towards understanding cross and self- attention in stable diffusion for text-guided image editing. In Proceedings of the TFFF/CVF Conference on Computer Vision and Pattern Recognition, pp. 7817 7826, 2024.
Wenxi Lv, Qinliang Su, and Wenchao Xu. One-for-all sho-shot anomaly detection via instance-induced prompt learning. In The Thirleenth International Conference on Learning Representations.
13

---

Published in Transactions on Machine Learning Research (10/2025)
Bjoern H Menze, Andras Jakab, Stefan Bauer, Jayashree Kalpathy- -ranmerr Keyvan Farahani, Justin Kirby, Yuliya. Burren, Nicole Porz, Johannes Slotboom, Roland Wiest, et al. The multimodal brain tumor image segmentation benchmark (brats). 1EEE transactions on medical imaging, 34(10):1993 2024, 2014.
Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International conference on machine learning, pp. 8162 8171. PMLR, 2021.
Zhen Qu, Xian Tao, Mukesh Prasad, Fei Shen, Zhengtao Zhang, Xinyi Gong, and Guiguang Ding. Vep- clip: A visual context prompting model for zero-shot anomaly segmentation. In European Conference on Computer Vision, pp. 301-317. Springer, 2024.
Alec Radford, Jong Wook Kim, Chris Hallacy Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference machine learning, pp. 8748 8763. PMLR, 2021.
Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Πya Sutskever. Zero-shot text-to-image gencration. In International conference on machine learning, pp. 8821 8831. Pmlr, 2021.
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hicrarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 1(2):3, 2022.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models, 2021.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10684-10695, June 2022.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical image computing and computer-assisted Attevention-MICC... national conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18, pp. 234 241. Springer, 2015.
2015: 18th inter-
Karsten Roth, Latha Pemula, Joaquin Zepeda, Beruhard Schölkopf, Thomas Brox, and Peter Gehler. To- wards total recall in industrial anomaly detection. In Proceedings of the TEFE/CV conference on computer vision and pattern recognition, pp. 14318 14328, 2022.
Marco Rudolph, Bastian Wandt, and Bodo Roscnhahn. Same same but differnet: Semi-supervised defect detection with normalizing flows. In Proceedings of the IEEE/CVE winter conference on applications of computer vision, pp. 1907-1916, 2021.
Christoph Schuhmann, Richard Vencu, Romain Beaumont, Robert Kaczmarczyk, Clayton Mullis, Aarush Katta, Theo Coombes, Jenia Jitsev, and Aran Komatsuzaki. Laion-400m: Open dataset of clip-filtered 400 million image-text pairs. arXiv preprint ar arxi:21111.0011 2021.
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade W Gordon, Ross Wightman, Mchdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, Patrick Schramowski, Sri- vatsa R Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarczyk, and Jenia Jitsev. LAION-5b: An open large-scale dataset for training next generation image-text models. In Thirty- sizth Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2022. URL httpee/rrrreeeett..fffuuaaaaaaaa (ot
Arnaud Arindra Adiyoso Setio, Alberto Traverso, Thomas De Bel, Moira SN Berens, Cas Van Den Bo- gaard, Piergiorgio Cerello, Hao Chen, Qi Dou, Maria Evelina Fantacci, Bram Geurts, et al. Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: the luna16 challenge. Medical image analysis, 42:1-13 2017.
14

---

Published in Transactions on Machine Learning Research (10/2025)
Shelly Sheynin, Sagie Benaim, and Lior Wolf. A hicrarchical transforattion generative model for few shot anomaly detection. In Proceedings of the IEEE/CVF international conference on compuler vision, pp. 8495 8504, 2021.
Yang Song, Jascha Sohl-Dicksten Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modcling through stochastic differential equations. arXiv preprint arXiv:2011.13450, 2020.
Patrick von Platen, Suraj Patil, Anton Lozhkov, Pedro Cuenca, Nathan Lambert, Kashif Rasul, Mishig Davaadorj, Dhruv Nair, Sayak Paul, William Berman, Yiyi Xu, Steven Liu, and Thomas Wolf. Diffusers: State-of-the-art diffusion models. bttpps//githhu.... httddd/stt///iittt...o////gggee
2022.
Monika Wysoczańska, Oriane Siméoni, Michaël Ramamonjisoa, Andrei Bursuc, Tomasz Trzchíski, and Patrick Pérez. Clip-dinoiser: Teaching clip few dino tricks for open-vocabulary semantic segmenta- tion. In European Conference on Computer Vision, pp. 320 337. Springer, 2025.
Xincheng Yao, Zixin Chen, Chao Gao, Guangtao Zhai, and Chongyang Zhang, Resad: A simple framework for class generalizable anomaly detection. Advances in Neural Information Processing Systems, 37:125287 125311, 2024.
Vitjan Zavrtanik, Matej Kristan, and Danijel Skočaj. Draem-a discriminatively trained reconstruction bedding for surface anomaly delection. In Proceedings of the IEEE/CVE international conference computer vision, pp. 8330 8339, 2021.
Xinyi Zhang, Naiqi Li, Jiawei Li, Tao Dai, Yong Jiang, and Shu-Tao Xia. Unsupervised surface anomaly detection with diffusion probabilistic model. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6782-6791, 2023a.
Xujie Zhang, Binbin Yang, Michael C Kampffmeyer, Wenqing Zhang, Shiyue Zhang, Guansong Lu, Liang Lin, Hang Xu, and Xiaodan Liang, Diffcloth: Diffussin based garment synthesis and manipulation via structural modal semantic alignment. In Proceedings of the IEEE/CVF International Conference on Computer Vision pp. 23154-23163, 2023b.
Wenliang Zhao, Yongming Rao, Zuyan Liu, Benlin Liu, Jie Zhou, and Jiwen Lu. Unleashing text-to-image diffusion models for visual perception. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5729 5739, 2023.
Chong Zhou, Chen Change Loy, and Bo Dai. Extract free dense labels from clip. In European Conference on Computer Vision, pp. 696-712. Springer, 2022.
Qihang Zhou, Guansong Pang, Yu Tian, Shibo He, and Jiming Chen. Anomalyclip: Object-agnostic prompt learning for zero-shot anomaly detection. Xiviv preprinl XrXiv:2310.1 10.00.181999 18961, 2023.
Jiawen Zhu and Guansong Pang. Toward generalist anomaly detection via in-context residual learning with few-shot sample prompts. In Proceedings of the TEEE/CVF conference on compuler vision and pallern recognition, pp. 17826 17836, 2024.
Yang Zou, Jongheon Jeong, Latha Pemula, Dongqing Zhang, and Onkar Dabeer. Spot-the-differeet self- supervised pre-training for anomaly detection and segmentation. In European Conference Computer Vision, pp. 392 408. Springer, 2022.
A Appendix
A.1 Further motivation for feature-based anomaly detection
Addressing anomaly detection with a single-perspective approach is challenging due the open-ended nature of abnormality. Discriminative models focus on global features but often miss fine- grained details, which are crucial for detecting local structural or textural anomalies.
15

---

Published in Transactions on Machine Learning Research (10/2025)
When utilizing generative models in anomaly detection, the reconstrrucion--asee approach has traditionally been the dominant, paradigm (Zavrtanik et al., 2021; Zhang et al., 2023a; Guo et al., 2023). This approach compares the original image and the reconstructed image at both ends. However, since generative models are primarily designed for creative image generation, they can produce high reconstruction errors even for normal images. In addition, deciding how to apply the mask, including its location and size is challenging. As result, the feature-based approach has recently gained prominence.
Building on this shift, propose that a more fundamental approach to utilizing generative models for anomaly detection is to extract features midway through the process. This leverages the image understanding capability that underlies their image generation capability.
A.2 Further experimental details
Data pre-processing For PatchCLIP, we use the same pre-processing pipeline as in WinCLIP (Jeong et al., 2023), which is specified in OpenCLIP (Ilharco et al., 2021), for both MVTec-AD and VisA datasets. First, bicubic interpolation is used to resize images to a resolution of 240 to fit the ViT-B-16-pluse After which, channcl-wise standardization is applied with the precomputed values of (0.48145466, 0.4578275, 0.40821073) for the mean and (0. (0.26862954, 0.26130258, 0.27577711) for the standard deviation. For MapDiff, we use bilinear interpolation to resize images to a resolution of 512 before feeding into a customized Stable Diffusion Inpainting pipeline from the diffusers library (von Platen ct al., 2022).
model.
Prompts The prompts are derived from those used to train CLIP for the ImageNet dataset (Deng et al., 2009). For example, in the case of a segmentation task on MVTee--AD, PatchCLIP uses the prompt "a good, cropped picture of the [state] [object] for classification" where [state] is either "damaged" or "perfect". MapDiff uses the prompt "a close-up cropped png photo of a [object] with [state] for anomaly segmentation" for query images. The [state] includes commonly observed anomaly types "crack", "hole", and "residue", along with a generic descriptor "damage". For reference images, it uses the prompt "a photo of a perfect [object]".
Evaluation metrics We follow previous studies (Cohen & Hoshen, 2020; Defard et al., 2021; Roth et al., 2022; Jeong et al., 2023) and use the following performance metrics. For anomaly classification, we report the Area Under the ROC Curve (AUROC) the image level and the Area Under the Precision-Recall Curve (AUPR). For anomaly segmentation, we report the AUROC at the pixel level and the Area Under the Per-Region Overlap Curve (AUPRO) (Bergmann et al., 2020; Li et al., 2021). Both AUPR and AUPRO are particularly sensitive to class imbalance.
Hyperparameters We compare weights of 0), (0, 1), (0.75, 0.25). (0.25, 0.75), and (0.5, 0.5) between the CLIP and diffusion models. We observe consistent trend: (0.25, 0.75) is effective for segmentation tasks, while (0.75, 0.25) performs better for classification tasks. The same weights are used across all datasets for each task. Please refer to Table
Baseline performance The performance of Trans-MM, MLPP PaDiM, PatchCore, CLIP-AC, and WinCLIP are referenced from the WinCLIP paper (Jeong et al., 2023).
A.3 Further empirical study
Table 9 demonstrates the offectiveness of extracting cross-attention maps from the encoder (down-blocks) and decoder (up-blocks) while excluding the bottleneck (mid-block). Integrating the bottleneck results in suboptimal performance due to its lack of spatial granularity critical for tasks like anomaly detection.
Table 10 illustrates the impact of using state ensemble for cross-attention map. Without ensemble, only generic term "damage" is used to generate the cross-attention map. In contrast, the ensemble approach
Since the location of the anomalous part not known, it is unclear where to apply the mask. Also, if the mask is too large, the reconstructed image may deviate significantly from the original image. Conversely, if the mask too small and fails fully cover the anomaly, the anomaly may persist or even extend in the reconstructed image.
16

---

Published in Transactions on Machine Learning Research (10/2025)
Table 9: Denoiser components for cross-attention maps.
Task
Denoiser Component
Dataset
Enc
Bneck
Dec
MVTec-AD
VisA
x
x
94.4
96.0
Seg.
/
/
94.4
96.4
v
X
/
95.1
96.5
V
83.2
74.1
Cls.
V
V
V
86.8
76.1
V
X V
87.2
78.9
averages cross-attention maps derived from both the generic term "damage" and more specific terms such "crack," "hole,' and "residue." This approach significantly improves performance across tasks. By incorporating these multiple specific states, the method generates diverse cross-altention maps that capture fine-grained and complementary aspects of anomalies.
Table 10: State ensemble (generic and specific).
Task Ensemble
Datasct MVTec-AD
VisA
w/o
94.0
95.8
Seg.
w
95.1
96.5
Cls.
W
85.4
73.1
w/
87.2
78.9
Table 11 shows the impact of customizing state descriptors for each object class on performance. For example, for both Cable and Pill, replacing the default of states with class-specific set composed of frequently occurring anomaly types leads to consistent performance improvements.
Table 11: Effect of state customization.
Object
State Set
AUROC
"crack", "hole", "residue", "damage" "crack", "poke", "scratch"
94.6
Cable
95.0
"crack", "hole", "residue", "damage" "crack", "scratch", "residue"
83.0
Pill
83.5
Table 12 presents results obtained with different sets of states. The results show our method remains reasonably robust across these variations.
Table 12: Different sets of states.
States
Task
Dataset MVTec- AD
VisA
Seg.
95.8
97.0
"contamination" "bend", "cut", "daaage"
Cls.
95.4
85.9
Seg.
95.8
97.3
"break", "fold", "stain", "damage"
Cls.
95.6
86.1
A.4 Further results in the one-for-all paradigm
We evaluate anomaly detection performance under the one-for-all paradigm, where single model is applied across all object categories. Recall that, in this paradigm, the one-shot setting provides one image per
17

---

Published in Transactions on Machine Learning Research (10/2025)
category, for total of N images across N object categories. IIPAD (Lv et al) trains prompt generator using these examples for the one-for-all setting. In contrast, CLIPFUSION requires no additional training. Nevertheless, it outperforms or matches IIPAD across all configurations and datasets for both segmentation and classification, as shown in Tables 13 and 14.
Table 13: Quantitative results for anomaly segmentation in the one-for-all paradigm.
Setup
Method
MVTec-AD
VisA
AUROC
AUPR
AUROC
AUPR
IIPAD
96.4
89.8
96.9
87.3
1-shot
CLIPFUSION
96.0
91.1
97.2
87.3
IPAD
96.7
90.3
97.2
87.9
2-shot
CLIPFUSION
96.4
91.9
97.4
88.1
IPPAD
97.0
91.2
97.4
88.3
4-shot
CLIPFUSION
96.8
92.5
98.1
88.6
Table 14: Quantitative results for anomaly classification in the one-for-all paradigm.
Setup
Method
MVTec-AD AUROC
VisA AUROC
AUPR
AUPR
IIPAD
94.2
97.2
85.4
87.5
1-shot
CLIPFUSION
94.5
97.3
85.5
88.1
IPAD
95.7
97.9
86.7
88.6
2-shot
CLIPFUSION
96.0
97.9
86.8
89.3
IIPAD
96.1
98.1
88.3
89.6
4-shot
CLIPFUSION
96.4
98.2
88.3
90.0
A.5 Limitations of few-shot training
Training-based methods inherently suffer from epoch selection sensitivity. Figure 4a shows that the epoch which PromptAD performs best differs significantly across object categories. In conventional training setups, such epoch selection is guided by validation set. However, in few-shot anomaly detection, where only one to four images are available, it is practically hard to reserve any for separate validation set. The authors of PromptAD circumvent this by using the test set for validation in their official implementation." However, test sets are not accessible during model training in real-world applications. In contrast, CLIPFUSION requires no additional training and uses few-shot examples solely for feature extraction, making validation sets and epoch selection unnecessary.
Figure 4b reveals that even when the training epoch is fixed, performance varies considerably across different random seeds. This highlights the instability and seed sensitivity of few-shot training, making it difficult to ensure consistent and reproducible results without validation set.
A.6 Effect of weights between the CLIP and diffusion models
We analyze the effect of varying the weights assigned to the CLIP and diffusion models. Recall that a and (1 ax) denote the weights of the CLIP and the diffusion model, respectively. By changing a, we adjust their relative contributions and evaluate the performance. As shown in Table 15, segmentation achieves the best performance among the tested alpha values a 0.25, while classification performed best at 0.75.
hhttps://github... cacchhub.com//aaii www/Fuli--eeeeoooooooeeo 0//rdt---O/rompp0
18

---

Published in Transactions on Machine Learning Research (10/2025)
Seerh0 Sewrdo
180
Ryuce
84
82
AUROC
00
78
74
Eposh
(a) AUROC for individual objects.
(b) Average AUROC over objects.
Figure 4: AUROC across training epochs on the VisA datasct (1-shot setting).
Table 15: Effect of weights on performance.
Task
Weight
Dataset
CLIP
Diff
MVTec-AD
VisA
0
94.7
93.7
0.75
0.25
95.4
96.1
Seg.
0.5
0.5
95.7
97.2
0.25
0.75
95.8
97.3
0
1
95.1
96.5
0
93.5
82.2
0.75
0.25
95.4
86.5
Cls.
0.5
0.5
95.0
86.4
0.25
0.75
92.7
83.9
1
87.2
78.9
A.7 Failure cases
Figure 5 provides reprosentative failure cases. In Figure 5(a), although the anomaly lies in the outer sheath of the cables, the diffusion-based model over-activates on the dense and repetitive wire patterns, failing to highlight the true defect. Figure 5(b) shows logical anomaly, where the absence of transistor itself constitutes the defect. Lacking this contextual understanding, the models struggle to determine where to attend, resulting in mislocalization.
A.8 Results on different backbones
Table 16 reports the performance of our framework with different diffusion backbones. We observe that the results are consistent across backbones. This indicates that our method is robust to the choice of diffusion backbone.
A.9 Pseudocode
19

---

Published in Transactions on Machine Learning Research (10/2025)
Mark
NUYUU
Mupoil fulmm
(a)
Attention saturation (Cable)
Tage
Mask
Nuc Scor texmr
Mupoil Scare teatruu
(b) Logical anomaly (Transistor)
Figure Failure analysis examples.
Table 16: Different backbones.
Backbone
Task
Dataset
AD
VisA
Seg.
95.8
96.7
Stable Diffusion v1.5
Cls.
93.5
86.4
Seg.
95.8
97.3
Stable Diffusion v2.0
Cls.
95.4
86.5
Seg.
95.8
97.3
Stable Diffusion v2.1
Cls.
94.6
86.5
20

---

Published in Transactions on Machine Learning Research (10/2025)
Algorithm CLIPFUSION for Anomaly Segmentation
Require: Query image æ; optional k normal reference images R (ri >== object label o; state set C; CLIP image and text encoders fi,fr; diffusion denoiser eg; CLIP blocks B; Diffusion timestep-block pairs
T= [t, fusion weight [0,
Ensure: Pixel-level anomaly map M
1: Initialize MLI.P Mifff. +
P See Sec. 4.1.1 zero-shot
(CLIP, languago-guided)
Encode with fr obtain the patch embeddings Encode prompts "a photo of the perfect [o]" and "a photo of the damaged [o]" with fr
5: Form MCLIP,L by per-patch aligument (abnormal vs. normal)
6: (Diffusion, language-guidest
7: for state E C do
8: Run 6e on x with empty mask and prompt "a photo of a [o] with [s]"
Extract cross-allention map for state 8; accumulate Moart,L MDire, + CrossAtin(s)
10: end for
11: Moitt, t - /|C|
12: Initialize MCLIP, V <- 0
b See Sec. 4.1.2 few-shot
13: if h > 0 then
14: (CLIP, vision-guided)
15: For each r R and E B, extract features from fr and store into Rd
16:
For each (h,w). w),
17:
MALPP.
ELEB min 2/2 (1 sim{ Fh Fl..: +))
(Eq.
18:
(Diffusion, vision-guided)
19: For cach r E R and (t,b) E T, extract features from and store into R
20: For each w),
M
+ E(.yET min 2 (1 sim (F,(;)
21:
(Eq. 8)
22: end if
23: Fuse maps M + a(MCLIP, + MCLIP,V) + (1 a) Mottt.
(Eq.
24: return M
21

---

Published in Transactions on Machine Learning Research (10/2025)
Algorithm CLIPFUSION for Anomaly Classification
Require: Query image x; optional k normal reference images R. = (ri) object label o, state set C; CLIP image and text encoders fi.fr: diffusion denoiser EØ; CLIP blocks B; Diffusion timestep- block pairs
T= ((t fusion weight [0
Ensure: Image-level anomaly score S
Initialize SCLIP,4, SDief, +-
See Sec. 4.2
2: (CLIP, language-guided)
3: Encode with fr to obtain the class token embedding
4: Encode prompts "a photo of the perfect [o]" and "a photo of the damaged [o]" with fr Calculate SeLIP,L via Eq. (1)
(Diffusion, language-guided)
Compute M in Alg.
guitt,L 1 - modian(MMitta. max(Mrrtt.))
(Eq. 10)
9: Initialize SCLIPVV g0itt.V +
10: le > 0 then
11: (CLIP, vision-guided)
12: Build memory banks R² from fr; compute V as in Alg.
13:
SCLIP,V max(A/
(Eq. 11)
14:
(Diffusion, vision-guided)
15: Build memory banks R from eØ; compute V in Alg.
16:
SDift,V +- mmx((Ma) max(MtDttt.V)
(Eq. 11)
17: end if
18: Fuse scores + ) + (1-
a) + soiet.y)
(Eq. 9)
19: return S
22

---

Published in Transactions on Machine Learning Research (10/2025)
A.10 Detailed quantitative results
Table 17: AUROC (%) results on MVTec-AD classification.
Categury
k-0
1-1
k-2
k-4
CLIPPUSION
PADOM
Patch/Core
WinCLIP
CLIPPUSION
PEDIM
PaschCacu
WInCLIP
CLIPEUSION
PRDIE
PaichCure
WInCLIP
CLIPT USION
Buith
10200)
05.22055
1006:5.0
98.2210
91:2:01
$5:3:13
0000200
35,2412
1000000
Gaide
80 Si.cs00
85 12.200
077744
88 8.
05.9:15
$5,020.9
02,855.
9.0:2.7
55.4207 7
56.180
70026.1
910827
00.3:3.
06.701.4
Capezto
T2M400
8511216
52.76711
07 07.8224
723228.3
61616.9
01212:0
718174
77:3288
58/2227
53:2325
718270
82:2:5
00.22145
Carpst
0000:00
16
013:0.5
032401
1000-600
07 17.4
936203
$5:3-03
1010401
37.340
100040.0
103000.00
56.1204
Teather
1063-00
010000
$3.3000
15.1000
07.5405
55-3-00
1010-00
073107
105010.
1030-0.0
Mesal VEE
07 1-000
002100
530-334
23:0225
95.1103
51512.5
S5-1-02
10:1-02
567153
77.2485
1010:00
Pil
23 10.1
772228
01.2234
11 219222
0::2221
35.12114
01.125-4
023523
52.32013
16.22012
$4/122
$13222
50.5210
Scoee
723186
414:14
35-1185
715185
548114
10:01:1
13.1151
30/11:1
168138
Twollens's
$75816
06.722
8
16.7.166
81.4353
90311.44
72816-3
13:3117
52.1165
M5.2225
Riod
6:00
077:006
1:11
07 8:80
93:9:0.
96 08.405
05:2:05
$5:500
56.580
97 0788
15.2506
06. 1
Zijes
01 0:003
910:00
02 9:03
65:8-5.
965407
953456
010-9
910:14
66.1-22
55.3420
1100211
045599
Table 18: AUPR (%) results on MVTec-AD classification.
Categury
h-0
1-1
1-2
ked
WisCLIP
CLIPFUSION
P.DOM
PatehCore
WinCLIP
CLIPFUSION
PsDMM
PaschCuce
WinCLIP
CLIPEUSION
PaDIM
PrichCure
WInCLIP
CLIPF USION
1kittte
1M8200
156/221111
M12533
91.42863
11410:5:11
91322113
9122211
$5.3211
101102602
29.2311
$532111
16823911
Calde
01 9-000
0272406
610-3.5
02::2:3
912-111
95.1005
03645-6
95.1:133
$5:9-06
07.54033
761455
$7.1407
B6:1:03
[6. 161.
Capealo
91 81.28.3
072286
MIS211
81422/
91/8227
5533222
56.2211
SLR223
$1.32:66
47,13155
113211
56,1333
Cames
28602111
100:20111
9332261
0010202
Saedur:
20:1202
Leatles
10002500
100:221
01.283
01 1:52
10000530
1000
91.2203
912261
10000200
100 02600
39.221L1
$5/2011
10102000
103
Manal see
M-2-00
W161B6
11
95311011
02-211-4
9131:2
$53.00
65-111.7
S7/126
163-01
PiI
F-003
918:06
05.2303
07 57.0226
938207
$6.2011
16.32000
57.0815
36.9201
16.1-12
1
Saace
912:00
5722677
72.0:54
911111
57,4532
718+10
013-08
06
Tewt
188:06
9577486
95.0105
5523116
56.7503
55
167el
1:000
S47221
06:227.
71.725.1
7188260
$1A21.5
01112533
813253
567225
55.3217
77.02844
5432900
N67:12
121213
Wood
18.5251
01328.1
11
31.72111
913221
INIR250
18.32111
19.1211
5555212
9032001
Piyper
07 8-06
5-
07 9-8-
518-11
$4.9103
95 45.0410
978811
1-33-01
14.3-1.0
96.1105
96.1407
1655-09
76.7
06,800,0
95.5205
07.8:0.4
91.8117
97.9:02
90.8816
94.816
66.360
90.5:0.1
Table 19: AUROC (%) results on MVTec-AD segmentation.
Calegory
Yoel
1=2
kadd
CLIPPUSION
PaDIM
PatehCore
WinCLIP
CLIPFUSION
PaschCCrv
WinCLIP
CLIPFUSION
PaDIM
PutchCure
WinCLIP
Botts
9411211
911122
07.02811
97.2201
30.22122
91.2211
95.120
$1.75111
(7.32in
17.12111
3522300
57.32011
97.7el11
Calle
0--0-
213106
00.5-8.4
$13:DD
$:3100
95/103
$133-01
00.3003
23110.1
$75103
BL3-01
Caperlo
86. B/0
91615C
$16185
053105
955:B4
S1:103
18.1002
$63181
S13186
163:03
16:1d1
Camps
93.22005
553200
56.5000
003:00
MESLDD
15.1122
95.111
$6.21011
16.11111
17.11112
S7.1111
16.10011
16.7101
Leather
I6 7550
012500
02 8:02
9322:00
952200
06 067500
91022.1
$535:00
56.350
35.250.0
$5,1500
00.200
06 00.
Matal ant
7422366
74863.1
813:3.4
9331311
00223322
0133223
916214
$2413144
10.7211
1277230
1032204
S4N:33
111:1:1
33-22813
3521112
57331177
9122111
27/0002
15.12012
55.11115
04321144
07.12011
16
900:07
Towtllenak
90 5020
9122516
91.12111
97,72003
9232212
$5.12011
16.15111
38.12112
55.42114
18.429.
16228.1
28 2-77-0-
012106
819-37
$50-15
673126
9131311
816105
6-33-10
4.3105
S10127
967111
585-19
103-06
Whad
913288
92 T280
94821.
35.721.2
0192161
9122307
48.32014
564291
12,28111
53,52115
56,4202
Ziyper
911 6..00
912166
97 1
$13118
$8/11013
95 05/1163
91:1:07
16.1101
96.1102
96.1001
P13100
163101
Meen
00:00.00
993-59
000:10
95:3401
968--6.1
91 3667
93:3:04
940003
96.3-61
95,667
$13305
062603
9.5-0.11
Table 20: AUPRO (%) results on MVTec-AD segmentation.
Category
k-0
A-2
k-4
Tkittle
26.0.00
$1.2101
$3.7104
91.7103
$19:03
$1.1003
P6.101
23.31012
$10102
Galde
22.9003
TL/488
00123.2
847=10
72.542.5
75.74115
01545.
55.2213
747222
52.3810
142215
$1/2316
17.2e111
$6.42.0
Capeato
621.00
845.11
W11113
K18184
41:1127
01511.1
81312-1
856114
905186
111182
55,2113
S7A119
16111.
Carpes
451155
21755.
$1:8225
08.5200
Haadur
94.23013
04.7515
30.131LT
5102115
04.200
Leather
01 1-00
07 7-0
$5.500
$735001
9524013
010:03
$5-3-01
057701
36.3001
S5-1001
06:300
07
Meall su
9-80
38
312-1-
756--3
S17-05
310165
$1.155
50-1-01
0132212
3101115
3431544
3432002
24.1202
5132114
51 01-03
31 01.010
815/11
32/1104
915103
71.115
815105
S54101
$1.100
$10101
685-03
Towillaak
07 7253
916286
821511
210:24
413216
9282299
813655
947514
02.805
97.181.7
$55250
96.233
<14.100
525186
715174
010114
618114
543130
75513.5
748111
(A1111
71.110
53.1174
71.1126
BA3L111
15:11133
Wind
24 1=00
12.1406
87 1:10
910505
353405
55 1H10
818114
$13206
55.1511
57.2401
017740
Ziypet
Re00
919500
9-11
850=17
554566
$16467
019564
8,4116
867808
57.2405
91,402
56
Mena
06 R550
87.130.0
745638
87.1114
90.4:8.1
762613
82881.1
564809
91-4602
$1.1815
$43816
80380
52,1202
23

---

Published in Transactions on Machine Learning Research (10/2025)
Table 21: AUROC (%) results VisA classification.
Chtegery
k-D
k-1
k-2
k-d
WaCCUP
CLIPTUSION
Patek/Oue
WinCLIP
CLIPPUSION
PuDiM
PuichCor
WHOLLIP
CLIPFUSION
PuichCare
WICCIP
CLIPFUSION
Ciasli
14:22412
D1324:1
S.1214
50-8214
TG:1223
863211
342533
554211
Tr.111
813211
StREEd
Capondo
S3.122.1
53:22403
01
01:25.4
94.6111
56.100
095-19
0533103
074108
071100
68-7121
074106
$7311.
075:00
815-33
$5310033
18.7 0:103
$7.7103
Pexun
20.305.1
58.2203
88325
55.1219
02.28 1
06.255.0
15,4224
33/224
56
1122550
350213
117224
Manuusil
7922802
02 12
12-2215
015227
10:2550
TL85666
332210
$13:1
0228257
862-39
010014
114110
01.3156
7181221
T100123
6:7.14
715-12
1172114
PCR2
0
60:2:00
06:32200
66.3011
07.m22
PCB3
72 40.00..
07 172400
00
80 2
16.6111
0031335
$6.521
788:19
7622-21
665:10
070:11
883:11
75 75
PCBI
769440
06 1:31
852550
033418.7
04.3533
5233563
S78:500
916-18
87 876-33
S11:18
Plye fora
33.3200
017:32
MUT8LT
11.211
908631
15.1618
56.1213
9652116
93,4267
963.94
1736512
Mean
78.510.0
028.34
835140
86./01
07.1153
81,6180
$16184
K7.00.5
M53121
873.18
A60000.2
Table 22: AUPR (%) results on VisA classification.
Categery
k
WIsCLIP
CLIPTUSION
PaDis
Patel/Oure
WinCLIP
CLIPEUSION
PaDiM
Puid&Cwe
WinCLIP
CLIPFUSION
PaDIM
PurchCore
WIaCLIP
CLIPFUSION
3:2
10.5115
T2.121.0
01:11311
50:9:1
72:1:111
55311311
Callew
$1.9400
07:3005
055-3.
063:03
$73362
66:2:04
6811-20
95.2107
07.7:94
$3.0006
Choviuggrz
15.2200
1602315
161219
5612212
S5925L1
9622313
881::1
91,22811
$1312211
(8112221
5482802
967210
812234
10.1213
1982822
163:315
3132204
91/
07:12261
Mararoill
75 1
411140
011-3
S13:13
03
T19:58
417:15
1.1-00
6-5-21
0311555
863-06
1:0.0
470100
086-34
751:35
255-150
2161364
3.3:433
17::22
87T-11
15313.5
PCB2
053:12
643255
01.2511
18:1226
8.2211
518240
112255
052:10
113:34
713220
PCB1
7822802
13:1210
131216
MI221
16.2515
861201
319214
R03=25
061:24
053211
868-1A
21RE14
77.11011
76.5102
26T-26
12..11
N5.335
186.11
676.113
86.1::4
7631163
818:56
$18:13
0:3121
Mran
51.540.0
43.610.0
092-10
875-33
85.1-1
85.1-0.3
71-5-35
615--3
858482
685-655
756-99
875181
886-18
485-0.11
Table 23: AUROC (%) results on VisA segmentation.
Catsgory
WinCLIP
CLIPFUSION
P.D:M
PukdCCure
WinCLIP
CLIPFUSION
PaDIM
PatekCare
WMsCMP
CLIPPUSION
P.DIM
ParchCare
WinCLIP
CLIPPUSION
553:00
17.12013
97.1m02
55.4:012
00.1:0.1
35,2201
17240.2
Cueberr
$42:30
12.1113
155.06
16.1.0.1
16.5.02
$8:11113
95.11b
18213
965.15.11
16.1.01
15.21013
94.111
457100
94115.
Clesingamm
56.2:00
28.160
00 41
063=03
96.6c0.1
16050.1
011:87
06 08.e85.
06.8-9.1
$1,2005
912501
98/20.
08 155611
18.3210
2251113
013:34
10.3:13
15:22113
911:14
9ER23-1
15312111
9121303
47.1111
2882113
181:33
16.2:11
1811111
MLTIIL
1852007
17112112
PCB)
61:2-00
080-33
0.1-15
6-6-96
16.5:05
981465
07
PCB2
1142310
11.12113
905:14
16,4212
16.220
16.13115
919250
11.7110
918252
948226
97 07.3333
PCRS
553300
45.1103
6.0-9.
163-03
M.3-03
16:1-02
951-05
07 1-31
101100
05.200
073100
358101
00 5254
PCBI
014100
53.183
164--6
81.-01
15.3105
917/64
156103
167-09
020163
961-663
97 175109
Plp Gyyu
1544500
57/220.3
96.5=0
96.500
5544801
9.1584
0.11
90.60011
$6.500
91.100
Menn
92.119.0
819:9A
16,4208
96.5304
97.3:0.1
9211332
9.1:333
97.7:011
$1,23055
918163
17221022
95,020.
Table 24: AUPRO (%) results on VisA segmentation.
Calegory
WiotiaP
CLIPYUSION
PsteaCorre
WIOCLIP
CLIPUSSUSS
PatchClore
WISCHII
CLIFFUSION
PSDDM
Parchisce
WIsCLLIP
CLIPPUION
Condee
533185
BLD-044
$1.107
104-33
01.-03
553100
911164
514163
803453
Capeaka
M.3210
53.2413
T5
41:222
770114
Chrsteggen
7641000
18.1113
$61.86
7.3-133
01..05
153112
140
S10104
915.57
77.11000
4421113
711-16
767-23
M61-11
63.1121
R151002
112113
S18113
437105
8531144
Menscail
5432000
27/500
023:14
55:113
SLS-23
500:055
718454
01.200
76342.1
210857
358805
0L5637
23.4:110
75.1113
545-16
M11-14
15:1210
018134
S53111
7421114
91511.1
PCRS
S72200
7333603
611:33
865-05
5
72045.1
$104.9
PCRS
161-00
562240
000:13
181-200
755-15
157410
710:23
83 815-11
00541
77.2405
$14413
S13:10
a17-11
2CB4
5042300
51.21113
001:1A
17.1:11
543215
028254
NLT2:22
061:12
S131225
615453
4122107
Pipe
5033:00
353969
164-05
861-06
164-03
965-34
003-01
442413
158165
96160
Menn
565500
85.350.0
80::25
8.1=21
$70:08
N.1554
8624.4
$7.3:03
7368119
$10514
876663
87.5503
24

---

Published in Transactions on Machine Learning Research (10/2025)
A.11
Detailed qualitative results
Normal
Anoualy Image
Ground Truth Mask
PaschCLPP Score
MapDiff'Sonn
CLIPPLSION Score
500
500
KOK
Figure 6: Qualitative results for 0-shot CLIPFUSION with images for PatchCLIP and MapDiff anomaly score maps on MVTec-AD.
25

---

Published in Transactions on Machine Learning Research (10/2025)
Normal Imagg
Anomaly Image
Groud Truth Mask
PatchCTIP Scure
MapDiffSeare
CLIPFUSION Score
Figure 7: Qualitative results for 0-shot CLIPFUSION with images for PatchCLIP and MapDiff anomaly score maps on MVTec-AD.
26

---

Published in Transactions on Machine Learning Research (10/2025)
Image
Anomaly Image
Ground Trath Mask
PatchCLIP Score
Magihiff Soure
CLIPPUSION Score
Figure Qualitative results for 0-shot CLIPFUSION with images for PatchCLIP and MapDiff anomaly score maps on VisA.
27


Reviews:
[{"review_id": "GsgXfAzpvQ", "reviewer": "Reviewer_VVJ4", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces a novel CLIPFUSION framework for anomaly detection based on CLIP and diffusion models. The proposed framework allows to extract intermediate features from an off-the-shelf diffusion model without any training or iterative denoising. Extensive experimental results demonstrate the effectiveness and computational efficiency of the CLIPFUSION framework in multiple benchmarks.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The motivation of the proposed framework for integrating CLIP and diffusion models is clearly explained. The technical details are easy to follow.", "audience_yn": "Yes", "audience_explanation": "This work provided a simple yet effective anomaly detection approach. Thus, TMLR's audience will be interested in the proposed framework as well as the empirical results shown in the experiments.", "requested_changes": "(1) The rationale behind the proposed framework is not well-explained. For example, it is unclear why the cross-attention mechanism and each of the score maps in Section 4.1 can improve anomaly detection.\n\n(2) Following previous concerns, one key hyperparameter in the proposed framework is the weight $\\alpha$ between the models. As shown in experimental settings, classification and segmentation require different values of $\\alpha$. However, the impact of this hyperparameter is not discussed. Besides, it is unclear how the value of $\\alpha$ is selected for the one-for-all paradigm results in Table 8.", "broader_impact_concerns": "No explicit ethical concerns in this work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "vOghfsUT3Z", "reviewer": "Reviewer_GBSn", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces CLIPFUSION, a training‑free framework that fuses a CLIP‑based branch (PatchCLIP) with a diffusion‑based branch (MapDiff) for anomaly segmentation and classification in both zero‑shot and few‑shot settings. On the diffusion branch, the author avoids doing iterative denoising; instead, they pass the clean image once through an inpainting pipeline with an empty mask to get cross‑attention maps and decoder feature maps, which contains rich spatial information for detecting anomalies. On the CLIP branch, they leverage patch‑level features  to provide a complementary global prior without sliding windows. The two branches are combined with fixed task‑dependent weights, yielding a single anomaly map or score. The method targets industrial benchmarks (MVTec‑AD, VisA), reports strong performance and speed, and also explores a “one‑for‑all” few‑shot setting in which a single model handles multiple object categories without any training.", "claims_and_evidence_yn": "Yes", "claims_explanation": "Yes, across both datasets, CLIPFUSION outperforms prior methods in all four settings (zero/few‑shot × segmentation/classification). For example, in zero‑shot segmentation, AUROC improves from 85.1 to 90.9 on MVTec‑AD and from 79.6 to 92.1 on VisA over WinCLIP; in zero‑shot classification, it edges out WinCLIP on both datasets as well. Tables 1–2 document these margins with averages across five seeds and standard deviations.\n\nThe author also implements ablation studies for the contribution of each model and modality. Using only CLIP or only diffusion degrades performance; combining them obtains the best performance for both tasks (Table 3). Splitting language‑guided and vision‑guided pathways shows that each brings complementary information (Table 4). These ablations directly substantiate the core premise that discriminative and generative foundations capture different, useful signals.", "audience_yn": "Yes", "audience_explanation": "Yes. The work integrates vision‑language modeling and diffusion‑based perception for anomaly detection, and it addresses a practical task of zero/few‑shot anomaly detection, where training data is scarce and deployment constraints matter. The idea of treating a text‑to‑image diffusion model as a feature extractor and fusing it with CLIP in a single, training‑free pipeline is likely to interest readers working on open‑vocabulary recognition and multimodal learning.", "requested_changes": "1 A strength of the paper is extensive per‑class reporting (Tables 14–21, pages 19–20). These tables also show a few informative counter‑examples. On MVTec zero‑shot classification, CLIPFUSION trails WinCLIP on Screw and Transistor (Table 14). On MVTec zero‑shot segmentation, Cable is lower than WinCLIP (Table 16). On VisA zero‑shot classification, several categories (e.g., Candle, Cashew, PCB4) are slightly behind WinCLIP (Table 18). A short failure analysis that overlays PatchCLIP and MapDiff maps for these cases, e.g., are diffusion attentions saturating on repetitive textures, or is CLIP over‑focusing on global shape? This would make the paper more convincing. It would also tell readers when to raise or lower the diffusion weight in practice.\n\n2 All results use Stable Diffusion v2 as the denoiser, with images resized to 512 for MapDiff and to 240 for OpenCLIP (Appendix A.2). Since the core claim is that “diffusion as a feature extractor” is broadly useful, it would strengthen the paper to show backbone sensitivity (e.g., a second diffusion checkpoint with different text encoders or training data) and prompt sensitivity beyond the state‑word ensemble (Appendix A.2–A.3).\n\n3 The proposed framework relies on a small, manually curated set of “state” words (damage, crack, hole, residue), with optional per-class customization, and then averages the resulting cross‑attention maps (Figure 3, page 6; Appendix A.2–A.3). This works well when the anomaly matches the list, but may not work well with defects that are subtle, composite, or product‑specific.", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "uJaWiQiRPR", "reviewer": "Reviewer_akeK", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The method proposes an anomaly detection method that fuses the feature maps from a CLIP and a diffusion model.  Then, it uses these fused features to compute the anomaly score.  The method is compared against different methods for 0 up to 4 shot classification in 2 datasets, and some ablation studies are included as well.\n\n**Strengths:**\n- The idea of mixing local and global features to improve the prediction of anomalies is sound\n- The use of pre-trained methods in the mixture is straightforward and reproducible\n- The results show improvements over the compared methods\n\n**Weaknesses:**\n- The design choices are not supported and are just stated\n- The improvements of the claimed mixed local and global maps are not fully verified since the CLIP and diffusion methods are trained on different datasets.  Thus, the improvements could come from the different training sets rather than the local/global characteristics", "claims_and_evidence_yn": "Yes", "claims_explanation": "Despite the weaknesses, the method claims to mix pre-trained models and to obtain better performance.  These claims are supported by the presented experiments according to the compared baselines. \n\nThe claim about the method being \"training free\" is misleading and should be reviewed.", "audience_yn": "Yes", "audience_explanation": "Yes.  Anomaly detection is an interesting topic; and the use of open models with a straightforward incorporation of the features to be converted into anomaly scores may be of interest.", "requested_changes": "- It is not clear what the authors mean by \"entirely training-free\" in the contributions.  The methods they are using are pre-trained, and the comments of what they are avoiding (fast deployment and sensitivity to training) still applies to the pre-trained models.  My guess is that they want to claim that using pre-trained models is beneficial, but it is a claim that should be revised, and even removed.\n\n- Why use different encoders for the diffusion and the CLIP models?  Could the benefit come from the redundancy that the encoders provide rather than the different models (CLIP and diffusion)?  One way of evaluating this will be to have two CLIP/diffusion models with two different encoders trained independently and then do the mixture as well as the model-wise evaluation to verify where the gains are coming from.\n\n- How are the abnormal conditions in Eq. 6 defined?  What is their impact in the final performance?\n\n- Why use different strategies for the diffusion and CLIP scores? (Eqs. 10 vs. 11)  This selection should be supported empirically or theoretically.\n\n- You should include pseudocode to fully define the behavior of your proposal.  While the parts are described throughout the paper, having a single point where the whole method is explained in detailed and in a clear way (with code), would help the reader to reproduce your work.", "broader_impact_concerns": "I have no concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)