Few-Shot Anomaly Detection on Industrial Images through Contrastive Fine-Tuning
Authors: Jingyi Liao
Xun Xu
Manh Cuong Nguyen
Chuan-Sheng Foo
Venue: Submitted to ICLR 2023 (2023)
Keywords: Anomaly Detection
Transfer Learning
Few-Shot Learning

Abstract: Detecting abnormal products through imagery data is essential to the quality control in manufacturing. Existing approaches towards anomaly detection~(AD) often rely on substantial amount of anomaly-free samples to train representation and density models. Nevertheless, large anomaly-free datasets may not always be available before inference stage and this requires building an anomaly detection framework with only a handful of normal samples, a.k.a. few-shot anomaly detection (FSAD). We propose two techniques to address the challenges in FSAD. First, we employ a  model pretrained on large source dataset to initialize model weights. To ameliorate the covariate shift between source and target domains, we adopt contrastive training on the few-shot target domain data. Second, to encourage learning representations suitable for downstream AD, we further incorporate cross-instance pairs to increase tightness within normal sample cluster and better separation between normal and synthesized negative samples. Extensive evaluations on six few-shot anomaly detection benchmarks demonstrate the effectiveness of the proposed method.

Under review as a conference paper at ICLR 2023
FEW-SHOT ANOMALY DETECTION ON INDUSTRIAL IMAGES THROUGH CONTRASTIVE FINE-TUNING
Anonymous authors Paper under double-blind review
ABSTRACT
Detecting abnormal products through imagery data is essential to quality control in manufacturing. Existing approaches towards anomaly detection (AD) often rely on substantial amount of anomaly-free samples to train representation and density models. Nevertheless, large anomaly-free datasets may not always be available before inference stage and this requires building anomaly detection framework with only a handful of normal samples, a.k.a. few-shot anomaly de- tection (FSAD). We propose two techniques to address the challenges in FSAD. First, we employ a model pretrained on large source dataset to initialize model weights. To ameliorate the covariate shift between source and target domains, we adopt contrastive training on the few-shot target domain data. Second, to encour- age learning representations suitable for downstream AD, we further incorporate cross-insten cross-instance pairs to increase tightness within normal sample cluster and better separation between normal and synthesized negative samples. Extensive evalua- tions on six few-shot anomaly detection benchmarks demonstrate the effectiveness of the proposed method.
INTRODUCTION
Industrial defect detection an important real-world use-case for visual anomaly detection meth- ods. In this setting, anomaly detection models typically have to be trained with only defect-free, or normal images, as defects rarely occur on functioning production lines. Anomaly detection methods for this one-class classification setting typically assume that normal images are available in abun- dance, even though this may not always the case. For example, in certain applications such as semiconductor manufacturing where image acquisition requires 3D scans using specialized equip- ment (Pahwa et al., 202I). acquiring defect-free images is time-consuming and costly. Flexible manufacturing systems also require rapid adaptation to changes in the type and quantity of products to be manufactured (Shivanand, 2006). As a result, large numbers of defect-free images may not be available for new products, in the initial stages of bootstrapping visual inspection system.
Although anomaly detection in general is a well-studied topic (Chandola et al. 2009; Pang et al. 2021b), anomaly detection on images with only few normal and abnormal images, or few- shot anomaly detection (FSAD), has only recently begun to receive attention from the commu- nity (Sheynin al. 2021; Huang al. 2022). In their pioneering work, /Sheynin et al. (2021) developed a generative adversarial model to distinguish transformed image patches from generated ones. However, such adversarial models may be tricky to tune (Kodali al. 2017) and the method requires multiple transformations on test samples at inference time, resulting in additional computa- tion overhead. The more recent work of [uang et al. (2022) learns a common model over multiple classes of normal images using feature registration proxy task, but their method requires a training set with normal images from multiple known classes, which a more restrictive setting.
In this work, we develop a simple yet effective method for few-shot anomaly detection. We achieve this by synergistically combining transfer learning from a pretrained model with representation learning on the few-shot normal data. Finetuning from backbone network pretrained on large source domain dataset, e.g. ImageNet (Russakovsky et al. 2013), allows reusing good low-level feature extractors and better initialization of network parameters (Kormblith et al. 2019). We be- lieve finetuning from pretrained weights could particularly contribute to few-shot anomaly detection when not enough training data is available for training good representations. However, pointed

---

Under review as a conference paper at ICLR 2023
out by some existing work (Xu et al. 2022 Li et al. 2021b), directly reusing the pretrained weights may not fully unleash the power of finetuning. This is probably caused by two factors. First, when the source domain data has a different data distribution from the target domain, the covariate shift (Wang & Deng, 2018) causes performance degradation. Second, due to the fact that anomaly detection requires feature representation that separates normal samples from abnormal ones. The representations learned from ImageNet pretraining tasks, mostly semantic image classification, is not necessarily optimal for anomaly detection.
To ameliorate the covariate shift between source and target domain data, we first propose to in- troduce contrastive training to adapt pretrained model weights to the target data distribution for downstream anomaly detection. Given initial model weights, we optimize a contrastive loss defined on all available few-shot normal examples so that the pretrained low-level features will be adjusted towards the target data distribution. We further encourage learnt feature representations to be suited to the downstream anomaly detection task by encouraging normal samples to form a cluster in fea- ture space. To achieve this, we introduce cross-instance positive pair loss that randomly samples two normal samples and encourages their feature embeddings to be close. Note that this differs from standard contrastive training as closeness is encouraged across two different normal samples instead of a sample and its augmented version. Finally, when prior knowledge on the anomalies is available, e.g. we able synthesize negative examples (Li et al. 2021a), we further introduce an additional negative pair loss to encourage better separation between normal and synthesized anomalous exam- ples. We empirically reveal that the choice of negative sample synthesis crucial to the success of FSAD and should be used only when concrete prior knowledge on the anomalies is available. We summarize the contribution of this work as below,
We approach anomaly detection for industrial defect inspection from a transfer learning perspective. We propose to do contrastive training on few-shot normal samples in the target domain to alleviate the distribution shift between source and target domains.
We further introduce an across instance positive pair loss to encourage normal samples to form tight cluster in the embedding space for better density-based anomaly detection.
When prior knowledge on negative sample is available a negative pair loss is further incor- porated to allow better separation between normal and synthesized negative samples.
We demonstrate superior performance on real-world industrial defect identification datasets and 2 synthetic corruption identification datasets.
2 RELATED WORK
Traditional anomaly detection (AD) methods include PCA, cluster analysis (Kim & Scott, 2012) and one-class classification (Schölkopf et al. (001). With the advent of deep learning, representation learning employed to avoid manual feature engineering and kernel con- struction. This leads to novel anomaly detection methods based on generative adversarial networks (GAN) Peereaa et al. al. 2019 them, anoGAN (Schlegl Schlegl et al. 2017)) and Autoencoders (Bergmann ct al. | (019a)). Among al. 2017) was proposed to learn the manifold of normal samples and anomalous samples cannot be perfectly projected onto the normal manifold by the generator learned solely with normal samples. However, it requires expensive optimization for detecting abnormal samples and training GANs is prone to some well-known challenges including instability and mode collapse. Among the autoencoder based approaches. (Bergmann al. 2019a) adopted SSIM metric as the similarity measure between input and reconstructed images. Recently an effective line of works approach AD through representation learning and formulate AD as detecting outliers in the learned representation space (Ruff et al.) 2018 Golan & El- Yaniv| [2018] Sohn et al. 2021). Among these works, deep SVDD (Ruff et al. (2018) proposed learn feature embedding that groups nor- mal samples closer to cluster center. Follow-up works develop self-supervised pretraining methods to learn representations suitable for separating abnormal samples from normal ones by optimizing proxy task (Golan & EI- Yaniv 2018 Sohn et al. 2021 Li et al. 202 Anomaly detection is then implemented through fitting a density model on the learnt representations of normal training samples. These approaches prevail in many anomaly detection benchmarks and are computationally efficient. Nevertheless, representation learning requires a substantial amount of training data which may not be readily available in certain industrial environments.
Anomaly Detection:
2

---

Under review as a conference paper at ICLR 2023
Few-Shot Anomaly Detection (FSAD) aims to enable detecting anomalous samples with only few normal samples as training data, and is an emerging topic in anomaly detection. We first distin- guish FSAD from the semi-supervised anomaly detection setting (Ruff et al. 2019) where limited number of labeled anomalies are available for training as it is sometimes referred to as few-shot anomaly detection in the literature (Pang et al.] (021a). The pioneering FSAD method of Sheynnnn et al. (2221) employs a hierarchical generative model to generate new samples from the few-shot examples. A discriminator
designed discriminate generated images from the real ones and different transformations. Anomalies are then determined by whether the discriminator can cor- rectly classify the type of transformations. Inspired by the few-shot learning paradigm, the RegAD method of Huang et al. (2022) uses a registration based proxy task for representation learning; this task aims to find the affine transformation that aligns the feature map of two samples from the same semantic class. Re requires additional related training data, for instance, data from other classes besides the target class on the MVTec dataset, for training the proxy task. The work of Ando & Yamamoto (202)) addresses different few-shot setting that requires normal samples to be provided with semantic labels. When normal data comprises multiple semantic classes, embedding all nor- mal samples into single cluster may result in the failure to detect anomalies occuring between semantic classes. Learning multiple prototypes was proposed to tackle this issue. In comparion, our method adapts pretrained weights to target data using only few normal training samples; unlike some of these other works, no additional data is required during the representation learning phase. This enables our method be applied in broader set of industrial anomaly detection scenarios.
Contrastive Learning: Pretraining feature representation through contrasting augmented samples of the same identity has demonstrated promising results. SimCLR (Chen al. 2020 He al. 2020) employed an N- N-pair loss (Sohn) 2016) to encourage two augmentations of the same instance (pos- itive pair) to be close in the feature space and other instances (negative pair) to stay faraway. The existence of negative pairs requires large batchsize for training. BYOL GGril et al.) 2020) intro- duced an exponential moving average model to avoid collapsed predictions and get rid negative pairs. Apart from representation pretraining, contrastive learning has been recently demonstrated to be effective for label efficient finetuning (Liu et al.) [202]] Xu et al.] 2022, Chen et al.) 2022 Li et al., 2021b). When source and target domain data distributions are subject covariance shift, contrastive training on the target data in a unsupervised fashion can potentially alleviate the domain shift (Xu al.) [222]] Li et al., 20216). In this work, demonstrate that contrastive training on the target domain data plays an important role in learning a good representation for downstream anomaly detection.
3 METHODOLOGY
In this work, we assume model pretrained large external image collection (e.g. ImageNet) is available. We refer to this external data as the source domain. Anomaly detection on industrial data the task be solved and is referred to the target domain. We first describe contrastive training for adapting pretrained model to the target domain distribution. We then introduce the cross- instance positive pair loss encourage normal samples form cluster in the feature space. When prior knowledge on how to synthesize negative samples is available, we can introduce negative pairs encourage better separation of normal and abnormal samples in the feature space. An overview of Lastly, we describe how to build
the proposed contrastive adaptation framework is shown in Fig. [ the density- based anomaly detection model on the learnt representations.
3.1 CONTRASTIVE TRAINING FOR ADAPTATION
We first denote the few-shot training examples from target domain as Dr (X,)==I...... The parameters of a backbone network are denoted as O and z f(X; +) encodes the input X into feature space. Contrastive training updates the model parameters by optimizing a contrastive loss in unsupervised manner in Eq. [T] In this work, consider the BYOL (Grill al. 2020) method for contrastive learning due to its smaller memory requirements.
g(g().)) g(2.)
Lcon
(1)
NT
[T(o(2:)) (2.
X.ED
To learn effective representations, contrastive training contrasts between two random augmentations of the same input image, denoted as t(X). The encoder network outputs the representation em- bedding for each augmented input as z f((X); The representations are further projected to
3

---

Under review as a conference paper at ICLR 2023
t(X)
Regular Augmentation
NNP
(K))
f(X:0)
c (X)
Negative Augmentation
((X)
Lpp
f(x; 0)
Online Encoder
Xj
Zi
(
g(-)
f(x; 0)
Target Encoder
((X)
Lpp
Positive Pair Loss
f(x: 0)
2
g' (2,)
Lcon
NNP
Negative Pair Loss
x(X,)
X
Con
Contrastive Loss
Figure 1: Illustration of adapting source domain pretrained model through combining contrastive training loss (green arrow lines), cross-instance positive pair loss (red arrow lines) and negative pair loss (blue arrow lines) for few-shot anomaly detection. The dashed arrow lines indicate no gradient backpropagation.
lower dimension through a projection head g(.). The cosine similarity is then calculated between the predictor's output ((g())) on the online view and the projector's output g(2) on the target view. To avoid trivial solution, e.g. an encoder function giving constant outputs, the target view is the output of an exponential moving average model, i... 2 f(X; Q) and θ, = _ + (1 B) where a moving average hyperparameter, When a source domain model o is available, contrastive training on the target domain is initialized by the source domain model, i.e. Ou O such that the low-level feature extractors can be reused. Therefore, contrastive training serves as adapting a pretrained network weights to the few-shot target domain training samples. A discussion on why contrastive training helps can found in the Appendix A.
3.2 GROSS-INSTANE POSITIVE PAIR LOSS
The contrastive training objective encourages adaptation to target distribution, but this does not guar- antee the learned representation suitable for downstream density- -baed anomaly detection. Since anomaly detection inference is often implemented as fitting multi-variate Gaussian distribution on the normal samples in the feature space, normal samples should ideally be embedded close to each other. Inspired by the success of one-class classification (Ruff al., 2018)) propose to encourage normal samples to form tight cluster the feature space. Specifically, we treat a pair of randomly selected normal samples positive pair, the representations of each positive pair encouraged to be closer by minimizing the cosine similarity as Eq.2 list N...
where p is random permutation of the
LPP -N EE ((t(X.):)) ((t(X)); JEP [I
T 1I
2NT
Compared with the alternative of maintaining a fixed cluster center as proposed in (Ruff et al. 2018 the cross-instance positive pair loss has two advantages. First, we do not need to fix the cluster center at the start of training. This avoids introducing too much regularization on the representation embedding as the cluster center may vary during the course of training. Second, cosine similarity between the online view and target view where the latter does not backpropagate gradients. This avoids collapse trivial solution (e.g. all zero weights) (Ruff et al. 2018). We note that the cross-instance positive pair loss is calculated on the features directly from the backbone network. This due the fact that backbone output feature will be used for anomaly detection so the loss should be optimized in the feature space.
minimize the
3.3 INCORPORATING NEGATIVE PAIR LOSS
Synthesizing negative examples have been demonstrated to be successful in pretraining represen- tation for anomaly detection. cell-calibrated synthesis approaches even achieved the sfate-of-the- art performance certain datasets where the synthesized ones can match the real anomalies very
4

---

Under review as a conference paper at ICLR 2023
well (Li et al. 2021a). In this work, propose to incorporate additional synthetic negative amples when prior knowledge is available. Specifically, we denote synthesizing negative sample as (X), encourage better separation between normal and abnormal samples minimize the cosine similarity between the original image embedding and the negative embedding as below:
((t,())); 1 Σ (IS(((,)):) -I (I, (X.); +I CNP
NT
It is worth noting that the negative contrasting is also carried out directly on the backbone output features to reflect the constraints are applied to the feature representations. A relevant design was presented in (Ruff et al. 2019) for semi-supervised anomaly detection by minimizing the reciprocal of the distance between annotated anomalies and normal sample cluster center. Again, we believe minimizing the cosine similarity is compatible with the contrastive training objective and cross- instance positive pair loss with no risk of having trivial solution. The final training loss combines the above three loss terms as Call LCon + AppLpp ANPLNP.
3.4 DENSITY-BASER ANOMALY DETECTION
To perform anomaly detection using the learnt representations, follow the density-based ap- proach in (Li et al., 202 and fit a multivariate Gaussian distribution to the few-shot normal sam- ples. Note that the learnt feature representations must be 12-normalized before density estimation and inference because during representation learning, we optimize the cosine similarity which is agnostic to the magnitude of feature representations. Moreover, increase the amount of data for fitting the Gaussian distribution we produce NA times augmented samples from the few-shot nor- mal samples. Formally, the mean μ and covariance E is obtained through maximum likelihood estimation as below where DTA=
Dr U UDT U... DT.
NA times
Σ
1UXX FX
Σ X,EDTA
1)
w)
T
(4)
Dral
X,EDTA
X;
The anomaly score then given by the Mahalanobis distance as in Eq. 5 test samples are ranked by the anomaly score for anomaly detection.
das(X) (((X)//11XX)) E- (-/////1))
(5)
EXPERIMENTS
We evaluate the performance of our method on four industrial defect identification datasets and two datasets with synthetic common corruptions. We benchmarked against state-of-the-arr anomaly detection methods and achieved very competitive performance. Finally, we carry out ablation studies on individual components and provide further insights into the negative pair loss.
4.1 DATASETS
We provide overview of the datasets used in the experiments. MVTec Dataset (Bergmann et al., 2019b) contains 15 object categories, including 10 non-texture object categories and texture ob- ject categories. Each category contains 60-300 normal samples for training and 30-400 normal and defect samples for testing. We follow the few-shot settings from (Sheynin et al.] (02T) to create 2/5/10-shot anomaly detection protocols. AITEX Dataset Slivestre-Blance et al., 2019) is dedi- cated to detecting defects in textile fabric. This dataset consists of 140 normal sample images and 105 defect sample images with corresponding defect mask for localization. The original image res- olution is 4096x256 pixels and the defects occupy very small percentage of pixels. To allow for easier defect identification and mimicking a realistic defect identification procedure, we randomly crop out 5 patches of size 256 256 pixels from each image. Then 5/10/50-shot normal patches are randomly sampled as training examples. In total, test set consists of 600 normal patches and 105 defect patches. Magnetic Tile Defects Dataset (Huang et al. 2020) consists of 6 different types of defect magnetic tile surface images, namely "Blowhole", "Crack". "Fray", "Break", "Uneven", and "Free" (no defects). There 1344 images in total of which 952 images contain defects. We
5

---

Under review as a conference paper at ICLR 2023
randomly select 5/10/50-shot normal examples from the "Free" class training data and evaluate on the remaining images for anomaly detection. SemiCon Dataset (Pahwa et al. (022) features 3D images collected from 3D X-ray microscopy (XRM) scans integrated circuit packaging intercon- nects. The original dataset contains 53 3I) images of memory dies and the anomaly detection task is to identify voids in the solder region: large void solder ratio may indicate defective products. To create an anomaly detection benchmark, we slice the 3D images to obtain 2D images. 5/10/50-shot normal images randomly selected for training, 110 defect samples and 500 normal samples are used for testing. CIFAR10/1009 (Hendrycks & Dietterich, 2019) are adopted to simulate indus- trial defects through synthesizing common corruptions. We propose an anomaly detection protocol by treating all corrupted test samples as anomalies for each class. The dataset contains 5 levels of 15 different corruptions, e.g., noises, blurry, foggy. For easy comparison we choose types of level 2 corruptions. 5000 clean testing samples from each class of original CIFAR 10/100 dataset are used as normal testing samples, the corresponding corrupted samples of that class are treated as defect testing samples. We evaluate 10-shot for both datasets. A few examples of anomalies for the industrial datasets are presented Fig. with more examples to be found the Appendix 4..4 A..4 MVTec
SemiCon
AiTex
MagneticTile
Defect Normal
Figure 2: Examples industrial image data used for anomaly detection.
4.2 COMPETING METHODS
We compare against multiple anomaly detection methods in the experiments. We first benchmark the vanilla auto-encoder (AE) employed in (Bergmann et al.] 2019a) which is trained to recon- struct input images and the difference between input and reconstructs measures the anomaly score. VAE (Kingma & Welling 2014) imposes constraints on the latent variables to be Gaussian. Multiple samples drawn in the latent space and decoded to image space for measuring anomaly & Cho. 2015). DeepSVDD (Ruff et al., 2018)) trains the network by forcing normal training sam- ples embed close cluster center. The learned cluster center can later serve as the prototype for anomaly detection by measuring the distance as anomaly score. CutPaste (Li et al. 2021a) introduced CutPaste as an augmentation approach to synthesize negative examples for pretraining feature representation network. It is worth noticing that CutPaste mimics the real anomalies that would appear in MVTec dataset, therefore effectiveness CutPaste may not generalize to other types of anomalies. RotNet (Golan & Yanivy 2018) proposed a self-supervised pretrain- ing approach by predicting the augmentations (rotations) applied. This approach is most effective in anomaly detection natural semantic images and the advantage may disappear
(An)
industrial images where the pose background is naturally more diverse. CSI (Tack et al. 2020) proposed contrast distribution shifted augmented images with original images to increase the gap between normal and abnormal samples, This is similar to maintaining only the negative pair loss proposed in this work. CFLOW-AD (Gudovskiy al., 2022) adopted conditional normalizing flow model for fast anomaly localization. We adapt CFLOW-AD to training on few-shot anomaly detection task. TDG (Sheynin et al. (02T) proposed to employ generative model for few-shot anomaly detection by differentiating image patches into either fake or one of a list of predefined augmentations. DifferNet (Rudolph et al., (02T)) estimates density through normalizing flow with few supporting training samples. The above two approaches are compared on the MV'Tec dataset. Ours (w/o np) optimizes on the combination of contrastive loss and cross-instance positive pair loss. Assuming prior knowledge tive pair loss and optimizes on the combination all three losses. Among these methods, CutPaste and Ours (w/ np) are built upon the prior knowledge of anomalies while other methods do not make explicit assumptions on how the anomalies would look like.
the anomaly is available, Ours (w/ np) incorporates the nega-
4.3 EXPERIMENT DETAILS
Training Details: For all experiments, we use the ResNet18 (He et al. 2016) backbone for feature extraction. For all competing methods, initialize backbone weights with ImageNet pretrained

---

Under review as a conference paper at ICLR 2023
weights. We set the weight for cross-instance positive pair loss 0.8 and the weight for negative pair loss as 0.6. We use the Adam optimizer optimizerKimgma & Ba (2015) for all experiments with learning rate initialized to 3 x 10-4, 81 0.9 and 52 0.99. We fix the batch size to 64, thus creating 64 pairs for contrastive training. To generate cross-instance pairs, we randomly permute the 64 images and each of the 64 image is paired with a randomly permuted one, resulting in 64 positive pairs. Similarly, we pair each image to its negatively augmented one to create another 64 negative pairs. For density model fitting, we use NA- = 10. The area under the ROC (AUROC) is used to assess performance, where anomalies are treated as the positive class.
For the negative pair loss, we synthesize negative examples t. (X) with Cut- Paste augmentations (Li et al., (221aa) for MVTec datasets. For all industrial datasets, regular data augmentation t(X) comprises affine transformation and color manipulations (e.g. blurring and grayscaling). For CIFAR S0AAR10/10--C, regular augmentation only includes affine transformation and negative
Data Augmentation:
augmente augmentation comprises blurring and randomly perturbing image brightness and contrast.
4.4 FEW-SHOT ANOMALY DETECTION ON INDUSTRIAL DATASETS
In this section, we explore identifying defects industrial images. We first evaluate the few-shot anomaly detection performance on MVTTc dataset with results in Tab. [ We make the following observations. First, without any specific prior knowledge on the anomalies, our method (Ours (w/o np)) outperforms all competing methods by a clear margin. Furthermore, with prior knowledge on the potential anomalies. our method (Ours (w/ np)) still outperforms CutPaste with the same negative augmentations in the 2-shot and 5-shot settings. We are only slightly behind CutPaste in the 10- shot case. Both observations suggest the effectiveness of contrastive adaptation and cross-instance positive pair loss. We further observe that CutPaste exhibits significant lead on leather, wood and toothbrush images. We attribute this to the fact that these categories contain many anomalies that can be synthesized from CutPaste and scar augmentation: the "cut" defect for the "leather" category, the "scratch" " defect for "wood" and sear like defects for "toothbrush". In contrast, our method relies more on adaptation from the pretrained model and learning from few-shot normal samples. As result, its performance is generally better on more diverse types of objects: in the 2-shot setting, our method outperforms CutPaste on 8 of the 15 categories while CutPaste winning on 4/15 categories. Table 1: Few-shot anomaly detection on MV'Tec dataset. Per-category AUROC is reported for all competing methods. All numbers are in %. The results of DiffNet and TDG are derived from (Sheynin et al. 202T]. where - indicates per-class results are not available.
bottle
cable caps. hazel. metal.
pill
serew
tooth.
transis.
zipper
carpet
grid
leather
tile
wood
avg.
AE
73.49
64.22
62.43
73.54
35.97
75.19
35.56
73.33
48.92
40.73
22.11
45.13
31.52
73.35
58.42
53.26
shot
VAE
73.82
41.20
76.08
72.22
10.85
51.75
DeepSVDD
85.79
66.06
51.62
S3.39
50.44
77.27
51.23
69.72
58.04
39.30
70.05
38.10
40.42
81.02
51.23
60.28
Ours (w/o mpl
88.06
59.05
5888
60.77
68.51
65.55
56.48
73.63
72.84
70.74
76.00
61.18
63.11
68.25
77.85
68.06
CutPaste
64.59
73.59
49.92
66.91
62.04
55.53
59.97
52.26
46.25
84.79
65.35
Ours (w/ up)
90.95
66.55
59.38
68.82
68.96
66.68
54.90
74.81
74.33
72.82
73.18
61.77
65.28
69.61
79.85
69.86
AR
76.59
65.
72.92
73.64
49.61
76.73
40.32
75.00
64.79
59.95
38.76
42.61
43,72
66.56
74.04
61.39
VAE
96
72:92
56.16
63.05
shot
DeepSVDD
86.19
68.29
60.23
54.07
52.16
78.42
$2.15
81.39
69.04
74.80
51.54
51.55
58.20
82.83
93.51
67.59
CSI
80.55
60.48
62.07
74.40
59,83
69. 29
32.71
79.
56.71
63.71
56,17
39.53
51.56
55.38
75.96
61.12
CFLOWAD
98.17
81.65
73.12
88.25 25.25
74,88
68.22
45.09
82.50
84.79
83.53
73.48
50.96
87.84
91.59
92.37
78.43
DifferNet"
72.10
TDG
77.90
Ours (w/o mp)
94.58
70.67
74.06
78.29
79.66
81.56
62.70
87.18
78.08
75.03
84.67
62.31
74,90
78.34
89.66
78.11
CutPaste
89.90
72.24
62.85
73.13
49.94
78.33
Ours (w/ np)
95.73
78.05
74.26
86.89
78.57
81.31
62.26
88.81
74.80
75.70
79.90
61.85
77.99
75.34
90.12
78.76
AE
81,91
69.34
73.54
74.14
57.72
78-10
50.07
93.11
66.49
60.29
41.43
19.96
45.07
72.62
95.70
64.922
VAE
73.59
56.74
78.07
73.41
56.05
76.55
94.56
67.02
DeepSVDD
86.75
68
65.05
74.04
70.82
42.42
53.13
86. 39
69.08
77.28
$2.077
51.71
58.27
86.33
94.02
71.48
CSI
83.44
62.65
64.30
77.07
61.98
71.78
33.89
82.30
57.71
66.00
19.
40.95
53.41
57.37
78.69
63.32
CFLOW-AD
99.50
84.58
91.42
77.57
70.67
46.71
85.47
87.84
86.53
52.79
91.00
81.10
DifferNet"
73.60
TDG
78.00
Ours (w/o mpt
97.84
71.07
79.23
78.72
80.57
82.81
61.85
95.12
87.21
83.71
84.73
63.10
76.88
80.92
CutPaste
98.71
18
83.15
94.47
85.63
64.55
91.
70.01
86,90
83,92
55,
99.50
91.81
96.21
84.82
Ours (w/ mp)
99.03
83.49
78.38
87.25
79.17
82.55
63.92
94.71
87.21
84.45
79.95
63.14
78.88
$1.14
97.15
62.71
We further evaluate defect identification performance on another three industrial datasets, namely SemiCon, AITEX and MagneticTile Tile, with results in Tab. 2 The following observations are drawn from the results. First, without any prior knowledge, our model achieves the ofate-of-the-an art perfor- mance under lower budgets of available training samples (5 and 10 shots) on all three datasets. is only slightly behind DeepSVDD at 50-shot on AITEX. This suggests adapting pretrained models to
7

---

Under review as a conference paper at ICLR 2023
target domain is effective for realistic industrial image defect identification tasks. Second, methods demonstrating strong performance on MVTec dataset may not generalize to other types of defects. For example, while CutPaste is one of the best performing methods on MVTec, its performance on SemiCon and AITEX is much worse than more traditional approaches. One potential reason for this poor performance is that the synthesized negative samples used in CutPaste not representative of the defects in SemiCon and AITEX datasets; a more detailed analysis can be found in the Appendix. Table 2: Few-shot defect identification results on additional three industry image datasets. AUROC reported as evaluation metries. All numbers are (%).
SemiCon 10-shot
AITEX 10-shot
Magnetic Tile 10-shot
5-shot
50-shot
S-shot
50-shot
5-shot
50-shot
AE
65.38
70,24
71.74
47.01
60.55
63.30
51.58
52.66
54.70
VAE
72:42
73.53
80.14
52.99
66.36
67.88
51.90
53.40
54.30
DDD
$2,02
71.40
79.09
70.15
74.40
80.07
54,80
55.90
57.73
RotNet
55.13
75.55
80.45
71.20
75.19
55.69
57.14
60.00
CutPaste
48.71
71.60
75.03
50.23
69.58
79.83
57.82
61.00
62.14
Ours (w/o up)
78.87
80.72
82.00
73.44
77.45
78.14
58.23
61,93
63.55
4.5 DETECTING SYNTHETIC NOISE CORRUPTIONS
In this section, we evaluate on detecting synthetic noise corruptions as anomalies. The synthesized corruptions mimic noise patterns that are commonly seen in industrial environments. We bench- mark on CIFARIO-C and CIFAR100-C with 10-shot training samples for this purpose. In a similar fashion to MVTec, we adapt ImageNet pretrained weights to each individual semantic class. For CIFARION 100-C, we choose the 20 superclasses as the semantic class for simplicity. We present the anomaly detection results each semantic category of CIFAR10- 10-C in Tab. 3 the results that in average our methods, both with and without prior knowledge, lead the competing methods with clear margin. The closest competing method, CutPaste, is 4% lower than Ours (w/ np). This is in contrast to the extraordinary performance demonstrated on MVTec by CutPaste. We attribute this to the fact that the corruptions in CIFARIO-C are diverse and may not be easily synthesized by the augmentation methods specifically tailored for the MVTec dataset. We further benchmark on CIFAR 100-C and compare with DeepSVDD and CutPaste. We draw similar con- clusions from the results. Our method is still stronger than CutPaste with a clear margin. This is caused by the mismatch between the negative samples synthesized by CutPaste and corruptions in the dataset.
We first observe from
Table 3: 10-shot anomaly detection on CIFARIO-C dataset. All numbers are reported as AUROC in %.
Airpl.
Auto.
Bird
Cat
Deer
Dog
Frog
Horse
Ship
Truck
Avg.
AE
54.38
53.67
54.86
53.79
56.15
54.3
51,70
54.79
50.38
53.99
VAE
$7.47
52.17
56.32
56.11
58.67
57.09
59.18
54.06
$6.90
50.61
55.86
DeDDSVDD
63.27
61.64
60.75
60.71
61.54
62.11
63.00
59.83
63.72
67.29
62.37
RotNet
69.21
53.21
66.03
64.00
67.30
67.84
70.20
70.84
59.05
63.04
64.07
CutPaste
58.62
73.01
70.56
75.57
68.65
71.32
72.26
70.82
72.53
66.79
70.01
Ours (wdo mp)71.68
64.68
75.74
74.69
79.10
60.02
72.27
Ours (w/ up)
72.23
67.66
78.41
77.45
75.33
75.20
75.35
75.88
72.30
75.85
74.56
Table 4: Anomaly detection on CIFARI00-C dataset. Per super class performance is reported. All
numbers are reported as AUROC in %.
Sup.Cls. Sup.Cls.
Sup.Cls. 2 Sup.Cls. 3 Sup.Cls. 4 Sup.Cls.
Sup.Cls, 6
Sup.Cls. 7
Sup.Cls. 8 Sup Cls.
Deep SVDD
75.10
73.48
70.35
63.52
73.53
76.19
69.91
74.76
79.70
67.13
CutPaste
75.96
69.59
70.51
75.71
72.50
74.14
76,22
76.69
71.79
78.31
Ours (w/ mp)
77.61
66.69
77.84
76.21
79.93
78.17
73.33
78.80
74.63
81.17
Sup.Cls. Sup.Cls. Sup.Cls. 12 Sup.Cls. Sup.Cls. Sup.Cls. 15 Sup.Cls. Sup.Cls. 17 Sup.Cls. Sup.Cls.
Avg
70.00
73.17
60.82
56.91
66.36
65.81
60.60
69.88
70.52
71.30
69.24
80.28
75.83
77.92
78.56
70.87
78.84
72.83
71.63
72.97
75.61
74.84
81.27
75.59
80.11
79.98
75.56
80.04
73.65
77.54
77.16
78.13
77.17
4.6
ABLATION STUDY
In this section, we investigate the effectiveness of the individual components using the CIFARIO-C dataset. In particular, we demonstrate the importance of contrastive training on few-shot target do- main normal samples (Contrast. Train), incorporating tross-instant positive pair loss (Positive Pair Loss) and incorporating the negative pair loss (Negative Pair Loss). We further evaluate incorpo- rating 1.2 normalization (I.2 Norm) during anomaly detection density model fitting and inference.
8

---

Under review as a conference paper at ICLR 2023
We present the ablation study results in Tab. We make the following observations from the sults. First, as we expected, reusing ImageNet pretrained pretraiede weights for downstream anomaly detection yields significant improvement in performance (52.54% - 67. 67.32%) This suggests the significance of good representation for anomaly detection. Adapting pretrained model to the target distribution through contrastive training further improves 2% in average (67.32% - 70.05%). To encourage feature embedding suitable for density-based anomaly detection, we further incorporate the positive pair loss and this again yields additional 3% improvement (70.05% + 73.68%). As an alternative approach, one could encourage all normal samples' features to embed close to fixed cluster cen- ter (F.C.) following Ruff et al. (2018). However, as the cluster center must be fixed through the first forward pass, this could pose too much constraint on the representation learning and yield inferior results (68.21 1)). Finally, when we combine negative pair loss, this gives a final boost of perfor- mance to 74.56%. We also hypothesize that 1.2 normalization on feature representation is necessary and the ablation study validated the hypothesis. By removing the L2 normalization on anomaly inference features the performance drops from 74.56% to 72.11% indicating the normalization is essential to fitting better density model and distance anomaly detection.
Table 5: Ablation study on CIFARIO-C 10- FSAD. CIPP stands for cross insttacc positive pair.
Pretrained Weights Contrast. Train
Positive Pair Loss
Negative Pair Loss 1.2 Norm Avg. Acc.
$2,54
IetgeNet
67.32
ImageNet
70.05
ImageNet
V
F.C. Ruff et al. 2018
68.21
ImageNet
C.P.P.
73.68
ImageNet
C.I.P.P C.I.P.
74.56
ImageNet
72.11
4.6.1 WHEN DOES INCORPORATING NEGATIVE EXAMPLES HELP?
As we discussed, incorporating negative example during adaptation is not always beneficial. The advantage hinges on whether prior knowledge on abnormal sample distribution is available. To verify this point, we evaluate incorporating negative pair loss on 3 industry datasets, SemiCon, Al- TEX and Magnetic Tile. SemiCon dataset features anomalies that are quite different from MVTec, while AITEX and Magnetic Tile are relatively more similar to the texture categories of MVTec. We choose CutPaste (Li et al., (021a)) as negative augmentation for these 3 datasets. The results in Tab. 6 demonstrate that when the negative augmentation is substantially different from the real anomalies, e.g. the void circles in SemiCon dataset, incorporating negative pair loss with inappro- priate augmentation will harm performance (78.87 + 62. 62.97%). On the contrary, when anomalies can be simulated, even imperfectly, e.g. Magnetic Tile dataset, incorporating negative pair loss will further improve performance. Overall, we conclude that incorporating negative pair loss is only helpful when prior knowledge on the potential anomalies is concrete and anomalies can be sim- ulated through negative augmentation. Our model without negative pair loss is suitable for tasks without prior knowledge or when generating negative augmentation difficult.
Table 6: Effect of negative pair loss with CutPaste augmentation on industry image datasets.
SemiCon AITEX Magnetic Tile
Ours (w/o np)
78.87
58.23 65.54
73. 44
Ours (w/ np)
76.29
59.45
5 CONCLUSION
Industry defect inspection requires the capability of anomaly detection with very limited normal samples for training. To meet this demand, we proposed few-shot anomaly detection approach through adapting models pretrained on large external image collections to few-shot normal samples from the target task. We achieve this adaptation by optimizing contrastive loss and cross-instance positive pair loss. When prior knowledge on possible anomalies is available we further incorporate a negative pair loss to separate normal sample embeddings from the synthesized negative samples. We extensively evaluated the performance of the proposed method on four real industrial defect detection datasets and two synthetic datasets mimicking realistic corruptions. Our method achieved state-of-tt thetthe-art performance Finally, we show that the benefit of using synthetic negative samples is task-dependent and should only be considered when accurate prior knowledge is available.
all datasets when only handful of normal samples are available.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Jinwon An and Sungzoon Cho. Variational autoencoder based anomaly detection using reconstruc- tion probability. Special Lecture on IE, 2015.
Shin Ando and Ayaka Yamamoto. Anomaly detection via few-shot learning on normality. In Euro- pean Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2022.
P. Bergmann, S. Löwe, M. Fauser, D. Sattlegger, and C. Steger. Improving Unsupervised Defect Segmentation by Applying Structural Similarity to Autoencoders. In International Joint Confer- ence on Computer Vision, Imaging and Computer Graphics Theory and Applications, 2019a.
Paul Bergmann, Michael Fauser, David Sattlegger, and Carsten Steger. Mvtec ad-a comprehensive real-world dataset for unsupervised anomaly detection. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019b.
Varun Chandola, Arindam Banerjee, and Vipin Kumar. Anomaly detection: survey. ACM com- puting surveys, 2009.
Dian Chen, Dequan Wang, Trevor Darrell, and Sayna Ebrahimi. Contrastive test-time adaptation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. International conference on machine learning. 2020.
Izhak Golan and Ran Yaniv.. Deep anomaly detection using geometric transformations. In Ad- vances in neural information processing systems, 2018.
Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Daniel Guo, Mohammad Gheshlaghi Azar, al. Bootstrap your own latent: new approach sulf-supervised learning. In Advances in neural information processing systems, 2020.
Denis Gudovskiy, Shun Ishizaka, and Kazuki Kozuka. Cflow-ad: Real-time unsupervised anomaly detection with localization via conditional normalizing flows. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2022.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In IEEE Conference on Computer Vision and Pattern Recognition, 2016.
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In IEEE/CVF Conference Computer Vision and Pattern Recognition, 2020.
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common cor- ruptions and perturbations. arXiv preprint arXiv: /903.. 2019.
Chaoqin Huang, Haoyan Guan, Aofan Jiang, Ya Zhang. Michael Spratling, and Yan-Feng Wang. Registration based few-shot anomaly detection. In European Conference on Computer Vision, 2022.
Yibin Huang, Congying Qiu, and Kui Yuan. Surface defect saliency of magnetic tile. The Visual Computer, 2020.
Kim and C. D. Scott. Robust kernel density estimation. Journal of Machine Learning Research, 2012.
Diederik Kingma and Jimmy Ba. Adam: method for stochastic optimization. In International Conference on Learning Representations, 2015.
Diederik Kingma and Max Welling. Auto-encoding variational bayes. 2014.
10

---

Under review as a conference paper at ICLR 2023
Naveen Kodali, Jacob Abemethy, James Hays, and Zsolt Kira. On convergence and stability of gans. arXiv preprint arXiv: 1eX.iv:100.07700 00.11,, 2017.
Simon Kornblith, Jonathon Shlens, and Quoe V Le. Do better imagenet models transfer better? In IEEE/CVF conference on computer vision and pattern recognition, 2019.
Chun-Liang Li, Kihyuk Sohn, Jinsung Yoon, and Tomas Plister. Cutpaste: Self-supervised learning for anomaly detection and localization. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021a.
Suichan Li, Dongdong Chen, Yinpeng Chen, Lu Yuan, Lei Zhang, Qi Chu, Bin Liu, and Nenghai Yu. Unsupervised finetuning, arXiv preprint arXiv:2110.095/0 2021b.
Yuejiang Liu, Parth Kothari, Bastien van Delft, Baptiste Bellot-Gurlet, Taylor Mordan, and Alexan- dre Alahi. Tu++: When does self-supervised test-time training fail or thrive? In Advances in Neural Information Processing Systems, 2021.
Ramanpreet Singh Pahwa, Ma Tin Lay Nwe, Richard Chang, Oo Zaw Min, Wang Jie, Saisubra- maniam Gopalakrishnan, David Ho Soon Wee, Ren Qin, Vempati Srinivasa Rao, Haiwen Dai, et al. Automated attribute measurements of buried package features 3d x-ray images using deep learning. In IEFE 7 7ltt Electronic Components and Technology Conference (ECTC), 2021.
Guansong Pang, Choubo Ding, Chunhua Shen, and Anton van den Hengel. Explainable deep few- shot anomaly detection with deviation networks. arXiv preprint (...x:2208.00440 2021a.
Guansong Pang, Chunhua Shen, Longbing Cao, and Anton Van Den Hengel. Deep learning for anomaly detection: A review. ACM Computing Surveys, 2021b.
P. Perera, R. Nallapati, and B. Xiang. OCGAN: One-Class Novelty Detection Using GANs With Constrained Latent Representations, In IEEE International Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, 2019.
Marco Rudolph, Bastian Wandt, and Bodo Rosenhahn. Same same but differnet: Semi-supervised defect detection with normalizing flows. In IEEE/CVE Winter Conference on Applications of Computer Vision, 2021.
Lukas Ruff, Robert Vandermeulen, Nico Goernitz, Lucas Deecke, Shoaib Ahmed Siddiqui, Alexan- der Binder, Emmanuel Müller, and Marius Kloft. Deep one-class classification. In International conference on machine learning, 2018
Lukas Ruff, Robert A Vandermeulen, Nico Görnitz, Alexander Binder, Emmanuel Müller, Klaus- Robert Müller, and Marius Kloft. Deep supe-supe anomaly detection. In International Conference on Learning Representations, 2019.
Olga Russakovsky, Jia Deng. Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 2015.
T. Schlegl, P. Seebück. S. M. Waldstein, U. Schmidt- Schmidt-Erfurth, and G. Langs. Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. Proceedings of Inter- national Conference Information Processing in Medical Imaging, 2017.
B. Schölkopf, C. Platt, J. Shawe-Taylor, A. Smola, and R. C. Williamson. Estimating the support of a high-dimensional distribution. Neural Computation, 13(7):1443-1471, 2001,
Shelly Sheynin, Sagie Benaim, and Lior Wolf. A hierarchical tranffrraaiionddisciiiiin genera- tive model for few shot anomaly detection. In /EEE/CVF International Conference on Computer Vision, 2021.
HK Shivanand. Flexible manufacturing system. New Age International, 2006.
Javier Pirestrre-Blnnes Teresa Albero Albero, Ignacio Miralles, Rubén Pérez-Llorens, and Jorge Moreno. A public fabric database for defect detection methods and results. Autex Research Journal, 2019.
11

---

Under review as a conference paper at ICLR 2023
Kihyuk Sohn. Improved deep metric learning with multi-class n-pair loss objective. Advances in neural information processing systems, 2016.
Kihyuk Sohn, Chun-Liang Li, Jinsung Yoon, Minho Jin, and Tomas Pfister. Learning and evalu- ating representations for deep one-class classification. In International Conference on Learning Representations, 2021.
Jihoon Tack, Sangwoo Mo, Jongheon Jeong, and Jinwoo Shin. Csi: Novelty detection via contrastive learning distributionally shifted instances. Advances in neural information processing systems, 2020.
Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 2008.
Mei Wang and Weihong Deng. Deep visual domain adaptation: survey, Neurocomputiigg 2018.
Xun Xu, Jingyi Liao, Lile Cai, Manh Cuong Nguyen, Kangkang Lu, Wanyue Zhang, Yasin Yazici, and Chuan Sheng Foo. Revisiting pretraining for semi-supervised learning in the low-label regime. arXiv preprint arXiv:2205.03001, 2022.
A APPENDIX
A.1 DISCUSSIONS ON CONTRASTIVE TRAINING HELPING ADAPTATION
Recent works have demonstrated that contrastive training helps adapt model parameters to target domain distributions (Liu et al. 2021 Xu et al. 2022; Chen et al.) 2022 Li et al., 2021b). We argue that contrastive training on target domain data can alleviate the negative impact of covariate shift. For simplicity, we denote the source domain dataset as Dg, e.g. the ImageNet dataset, and the target domain dataset as Dr, c.g. anomaly detection dataset. The objective supervised training on the source domain can be seen minimizing the following cross entropy loss where h(.) is the classifier on source domain and f()) is the backbone network to be transferred.
arg min
(c)( Σ X.BLEDs
(6)
Ds
Covariate shift between source and target dataset indicates a distributional misalignment, i.e. Px( # (X) which is easily manifested by the difference in the contents of the source and target domain data. Therefore, it is reasonable to believe the backbone network optimized for source domain model is not optimal for the target domain distribution. To ease the negative impact caused by covariate shift, we introduce contrastive training on target domain by optimizing an unsupervised contrastive loss with model parameters initialized by the source domain ones, as Eq. 7
θ T arg min
LCon (c XiEDr
Σ
s.t. o = S+
|Dr
By minimizing the contrastive loss, the network is able capture key features from the target do- main to discriminate non-identical instances and we empirically demonstrate this to be effective for adapting source model to target domain for downstream anomaly detection. We further provide another perspective into the effectiveness of contrastive learning. When the augmentations are cho- sen to be mimic the commonly seen variations within normal samples, contrasting two augmented images forces the network to produce similar representations regardless of the augmentations. This means the representation learned from contrastive training allows the network to learn features variant to common variations in appearance and pose that one could encounter in industrial imaging environments. Such ability will help bring normal samples closer in the feature space, thus benefit downstream anomaly detection.
A.2 FURTHER ANALYSIS
In this section, we provide additional evaluations on qualitative examination of representation learn- ing and discuss the when incorporating negative samples should be employed.
12

---

Under review as a conference paper at ICLR 2023
A.2.1 QUALITATIVE EXAMINATION OF REPRESENTATION LEARNING:
The benefit of incorporating each of the proposed three losses for adaptation to target anomaly de- tection are validated through empirical experiments on multiple datasets. In this section, we provide qualitative observations into the advantage of incorporating these losses through t-SNE visualiza- tion of test data representations Van der Maaten & Hinton (2008). Specifically, we randomly select 1,500 testing samples from CIFARIO-C dataset for visualization. The feature points are projected into 2D space and visualized in Fig. 3 The feature embedding with ImageNet pretrained weights only, (a) w/o Adaptation, shows substantial overlap between normal and abnormal samples. When contrastive training is applied, (b) w/ Contrastive Loss, we observe a clear seperation between nor- mal and abnormal samples. When additional positive pair loss, (c) PP Loss, and negative pair loss, (d) w/ NP Loss, are incorporated, the normal samples are further grouped into tighter cluster with larger distrinction between normal and abnormal samples.
(a) w/o Adaptation
(b) w/ Contrastive Loss
(c) w/ PP Loss
(d) w/ NP Loss
Figure 3: T-SNE visualization of anomaly detection testing data on selected testing samples from CIFAR10-C dataset. Blue and red colors indicate normal and abnormal samples respectively.
A.3 COMPARING REAL ANOMALIES V.S. NEGATIVE AUGMENTATION
In this section, we provide visual examples for both real anomalies and negative augmentation ob- tained through CutPaste [i et al.] (2021a). We randomly sample 4 normal training samples from SemiCon dataset and augment the selected samples through CutPaste. The synthesized negative ex- amples are compared with 4 randomly selected real anomalies in Fig. 4 The qualitative examination indicate that CutPaste augmentation may not be suitable for all types of industrial images and this is reflected in the relatively poor performance of CutPaste and Ours (w/ np) on SemiCon and AITEX dataset.
A.4 ADDITIONAL VISUALIZATION OF INDUSTRIAL DATASETS
We provide more examples of the industrial datasets used in this work, namely the SemiCon, AITEX and Magnetic Tile. We illustrate both normal and defect samples of AITEX and Magnetic Tile in Fig. 5] SemiCon in Fig. 4 We notice that most defects in the SemiCon datasets are caused by the circular void in the solder area. The defects AITEX are often curvilinear and the background is dense with repetitive patterns. The defects for Magnetic Tile is more subtle.
A.5 DISTRIBUTION OF ANOMALY SCORES
In this section, we further compare different ablated models through visualizing the distribution of anomaly scores on CIFAR 10-C few-shot anomaly detection task. Specifically, we compare a) w/o contrastive finetuning: b) w/ contrastive finetuning; c) additionally w/ PP loss; and d) additionally w/ NP loss. The results in Fig. G We make the following observations. First, directly using the model pretrained on ImageNet, a) w/o contrastive finetuning, failed to differentiate normal and abnormal samples as the two distributions are almost the same. With contrastive finetuning on target data, b) w/ contrastive finetuning, we see a clear gap between normal and abnormal distributions, suggesting better anomaly detection performance. By further incorporating the positive pair loss, e) w/PP loss, the gap between two distributions are more significant. Finally, incorporating negative pair loss, d) w/ NP loss, is most effectiv for differentiating normal from abnormal samples.
13

---

Under review as a conference paper at ICLR 2023
Figure 4: Samples of applying CutPaste on industry dataset SemiCon. First row is the original samples, and second row is the corresponding generated sample from CutPaste augmentation. Third row shows few samples of the real defective images.
Figure 5: Illustration of Aitex(upper) and Magnetic Tile dataset. The first row each dataset are normal examples, second rows are defective examples.
14

---

Under review as a conference paper at ICLR 2023
100
Aesmaaies
80
80-
60-
40-
70.
10
20
25
30
25
20
50
60
Anoomaly score
Aromaayy Score
(a) w/o contrastive finetuning
(b) w/ contrastive finetuning
100
Nommh Aromaics
hormals fecmakes
50
hiqunta
60-
130
40-
200
23
29
30
40
50
60
70
90
10
20
50
Josomaly
Anamaty Store
(c) w/ PP loss
(d) w/ NP loss
Figure 6: Comparing different ablated models through anomaly score distributions.
15

---

Under review as a conference paper at ICLR 2023
A.6 EVALUATION ON ADDITIONAL BACKBONE
We evaluate the effectiveness of the proposed representation learning approach with stronger back- bone network. In specific, evaluate ResNet101 the SemiCon dataset in few-shot anomaly detection setting. The results Tab. reveal that contrastive finetuning and across instance positive pair are also effective with stronger backbone network.
Table 7: 5-shot anomaly detection with ResNet101 on SemiCon dataset.
Contrast. Train
Positive Pair Loss
Avg. Acc.
67.72
84.22
86.65
A.7 IMBALANCED ANOMALY DETECTION
We notice that the original anomaly detection dataset already highly imbalanced. For example, the AiTex dataset consists of 600 normal and 100 abnormal samples in the testing set. Since the ratio between normal and abnormal samples does not affect model training, we further implemented controlled experiment where manually decrease the number of anomalies in the SemiCon dataset. Specifically, we fix the normal samples to 500 and randomly subsample 5, 10, 20 and 50 abnormal samples for evaluation. The results in Tab. demonstrate the superiority of proposed method against existing competing methods.
Table 8: Generated by Spread-LaTeX
#Anomalies
10
deepsvill
curpaste
86.31
83.82
72.82
74.31
VAE
80.56
78.12
67.67
73.91
AF
62.12
65.52
76.83
74.39
Ours
88.36
86.50
85.87
80.72
16


Reviews:
[{"review_id": "oB2eUQAgqiw", "reviewer": "Reviewer_bC75", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper is focused on the few-shot anomaly detection problem. It presents a method that combines contrastive learning for adapting a pre-training model to the target domain, a cross-instance positive pair loss, an option to incorporate negative pair loss, and density-based anomaly detection. They show some promising experimental results for few-shot anomaly detection on several public datasets.", "summary_of_the_review": "The paper still has room for improvement in terms of paper presentation, technical novelty, and experimental justification. Please see the comments detailed in the previous sections. ", "strength_and_weaknesses": "Strengths: \n1. The proposed idea of applying contrastive learning for adaptation seems interesting. \n2. The experimental results on 4 different datasets are reported with promising results.\n\nWeaknesses:\n1. The paper is not well written. It is difficult to understand the main idea of the paper. How is the adaptation of a pre-trained model done with contrastive learning? Why doers the proposed training method provide good model adaptation? In addition, many important delaits for the implementation are missing. They didn't provide the network architecture of the proposed model.\n\n2. The novelty proposed in this paper is limited. All the componenets used in this paper are not new. The idea of extending the contrastive training to model adaptation is not clear. \n\n3. The notations used in describing the proposed method are not always defined or explained in the paper. For example, what are online encoder and target encoder in Figure 1 and eq. 2?\n\n4. The experimental comparisons used in this paper should include comparison with more recent SOTA methods, such as \nCFLOW-AD: Real-Time Unsupervised Anomaly Detection with Localization\nvia Conditional Normalizing Flows, WACV 2022\nRegistration based Few-Shot Anomaly Detection, ECCV 2022\n\n5. The experiment protocols should follow the previous experiments published in the previous papers, such as those in Registration based Few-Shot Anomaly Detection, ECCV 2022, for a fair comparison.\n\n6. For the denisity-based anomaly score for anomaly detection given in eq. (5), the covariance matrix could be signular if there is not enough samples in the convariance matrix estimation in eq. (4). There is no discussion on how to solve the problem.\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "njnu_hSan5", "reviewer": "Reviewer_gtEo", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 1, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes contrastive training and transfer learning applied to the problem of industrial defect identification via \"anomaly detection\". In particular, the negative pair loss is introduced as novelty in the paper.", "summary_of_the_review": "The paper is well written and has some clever ideas, but there is no significant insight on the application, no reasons or insights are provided on why the methods work in the anomaly detection setting. Also, there is not much novelty on the main proposed methods.", "strength_and_weaknesses": "Strenghts:\n- The approach shows good empirical results in 4 datasets\n- The ideas for tweaking the learning using across distance positive pair and the negative pair loss may be useful in some scenarios beyond anomaly detection applied to industrial defect \n- Many experiments are provided to investigate the behaviour of the method and the proposed ideas\n- Using CutPaste as a way to synthesize negative instances is a good insight.\n\nWeaknesses\n- All new ideas are somewhat already proposed before. Although the authors put together all those with success for the application at hand, that makes the paper less novel\n- The backbone may be deterrent to the final results. Maybe using a stronger backbone such as ViT-Dino could improve results of all baselines and diminish the amount of improvement claimed by the method.\n- In terms of the application the paper does not provide a lot of insight. That is, why the proposed approaches benefit the application? Why does the defects are easier to spot after the representations are learnt with constrastive learning? Why this application is a good downstream task?\n- The batchsize may play a significant role in the learning process, but this is not mentioned or studied in the paper.\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "crxGfRMu_t", "reviewer": "Reviewer_n46i", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper focuses on few-shot anomaly detection where only a few normal samples are available in training. The new method leverages contrastive learning to transfer pre-trained from the source domain to target domains supported by few-shot samples. In addition, the instance positive pair loss is used to tight up normal samples, while incorporating negative pair loss to separate the anomaly and normal samples (with prior and generated samples). Last, the multivariate Gaussian and density method are used for final detection.", "summary_of_the_review": "In brief, this paper presents an interesting few-shot AD method, which includes two steps: domain adaptation for pre-trained networks, and contrastive learning via building positive- and negative sample pairs. Evaluations are extensive and promising. However, the major concern is the novelty and its difference and comparison with the SOTA method such as CSI.", "strength_and_weaknesses": "Strength:\n* One of the major strengths of this paper is the extensive experiments on six benchmarks and convincing results.\n* Another strength is the idea of adapting pre-trained networks to target domains through contrastive learning under few-shot AD.\n\nWeakness:\n* It would be helpful to provide visualization after the outputs fit the multivariate Gaussian. This may further demonstrate if positive/negative pairs are grouped/separated as expected.\n* Some experiment details are missing, e.g., how many normal images were used in Sec. 4.5. \n3* Some competitive methods such as TDG (as mentioned in the paper) could be added to the experiments on cifar10-c and cifar100-c.\n* The state-of-the-art CSI [1] can run under few-shot setting and could be evaluated here, too. Note most key steps in this paper are very similar to CSI, so it is strongly recommended to compare with CSI in few-shot tests.\n* It should be noted that in Fig. 1 after negative augmentation, the image however did not change to an anomaly, as expected.\n\n[1] Jihoon, et al; CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instances. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "VViEKUfLYO", "reviewer": "Reviewer_LkYy", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper considers outlier detection in a few-shot setting with a small samples size in the target domain. \n\nThey propose to combine self-supervised learning with two loss terms, one loss term to encourage larger similarities between in domain samples, and a second loss term to encourage smaller similarities between in domain samples and simulated outliers.\n\nThey use a mahalanobis distance using normalized features to score for outlierness, without using a mean term to subtract from the features.\n\nThey go ahead to demonstrate results on four industrial defect datasets and provide an ablation study on the terms used.\n", "summary_of_the_review": "Effectively the paper shows the usefulness of self-supervised finetuning with an added positive pair loss.", "strength_and_weaknesses": "Strengths:\n*Easy to read\n*evaluation on industry datasets as opposed to simplistic standard image benchmarks\n*Mostly an improvement over Cutpaste, showing as insight limited usefulness of negative examples as created by CutPaste for certain real datasets.\n*Ablation study\n\nweaknesses: \n*straightforward extension with limited novelty\n*limited usefulness of the second proposed loss term\n*improvement over cutpaste on three datasets due to omission of the NP term, effectively demonstrating the gain due to the term encouraging clustering between positive samples.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject