ReContrast: Domain-Specific Anomaly Detection via Contrastive Reconstruction
Authors: Jia Guo
shuai lu
LIze JIa
Weihang Zhang
Huiqi Li
Venue: NeurIPS 2023 poster (2023)
Keywords: Unsupervised Anomaly Detection
Contrastive Learning
Medical Anomaly Detection
Transfer Learning
TL;DR: We combine the essence of contrastive learning and feature reconstruction for unsupervised anomaly detection, addressing the poor transfer ability of the frozen ImageNet encoder when applied to images with a large domain gap.

Abstract: Most advanced unsupervised anomaly detection (UAD) methods rely on modeling feature representations of frozen encoder networks pre-trained on large-scale datasets, e.g. ImageNet. However, the features extracted from the encoders that are borrowed from natural image domains coincide little with the features required in the target UAD domain, such as industrial inspection and medical imaging. In this paper, we propose a novel epistemic UAD method, namely ReContrast, which optimizes the entire network to reduce biases towards the pre-trained image domain and orients the network in the target domain. We start with a feature reconstruction approach that detects anomalies from errors. Essentially, the elements of contrastive learning are elegantly embedded in feature reconstruction to prevent the network from training instability, pattern collapse, and identical shortcut, while simultaneously optimizing both the encoder and decoder on the target domain. To demonstrate our transfer ability on various image domains, we conduct extensive experiments across two popular industrial defect detection benchmarks and three medical image UAD tasks, which shows our superiority over current state-of-the-art methods.

ReContrast: Domain-Specific Anomaly Detection via Contrastive Reconstruction
Jia Guo' Shuai Lu Lize Jia" Weihang Zhang? Huiqi L
School of Information and Electronics, Beijing Institute of Technology, China School of Medical Technoloy, Beijing Institute of Technology, China g jeremy@gmail.....m.. com, lushuaie0163. (3220212096, zhangueihang, huiqili)@bit. edu..tteddu.e. cn
Abstract
Most advanced unsupervised anomaly detection (UAD) methods rely model- ing feature representations of frozen encoder networks pre- ore-trained on large-scale datasets, c.g. ImageNet. However, the features extracted from the encoders that are borrowed from natural image domains coincide little with the features required in the target UAD domain, such industrial inspection and medical imaging. In this paper, we propose a novel epistemic UAD method, namely ReContrast, which optimizes the entire network to reduce biases towards the pre- pre-trained image domain and orients the network in the target domain. We start with a feature reconstruction approach that detects anomalies from errors. Essentially, the elements of con- trastive learning are elegantly embedded in feature reconstruction to prevent the network from training instability, pattern collapse, and identical shorteut, while simultaneously optimizing both the encoder and decoder on the target domain. To demonstrate our transfer ability on various image domains, we conduct extensive experiments across two popular industrial defect detection benchmarks and three medical image UAD tasks, which shows our superiority over current oaate-of-the--e methods. Code is available at: https://github.com//uu hot.a////./.oiiaareo//ooooootott
Introduction
Unsupervised anomaly detection (UAD) aims recognize and localize anomalies based the training set that contains only normal images. UAD has a wide range of applications, e.g., industrial defect detection and medical disease screening, addressing the difficulty of collection and labeling of all possible anomalies.
Efforts UAD attempt learn the distribution of available normal samples. Most state-of-the-ane utilize networks pre-trained on large-scale datasets, e.g. ImageNet I for extracting discriminative and informative feature representations. Feature reconstruction
and feature distillation methods [5) are proposed to reconstruct features of pre-trained encoders as alternatives to pixel g because learned features provide more informative representations than raw The above approaches can be categorized as epistemic methods, under the hypothesis that the networks trained normal images can only construct normal regions, but fail for unseen anomalous regions. Feature memory & modeling methods [1O: 12
(
reconstruction
pixels 2
3 9
B3] [4] memorize and model all anomaly-free features extracted from pre-trained networks in training, and compare them with test features during inference. In feature reconstruction. the parameters of encoders should be frozen to prevent the networks from converging to trivial solution, i.e., pattern collapse (2 3 In feature distillation and feature memory & modeling methods, there are no gradients for optimizing
*Corresponding Author
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

flow input
gradient backprop.
stop gradient
GAP
global average pooling
x' (enother view
Encoder
Encoder
pretrained Encoder
pretrained
pretrained
Encoder
CAP
(another view of
Decoder
Decoder
Decodar
Predictor
Predictor
regional
distance
representation
distancc
global distance
Contrastive
Learning
(b) Fenture Reconstruction UAD
(c) RcContrast (Ours)
(b) Feature reconstruction UAD 2 9 (c) Proposed ReContrast. Two decoders share the same weights. is the output of encoder. q is the output of predictor or decoder. and q' is another view of and q. respectively.
Figure 1: Comparison of architectures. (a) Contrastive learning 16 17
feature encoders. Unfortunately, poor transfer ability is caused by the semantic gap between natural images on which the frozen networks were pre-trained and various UAD image modalities.
Most recently, sparse studies have addressed the transfer problem UAD settings. CFA [4] and SimpleNet [15] unanimously utilize learnable linear layer adapt the output features of the encoder to task-oriented features, while still keeping the encoder frozen. However, the layer after frozen encoder may be insufficient for adaptation it can hardly recover the domain-specine information already lost during extraction, especially when the target domain is far from natural images, c.g. medical images. More related works presented in Appendix A.
In this work, we focus on orienting the whole UAD network in target domains. We propose Reconstructive Contrastive Learning (ReContrast) for domain-specific anomaly detection by op- timizing all parameters end-to-end. We start with epistemic UAD methods that detect anomalies by reconstruction errors between the encoder and the reconstruction network (decoder) 2 By inspecting the behavior of encoder, we find that direct optimization of the encoder causes diversity degradation of encoder features and thus harms the performance. Accordingly, we introduce three key elements of contrastive learning (Figure aa) into feature reconstruction UAD (Figure [()),, building 2-D feature map contrast paradigm train the encoder and decoder jointly on target images without obstacles. First, to mimic the role of global average pooling (GAP), a new optimization objective, namely global cosine distance, is proposed make the contrast in global manner to stabilize the optimization. Second, we bring in the essential operation, i.e., gropggradient, which prevents positive- -air contrastive learning [16] 17] from pattern collapse. Third, image augmentation can introduce potential anomalies, we propose to utilize two encoders in different domains to generate representations in two views from the same input image. In addition, we propose hard-mining strategy for optimizing normal regions that are hard to reconstruct, to magnify the difference between intrinsic reconstruction error and epistemic reconstruction error. is noted that ReContrast NOT self-supervised learning method (SSL) for pre-text pre- pre-training like SimSiam [6] and SimCLR (18) but an end-to-end anomaly detection approach.
To validate our transfer ability on various domains, we evaluate ReContrast with extensive experiments on two widely used industrial defect detection benchmarks, i.e., MVTec AD [9] and VisA 20). and three medical image benchmarks, including optical coherence tomography (OCT) 21 color fundus image [22], and skin lesion image (23). Without elaborately designing memory banks handcrafted pseudo anomalies, our proposed method achieves superior performance compared with previous state-of-the-arts. Notably, we present an excellent image-level AUROC of 99.5% MVTec AD, which reduces the error of the previous best by nearly half. The contributions of this study can be summarized as:
We propose a simple yet effective UAD method to handle the poor transfer ability of the frozen ImageNet encoder when applied to UAD images with a large domain gap. Targeting the similarity and difference between contrastive learning and feature reconstruction, three key elements are elegantly introduced into feature reconstruction UAD for optimizing the entire network without obstacles.
2

---

Inspired by the GAP in contrastive learning, we design new objective function, introducing globality into point-by-poim cosine distance for improving training stability.
We utilize the key operation of positive-pait contrastive learning, i.e., stop-gradient, to prevent the encoder from pattern collapse.
We propose a novel way of generating contrastive pairs to avoid potential anomalies intro- duced by image augmentation.
Method: From Reconstruction to ReContrast
In the following subsections, we present an exploration journey starting from a conventional feature reconstruction baseline our ReContrast. We propose three cohesive components inspired by the elements of contrastive representation learning and present them in a "motivation, connection, mechanism" manner. We believe such progressive exploration would illustrate the motivation of our method better.
cxcée Ie
rascaa cocac
Gcodde loyer
segional
maade bomn?
Confie A: RDHAD
Coofgg 1l A Lightnn
Confe.C # optimize encoder
dacoderbyy
escoder layer
dacode laver
dlobal coelee diannce
Neck
battlanesk
dola conte detance
globol
flow
coune
disoace
undie:: beokgrog.
Smap
Config C gndient
Confix D contrastive (RoContast)
Figure 2: From RD4AD to ReContrast. (a) Training configurations. (b) Calculation of anomaly map. The calculation and optimization of regional and global cosine distance is presented Figure
2.1 Preliminary: Reverse Distillation
First, we revisit simple yet powerful epistemic UAD method based feature reconstruction. i.e., Reverse Distillation (RD4AD) (2]. The method is illustrated Config.A of Figure 2 RD4AD consists of an encoder (teacher), bottleneck, and a reconstruction decoder (student). Without loss of generality, the first three layers of a standard 4-layer network are used as the encoder. The encoder extracts informative feature maps with different spatial sizes. The bottleneck is similar to the fourth layer of the encoder but takes the feature maps of the first three layers as the input. The decoder is the reverse of the encoder, i.c., the by up-sampling. During training, the decoder learns to reconstruct the output of the encoder layers by maximizing the similarity between feature maps. During inference, the decoder is expected to reconstruct normal regions of feature maps but fails for anomalous regions as has never seen such samples.
operation at the beginning of each layer replaced
Let fh, fb E -C"**x**" x denote the output feature maps from the th layer of the encoder and decoder respectively, where C+ H and W denote the number of channels, height, and width of the kth layer, respectively. Regional cosine distance (2 6 24) is used to measure the difference between f and fo point by point, obtaining a 2-D distance map M I ***

---

feature encoder
Seatire map from decoder
Eregion
cosine distance after GAP
Batley
global
cosine distance
flatten
regional
envine distance
global
Egtobat- km
(b)
Figure 3: Cosine distances. (a) Calculation of regional and global cosine distance. (b) Optimization of distance losses. The black arrows denote optimization direction. The black and red dotted circles indicate the range whether the feature point pair close enough to be selected as easy-normal.
S (h. w)²
$B (h, w)
M (h, w) =
1 (h, w)ll S (h, w)ll
where ||-|| is f2 norm, and a set of (h, w) locates a spatial point in the feature map. The calculation is depicted in Figure 3 a). large value in M indicates anomaly at the corresponding position. Final anomaly score map Smpp is obtained by up-sampling all M to image size and sum operation. The maximum value of Smap is taken as the anomaly score for the image, denoted as Simg During training, the anomaly maps minimized normal images by the regional cosine distance loss, given as:
W
SB (h, w)) 1' (h,, w)
Eregion
H
f (h, w)ll | (h, w)ll
h=1
The optimization of Cregion is illustrated in the lefer-left of Figure 3b)) Notably, though Lregion was proposed clearly as the loss function of RD4AD 2 the official code implemented a different function that makes extra dimension flatten operation (maybe by coding bug). However, observe extreme instability during the training with Cregions producing degenerated I-AUROC of 95.3% MV'Tec AD compared with 98.5% reported in the paper. The I-AUROC of 95.3% is more consistent with other methods using Lregion [6] 24). In the next subsection, we will show that the seemingly unreasonable "bug" in the code improves the performance by explicitly introducing globality into optimization.
2.2 Global Cosine Similarity
is a convention to use global average pooling (GAP) to pool the 2-D feature map of the last network layer 1-D representation, in both supervised classification and self-supervised contrastive learning. In UAD, GAP will simply mess up all feature points together, losing the ability to distinguish normal and anomalous regions, as shown in the upper-right of Figure 3b.. Therefore, try directly bring globality into the loss function while maintaining the point-to-point correspondence between f and fk. Let F denote flatten operation that casts 2-D feature map f CCX a vector E CRCWW Our proposed global cosine distance loss is given
3
F(A)) F(F))
Lyiobal
(3)
(F II [FB)

---

where the number of cosine dimension is C- H W instead of C' in Cregion, as depicted in Figure B (a). Intuitively, .M is also minimized along with the minimization of Globbat. It is easy to prove that minimizing globbal to 0 equals to minimizing M to O. Though losses are not completely optimized in neural network training, the regional Smap still works the anomaly map. The model trained with Lglobal is named as Config.B.
As shown in Figure 5 a), the performance of Config.B B (blue) is much more favorable and stable than Config.A (purple) during training. Here, we briefly analyze the underlying mechanism by plotting the landscape of Smop (average of all samples) against model parameters near optimized minima in Figure 4 We observe two differences. First, the overall landscape of the model optimized by Lytobal flatter than that by Lregion. which implies Smap is more stable during training with Colobat. The instability of Cregion can be caused by the point-by-point distance, where the fitting of one region may cause under-fitting of the others. Lglobal can be regarded as the distance between manifolds of feature points, as depicted in the lower-left of Figure 3(b), which measures the consistency globally instead of excessively focusing on individual regions. Second, the landscape around the final minima of the model trained by giosbal is sharper than that by Lregion: Previous explorations on the geometry
of loss landscapes suggest that flat minima generalize better on unseen samples (25 26 which
unwanted in UAD settings. The real mechanism can be more complicated and is still under further investigation.
sharper
flatter
0mmo alEeAA
2.
drau ARRiDas
naiter
sharper
60
paramatte directions
parameter direetion
parameter dirsction
paramater direction x
Landscape of average 8 gmap
Landscape of average map optimized by Lgiobat
(b)
optimized by Fregion
Figure 4: Smap landscape in parameter space (reduced to 2-dimension), on APTOS 22 visualized
using (27
2.3 Feature Degradation of Optimized Encoder
seems easy to adapt the network in the target domain by jointly optimizing the encoder and decoder, as illustrated in Figure 2 Config.C. C. However, previous attempts [9] suggest that training encoders would lead encoders to extract indiscriminative features that are easy to reconstruct, which is also described pattern collapsing converging to trivial solutions [2]. We observe similar phenomenon that the training loss quickly decreases to nearly zero, as shown by the orange dotted line in Figure 5(a).
To inspect the behavior of the encoder, we probe the diversity of encoder feature maps by the standard deviation (std) of &g normalized feature points få (h, w)/ [[f (h. w) w)]l The larger the std. the more discriminative the feature, and the more distinctive between different regions. We plot the feature diversity of outputs of the 2nd encoder layer during training in Figure (b. The sid of Config. C drops quickly during training, while the sid of Config.B B keeps constant as a comparison. In addition, the sid of Config. C does not completely collapse to zero, which explains the passable performance. The outputs of other middle layers follow the same trend. The observation indicates that the degeneration of performance is attributed to the degradation of encoder features. It is noted that the std starts from different values because of the different modes of batch normalization (BN) (8 during training
2 Enccddr BN is in eval mode in Config.A using pre-trained running mean and variation, and it is in train mode using batch mean and variation if optimized.
5

---

anomalous image
S with
Conffig A
Config.
Couffi. C
Config
Comfig
Liiobai-nm
1.00
0.90
AUROC
Loss [w-r))
Config A & (overlap)
smop with Cigthhat
casy-normal
hard-normal
unomaly
0.50
Iterations
Iterationn
(u) I-AUROC & loss
(b) sid of
(c) visualization of mapp
Figure 5: (a) I-AUROC and loss during training on APTOS(22). (b) Feature diversity during training on APTOS (c) Example Smap of Hazelnut, optimized by Lyiobal Lglobal-hme
2.4 Stop Gradient
Contrastive learning for SSL maximizes the similarity between the representations of two augmenta- tions (views) of one image while trying avoid collapsing solutions in which all inputs are encoded to constant value. SimSiam [16] indicated that the stop-gradient operation plays an essential role in preventing collapsing for SSL methods with positive pairs only [16 17]. Intriguingly, we found that contrastive SSL can be explained from the perspective of feature reconstruction. The predictor (a multi-layer perceptron, MLP) can be regarded as a reconstruction network that constructs the representation of one view match the representation of another view x', as in Figure [ Vice versa, feature reconstruction UAD can be transformed into contrastive learning as well, regarding the reconstruction network as predictor.
To this end, we introduce the stop-gradient operation into feature reconstruction, transforming into a contrastive-ikke paradigm. The network is optimized by contrasting the feature maps of the encoder and decoder, shown in Config.D of Figure 2 The gradients do not propagate directly into the encoder, but back into the encoder through the decoder. It is implemented by modifying (3) as:
3
sg (F (JK))* -F (JB)
Lylobal
Σ
8g F (SE) [F(
where sg denotes the stop- noo-ggradient operation. The training of this configuration can be intuitively explained "mutual reinforcement". The optimization of ff drives the encoder to be more specific and informative in the target domain by the gradients from the decoder, while the more domain- specific f of encoder requires further optimization of fB- There are previous works trying explain the deeper mechanism of stop gradient contrastive learning. In [16]. the authors hypothesize this implementation of Expectation- Maximmiaiiaaiiiiiiiininn (EM) algorithm that implicitly solves two underlying sub-problems with two sets of variables. In (29), the authors argue that there are flaws in the hypothesis of [16] and suggest that the decomposed gradient vector (center gradient) helps prevent collapse via the de-centering effect.
configuration
As shown in Figure 5]a) and (b), Config. can constantly extract more discriminative features without degeneration, boosting to perform better than Config. C at the beginning of training. Previous works [ 3 suggested that the reconstruction decoder may learn an "identical shorteut" that well reconstructs both seen and unseen samples, which explains the performance drop of Config. after reaching the peak in Figure 5).). To handle this issue, contrastive pairs are introduced.
2.5 Contrastive Pairs
Contrastive learning methods make use of (positive) contrastive image pairs, based on the intuition that the semantic information of different augmented views of the same image should be the same. Without image augmentations (z=x'), contrastive learning degrades into a selfsreconstruuctonn paradigm that the predictor predicts the input of itself, which may also collapse into an "identical shortcut" just like Config. D. In epistemic UAD methods, it is impossible to construct contrastive pairs by image augmentation, because 1) any augmentation (flip, cutout, color jitter, etc.) could be potential

---

anomalies; and 2) the feature maps of two augmented views lost the regional correspondence after spatial augmentations (shift, rotate, scale, etc.). Therefore, we propose an augmentation method to construct two views of one image.
In previous explorations, the encoder is adapted the target image domain and generates feature representations in a domain-specific view. We additionally introduce a frozen encoder network without any adaptation throughout the training. which perceives images in the view of pre-train image domain. As shown in Config. E of Figure the decoder and bottleneck take the features of the domain- sppecffcc encoder to reconstruct the frozen encoder, vice versa, they simultaneously take encoder, building a complete
the features of the frozen encoder to reconstruct the domain- two-view contrastive paradigm. This configuration can also be interpreted as cross- cross-reconstrue reconstruction, as the decoder reconstructs the information of the frozen encoder given the input of the adapted encoder; and reconstructs the information of the adapted encoder given the input of the frozen encoder. With six pairs of feature maps, the Smep is obtained by adding up the six up-sampled cosine distance maps. As shown in Figure (bb), the encoder feature diversity of Config. E not only holds but slightly increases during training.
2.6 Magnify Epistemic Error by Hard-Normal Mining
Due network information capacity, intrinsic reconstruction errors exist in available normal regions. The errors of normal details and edges (hard-normal) are generally higher than other plain regions (casy-normal), leading to confusion between the intrinsic error of hard-normal regions and the epistemic еπrоr of unseen anomalies, as demonstrated in the lower-left of Figure (lc). ) It was proposed to hard-mine the hard-normal regions by simply discarding the "easy" pairs of J (h, w). fh (h, w) with small cosine distance in Lregion 244] However, arbitrarily discarding feature points in 1b breaks the global feature point manifolds in Cylobal. Therefore, we propose a hard mining strategy, namely Cylobul-hm- focusing on the optimization of the epistemic errors caused by unseen anomalies and the errors of all normal regions. The gradients of easy I& (h, uz) are discarded instead of eliminating f (h, w) itself, by modifying S3 as:
regions to widen the gap between
Sb w) = sg (fb (h, w)) if M (h, w) (M* (h, w)) + a (M* w))
(5)
1b (h, (), else
where u (M* (h, w)) is the average regional distance on the dataset (approximated on mini- biini, (M* (h, w)) is the standard deviation of the regional distance, and is haper-yprrmeeee parameter control the discarding rate. The optimization of Lytohai-hm hm is depicted in the lower-right of Figure (b).. As shown in Figure the model trained by Lglobal-km activates fewer normal regions than the model trained by Lglobal- The usage of hard mining strategy can also mitigate the over-fitting of easy training samples.
3 Experiments
In this section, we evaluate the proposed method on two popular UAD applications, i.e., in- dustrial defect inspection and medical disease screening. We refer to complete approach (ReContrast++ggioall-a as ReContrast and Ours in the Experiments section for simplicity.
3.1 Experimental Settings
Datasets. MVTTc AD is the most widely used industrial defect detection dataset, containing 15 categories of sub-datasets (5 textures and 10 objects). VisA is a challenging industrial defect detection dataset, containing 12 categories of sub-datasets. OCT2017 is an optical coherence tomography dataset (2I). APTOS is a color fundus image dataset, available as the training set of the 2019 APTOS blindness detection challenge 22 ISIC2018 is a skin discase dataset, available as task 3 of ISIC2018 challenge 23 Detailed information is presented in Appendix B.
Implementation. WideResNet50 30 pre-trained on ImageNet [1] is utilized as the encoder by default. AdamW optimizer (31 is utilized with 3=(0.9,0.999) and weight decay=le-5. The learning rates of new (decoder and bottleneck) and pre-trained (encoder) parameters are 2e-3 and le-5, respectively. The network is trained for 3,000 iterations on VisA, 2,000 on MVTec AD and ISIC2018, and 1,000 on APTOS and OCT2017. The in equation (5) linearly rises from -3 to I in the first
7

---

Table 1: Anomaly detection performance on MVTec AD, measured in I-AUROC (%).
ADTR 3
RD4AD
CFlowAD [4]
CutPaste 35
DRAEM
PaDiM
PatchCore [1]
Ours
2
36
10
Carpet
98.7
98.9
98.7
93.9
97.0
99.8
98.7
99.8
Grid
95.0
100
99.6
100
99.9
96.7
98.2
100
Leather
98.1
100
100
100
100
100
100
100
Tile
93.8
99.3
99.6
98.1
98.7
99.8
Wood
91.2
99.2
99.1
99.1
99.1
99.2
99.2
99.0
Text. Avg.
95.4
99.5
99.5
97.3
99.1
98.9
99.0
99.7
Bottle
98.0
100
99.2
99.9
100
100
Cable
96.8
95
97.6
81.2
91.8
92.7
99.5
99.8
Capsule
99.1
96.3
97.7
98.2
98.5
91.3
98.1
97.7
Hazelnut
98.6
99.9
100
98.3
100
92.0
100
100
MetalNut
97.0
100
98.7
98.7
100
100
Pill
98.3
96.6
94.9
98.9
93.3
96.6
98.6
Screw
99.3
97.0
91.9
88.7
93.9
85.8
98.1
98.0
Toothbrush
98.5
99.5
100
99.4
100
96.1
100
100
Transistor
97.9
96.7
95.2
96.1
93.1
97.4
100
99.7
Zipper
97.2
98.5
98.5
100
90.3
99.4
99.5
Obj. Avg.
98.1
98.0
97.7
95.5
97.4
93.8
99.2
99.3
All Avg.
97.2
98.5
98.3
96.2
98.0
95.4
99.1
99.5
Table 2: Anomaly segmentation performance on MV'Te AD, measured in P-AUROC/AUPAR (%)
RD4AD 2
DRAEM 36
SPADE 37
PaDiM 10
PatchCore II
Ours
Text. Avg.
97.7 / 95,00
97.5 / 92.1
92.9 / 88.4
97.5 93,6
98.0/ 96.2
Obj. Avg.
97.9 93.4
98.3 / 93.3
97.6 / 93.4
97.8 99.6
98.4 93.3
98.6 94,8
All Avg.
97.8/
97.8
96.0
97.5 92.1
98.1 93.4
98.4 95.2
Table Anomaly detection and segmentation performance on VisA (%).
RD4AD 2
DRAEM 36
SPADE 37
PaDiM 1T
PatchCore
Ours
I-AUROC Avg.
96.0
88.7
92.1
89.1 98.1
95.1
97.5
P-AUROC Avg.
90.1
93.5
85.6
98.8
98.2
AUPRO Avg.
70.9
72.4
65.9
85.9
91.2
92.6
one-tenth iterations and keeps 1 for the rest training. More experimental details and environments presented in Appendix C.
Metrics. Image-level anomaly detection performance is measured by the Area Under the Receiver Operator Curve (I-AUROC). -c-se (F1) and accuracy (ACC) are adopted for medical datasets following (32) The operating threshold of Fl and ACC is determined by the optimal value of F1. For anomaly segmentation, pixel-level AUROC (P-AUROC) and Area Under the Per-Region-Overlan (AUPRO) 3 are used as the evaluation metrics. AUPRO is more meaningful than P-AUROC because of the unbalanced amount of normal and anomalous pixels 20
3.2 Anomaly Detection and Segmentation on Industrial Images
Anomaly detection results on MVTec AD are shown in Table I Our approach achieves a superior average I-AUROC 99.5%. In the aspect of error -IAUROC) we achieve only 0.5%, reducing the previous SOTA eтror of PatchCore (0.9% by relatively 44%. Anomaly segmentation results are shown Table (average of categories). Our ReContrast achieves SOTA performance measured by both P-AUROC and AUPRO. In AUPRO, we produce 95.2%, exceeding the previous SOTA RD4AD by 1.3%. We also present SOTA performances on texture (Text. Avg.) and object (Obj. Avg.) categories, respectively.

---

Table 4: Anomaly detection performances under unified multi-class setting on MVTec AD and VisA,
measured in 1-).
MVTèc AD
RD4AD
UniAD 9
Ours
VisA
RD4AD
UniAD 9
Ours
Carpet
98.3
99.8
98.3
candle
93.6
94.6
96.3
Grid
99.0
98.2
98.9
capsules
67.8
74.3
77.7
Leather
100
cashew
94.5
Tile
98.1
99.3
99.5
chewingum
95.3
98.7
98.6
Wood
99.3
98.6
99.7
fryum
94.4
90.2
97.3
Bottle
79.9
99.7
100
macaronil
97.2
91.5
97.6
Cable
86.8
95.2
95.6
macaroni2
86.2
83.6
89.5
Capsule
86.9
97.3
pebl
96.5
Hazelnut
100
99.8
100
peb2
93.0
92.5
96,8
MetalNut
99.8
99.2
100
pcb3
94.5
89.8
96.8
Pill
92.8
93.7
96.3
pcb4
100
99.3
99.9
Screw
96.5
87.5
97.2
pip_fryum
99.7
97.3
99.3
Toothbrush
97.5
94.2
96.7
Transistor
93.3
99.8
94.5
Zipper
98.6
95.8
99.4
All Avg.
95.8
96.5
98.2
All Avg.
92.7
91.5
95.1
Anomaly detection and segmentation results on VisA are shown in Table (average of 12 categories). Our approach achieves superior average I-AUROC 97.5%, exceeding previous SOTA RD4AD by 1.5%. In AUPRO, produce 92.6%, outperforming previous SOTA PatchCore by 1.4%.
3.3 Multi-Class Anomaly Detection with A Unified Model
Conventional UAD settings train separate models for different object categories. In UniAD 9 the authors proposed to train a unified model for multiple classes. We evaluate our ReContrast under the unified setting on MVTec AD and VisA. Our model trained for 5,000 iterations on all images containing all 15 MVTec AD categories (or 12 categories in VisA) and evaluated on each category following g The results are presented in Table On MVTee AD, our method produces an I-AUROC of 98.2% exceeding the previous SOTA 96.5% of UniAD by large 1.7%. On VisA, both
our ReContrast (95.1%) %) and bascline RD4AD (92.7%) outperform the previous multi-class SOTA UniAD (91.5%), while the proposed method exceeds RD4AD by 2.4%.
3.4 Anomaly Detection on Medical Images
The experimental results on three medical image datasets are presented in Table 5
Apart from industrial UAD methods, we also compare our ReContrast with approaches designed for medical images (f-AnoGan 77). GANomaly [8), Alollow (32), and contemporaneous methods proposed for better domain adaption ability (CFA [14), SimpleNet [55 We reproduce the methods that were not evaluated on the corresponding dataset if they are open-sourced. Our method achieves the best results in all metrics on APTOS and ISIC2018 8, and comparable best results on the relatively easy OCT2017, demonstrating that ReContrast can directly adapt to various medical image modalities.
3.5 Ablation Study
To verify the effect of the proposed elements, we conduct thorough ablation studies on MVTec AD and APTOS. Because some configurations are unstable during training on some categories, we report the last iteration's 's performances and those of the best checkpoint during training (best by I-AUROC, evaluate every 250 iterations). The results are reported in Table The use of Lytobul (Config. B) boosts the performance of Config-A trained by Lregion- Directly optimizing the encoder and decoder together (Config. C) causes great degeneration because of pattern collapse, while either stop- goadggrd (Config.D) contrastive pairs (Config. C++pp) recovers some of it. By introducing both (Config.E) performance is further improved to exceed frozen encoder baseline. Finally, training with Leglobal-hh hm yields SOTA performances (Ours). Meanwhile, Lylohai-han can also improve the baseline reconstruction method alone (Config. B+hm)). More ablation studies, experimental results with error bars, limitations, and qualitative visualization presented in Appendix.

---

Table 5: Anomaly detection performances on medical image datasets (%).
APTOS
OCT2017
ISIC2018
F-UROC
F1
ACC
T-AUROC
ACC
F-ARRO
F1
ACC
E-AnoGAND
89.64
89.71
85.23
83.02
88.64
82.13
79.80
67.03
68.39
GANomaly 8
83.72
85.85
79.37
90.52
91.09
86.16
72.54
60.95
56.99
77.17
85.47
77.24
96.88
95.24
92.80
82.13
72.04
73.06
RD4AD
92.43
90.65
86.44
99.25
97.79
96.70
85.09
74.53
78.76
PatchCore
90.45
90.18
85.57
99.61
98.34
97.50
78.94
68.57
71.50
AE-flow
N/A
N/A
N/A
98.15
96.36
94.42
87.79
80.56
84.97
CFA
94.21
94.39
92.03
98.01
96,40
94.70
81.31
72.31
74.61
SimpleNet 15
93.42
91.16
87.27
98.50
96,91
95.40
82.17
69.82
73.59
Ours
97.51
95.27
93.35
99.60
98.53
97.80
90.15
81.12
86.01
Table 6: Ablation study on MVTec AD and APTOS. Reported in last/best
optim. encoder
stop grad.
contrast pairs
hard mining
MVTec AD
APTOS
Gglobal
T-AUROC
AUPRO
T-AUROC
Config.A
1/935
93.34/94.05
90.12/y0.5
Config.
98. 98.86/99.07
94.51/94.59
92.49/93.62
Config. B+hhm
99 00.9909991
94,86/94.91
93.87/94.36
Config.C C
91.54/95.96
88.24/92.14
90.71/91.00
Config.D
94 64/91.07
84. 1/87,388
93.06/95.0 66
Config. C-
97.59/97.88
93.76/93.92
92.68/92.68
Config.E. E E-gl gl
98. 98.93/99.26
94.65/94.71
96.39/96.39
Eonfig.E
99. 13/99.99.3
94.59/99.60
97.32/97.43
Ours
99.45/99.52
95.20/95.29
97.51/97.51
4 Conclusion
In this study, we propose a novel contrastive learning paradigm, namely ReContrast, for domain- specific unsupervised anomaly detection. It addresses the transfer ability of the pre-trained encoder by jointly optimizing all parameters end-to-end. The key elements of contrastive learning are elegantly embedded in epistemic UAD method avoid pattern collapse, training instability, and identical shortcut. Extensive experiments on MVTec AD, VisA, and three medical image datasets demonstrate our superiority. The idea of optimizing encoders can further boost the application of UAD methods on more image modalities that are far from natural image domain.
5 Limitations
Scope of Application In this work, we mainly focus on UAD that detects regional defects (most common in practical applications like industrial inspection and medical discase screening). which is distinguished from one-class classification (OCC, or Semantic AD). In our UAD, normal and anomalous samples are semantically the same objects except local detects, e.g. good cable v.s. spoiled cable. In OCC, normal samples and anomalous samples semantically different, e.g. cat v.s. other animals. More details are discussed Appendix E.
Training Instability Our method still suffers some extent of training instability. Because of the absence of validation sets in UAD settings, whether last epoch (for reporting results) is in middle of loss spike and performance dip is related to random seeds. We found that number of UAD methods (RD4AD[), CFA [4, SimpleNet(15) are also subject to training instability when running their code. We discuss the reason and solutions to mitigate this effeett in Appendix E.
10

---

Acknowledgments and Disclosure of Funding
This research work is supported by the National Natural Science Foundation of China (NSFC) (GrantNo. 82072007), Beijing Natural Science Foundation (Grant No. IS23112), and Beijing Institute of Technology Research Fund Program for Young Scholars.
References
[1] O. Russakovsky al., "ImageNet Large Scale Visual Recognition Challenge," International Journal of Computer Vision, vol. 115, no. 3, 2015.
[2] H. Deng and X. Li, "Anomaly Detection via Reverse Distillation from One-Clase Embedding."" in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9737-9746 2022.
[3] Z. You, K. Yang, W. Luo, L. Cui, Y. Zheng, and Le, "ADTR: Anomaly Detection Transformer with Feature Reconstruction," " arXiv preprint wxXiv2229.008// 2022.
[4] Y. Shi, J. Yang, and Z. Qi. "Unsupervised anomaly segmentation via deep feature reconstruction," Neurocomputing, vol. 424, 2021.
[5] M. Salehi, N. Sadjadi, S. Baselizadeh, M. H. Rohban, and H. R. Rabiee, "Multiresolution knowledge distillation for anomaly detection," in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2021.
[6] G. Wang. Han, E. Ding, and D. Huang, "Student-teacher feature pyramid matching for anomaly detection," in The British Machine Vision Conference (BMVC), 2021.
[7] T. Schlegl, P. Seeböck, S. M. Waldstein, G. Langs, and U. Schmidt-Erfurth, "f-AnoGAN: Fast unsupervised anomaly detection with generative adversarial networks," Medical Image Analysis, vol. 54, 2019.
[8] S. Akcay, A. Htapor-Abarghooie and T. P. Breckon, "GANomaly: Semi-supervised Anomaly Detection via Adversarial Training," in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 1 11363 LNCS, 2019.
[9] Z. You el al., "A unified model for multi-class anomaly detection," in Advances in Neural Information Processing Systems (S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, eds.), vol. 35, pp. 4571-4584, Curran Associates, Inc., 2022.
[10] T. Defard, A. Setkov, A. Loesch, and R. Audigier, "PaDiM: A Patch Distribution Modeling Framework for Anomaly Detection and Localization," in Lecture Notes Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12664 LNCS, 2021.
[11] K. Roth, L. Pemula, J. Zepeda, B. Schölkopf, Brox, and Gehler, "Towards total recall in industrial anomaly detection," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 181228,,,,, 2022.
[12] J. Yi and S. Yoon, Patthh SVDD: Patch-Level SVDD for Anomaly Detection and Segmenta- tion," in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12627 LNCS, 2021.
[13] O. Rippel, P. Mertens, and D. Merhof, "Modeling the distribution of normal data in pre-trained deep features for anomaly detection," " in Proceedings International Conference on Pattern Recognition, 2020.
[14] S. Lee, S. Lee, and B. C. Song, "Cfa: Coupled-hypeesphere- baased feature adaptation for target-oriented anomaly localization," IEEE Access, vol. 10, pp. 78446-78454, 2022.
[15] 7. Liu, Zhou, Y. Xu, and Z. Wang, "Simplenet: A simple network for image anomaly detection and localization." arXiv preprint 2001v 2023.

---

[16] X. Chen and K. He, "Exploring simple Siamese representation learning." in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2021.
[17] J. B. Grill el al., "Bootstrap your own latent new approach to self-supervised learning," in Advances Neural Information Processing Systems, vol. 2020-Decem, 2020.
[18) T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, "A simple framework for contrastive learning of visual representations," " 37th International Conference on Machine Learning, ICML. 2020, vol. PartF16814, 2020.
[19] Bergmann, M. Fauser, D. Sattlegger, and C. Steger, "Mvtec ad comprehensive real-world dataset for unsupervised anomaly detection," in Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 9592-9699990 9600, 2019.
[20] Y. Zou, J. Jeong, L. Pemula, D. Zhang, and O. Dabeer, Sppo-thhediifferenc sulf-supervised pre-training for anomaly detection and segmentation," in Computer Vision-ECCC 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XXX, pp. 392- 408, Springer, 2022.
[21] D. S. Kermany et al., "Identifying Medical Diagnoses and Treatable Diseases by Image-Based Deep Learning," Cell, vol. 172, no. 5, 2018.
[22] Asia Pacific Tele-Ophthalmology Society, "APTOS 2019 blindness detection." " 2019.
[23] N. Codella et al., "Skin lesion analysis toward melanoma detection 2018: A challenge hosted by the international skin imaging collaboration (isic)." arXiv preprint arXiv: 03368, 2019.
[24] Y. Cao, Q. Wan, W. Shen, and L.. Gao, "Informative knowledge distillation for image anomaly segmentation," Knowledge-Bast Based Systems, vol. 248, p. 108846, 2022.
[25] N. S. Keskar, D. Mudigere, J. Nocedal, M. Smelyanskiy, and P. T. P. Tang, "On large-batch training for deep learning: Generalization gap and sharp minima," in International Conference on Learning Representations, 2017.
[26] Foret, A. Kleiner, H. Mobahi, and B. Neyshabur, "Sharpness-aware minimization for effi- ciently improving generalization," in International Conference on Learning Representations, 2021.
[27] H. Li, Z. Xu, G. Taylor, C. Studer, and T. Goldstein, "Visualizing the loss landscape of neural nets," Advances in neural information processing systems, vol. 31, 2018.
[28] S. loffe and C. Szegedy, "Batch normalization: Accelerating deep network training by reducing internal covariate shift," in 32nd International Conference on Machine Learning, ICML 2015, vol. 1, 2015.
[29] C. Zhang, K. Zhang, C. Zhang, T. X. Pham, C. D. Yoo, and I. S. Kweon, "How does sim- siam avoid collapse without negative samples? unified understanding with self-supervised contrastive learning." " in International Conference on Learning Representations, 2021.
[30] S. Zagoruyko and N. Komodakis, "Wide residual networks," arXiv preprint arXiv: 1605.07166. 2016.
[31] Loshchilov and F. Hutter, "Decoupled weight decay regularization," 7th International Conference on Learning Representations, ICLR 2019, 2019.
[32] Y. Zhao, Q. Ding, and X. Zhang. "AE-FLOW: Autoencoders with Normalizing Flows for Medical Images Anomaly Detection," in The Eleventh International Conference on Learning Representations, 2023.
[33] P. Bergmann, M. Fauser, D. Sattlegger, and C. Steger, "Uninformed students: Student-teacher anomaly detection with discriminative latent embeddings." in Proceedings of the IEEF/CVF conference computer vision and pattern recognition, pp. 4183-4192 2020.
12

---

[34] D. Gudovskiy, S. Ishizaka, and K. Kozuka, "Cflow-ad: Real-time unsupervised anomaly detection with localization via conditional normalizing flows," in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 98-107, 2022.
[35] C. Li, K. Sohn, J. Yoon, and T. Pfister, "CutPaste: Self-Supervised Learning for Anomaly Detection and Localization," " in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2021.
[36] V. Zavrtanik, M. Kristan, and D. Skočaj, "DRAM A discriminatively trained reconstruction embedding for surface anomaly detection," in Proceedings of the IEEE International Conference on Computer Vision 2021.
[37] N. Cohen and Y. Hoshen, "SSub-image anomaly detection with deep pyramid correspondences." arXiv preprint arXiv:2005.02337, 2020.
(38] A. S. Collin and C. de Vleeschouwer, "Improved anomaly detection by training an autoen- coder with skip connections on images corrupted with Stain-shaped noise," Proceedings International Conference on Pattern Recognition, 2020.
[39] P. Perera and V. M. Patel, "Learning Deep Features for One-Class Classification," IEEE Trans- actions on Image Processing, vol. 28, no. 11, 2019.
[40] D. P. Kingma and M. Welling, "Auto-encoding variational bayes," in 2nd International Confer- ence on Learning Representations, ICLR 2014 Conference Track Proceedings, 2014.
[41] Milkovic, Filipovic, M. Subasic, Petkovic. S. Lonearic, and M. Budimir, "Ultrasound anomaly detection based on variational autoencoders," in International Symposium on Image and Signal Processing and Analysis, ISPA, vol. 202t-Sep,,,,er, 2021.
[42] H. Zhao et al., "Anomaly Detection for Medical Images Using Self-Supervised and Translation- Consistent Features," IEEE Transactions Medical Imaging, vol. 40, no. 12, 2021.
[43] T. Reiss, N. Cohen, L. Bergman, and Y. Hoshen, "Panda: Adapting pretrained features for anomaly detection and segmentation," " in Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 2806-2811 2021.
[44] K. Sohn, C.-L.. Li, J. Yoon, M. Jin, and T. Pfister, "Learning and evaluating representations for deep one-class classification," in International Conference on Learning Representations, 2021.
[45] X. Gui, D. Wu, Y. Chang, and S. Fan, "Constrained adaptive projection with pretrained features for anomaly detection," " in Proceedings of the Thirty-Firs International Joint Conference on Artificial Intelligence, IJCAI-22 (L. D. Raedt, ed.), pp. 2059-2065, International Joint Conferences on Artificial Intelligence Organization, 7 2022. Main Track.
[46] G. Xie, J. Wang, J. Liu, J. Lyu, Y. Liu, C. Wang, F. Zheng, and Y. Jin, "Im-iad: Industrial image anomaly detection benchmark in manufacturing," arXiv preprint arXiv:2301.13350 2023.
[47] B. Schölkopf, J. C. Platt, J. Shawe-Taylor, A. J. Smola, and R. C. Williamson, "Estimating the support of a high-dimensional distribution," " Neural Computation, vol. 13, no. 7, 2001.
[48] D. M. Tax and R. P. Duin, "Support Vector Data Description," Machine Learning, vol. 54, no. 1, 2004.
[49] Bergmann, K. Batzner, M. Fauser, D. Sattlegger, and C. Steger, "Beyond dents and scratches: Logical constraints in unsupervised anomaly detection and localization," International Journal of Computer Vision, vol. 130, no. 4, pp. 947-969, 2022.
[50] I. Molybog, P. Albert, M. Chen, Z. Veto,, D. Esiobu, N. Goyal, P. S. Koura, S. Narang, A. Poulton, R. Silva, et al., "A theory adam instability in large-scale machine learning," arXiv preprint axXiv:2304.09877 2023.
[51] S. J. Reddi, S. Kale, and S. Kumar, "On the convergence of adam and beyond," arXiv preprint 19419:190..09237, 2019.
13

---

Appendices
First, we give the formulation of unsupervised anomaly detection (UAD). Let Dirain ( (i)) Yi E {-} be the training set of normal samples, where 2; is the 1th sample with its label ye- During test, Drest = 3:)) Yi E {-, +} (- for normal, + for anomalous) contains both normal and anomalous images. Our goal is to train a model using Dtrain to identify (zz, +) from (I, -) in Diest, while localizing anomalous regions at the same time.
A Related Work
Epistemic methods are based on the assumption that the networks respond differently during inference between seen input and unseen input. Within this paradigm, pixel reconstruction methods assume that the networks trained on normal images can reconstruct anomaly-free regions well, but
38 variational auto-encoder (VAE) [40] 411. generative adversarial network (GAN) 42 7 are used to restore normal pixels. However, pixel reconstruction models may also succeed in restoring unseen anomalous regions they resemble normal regions in pixel values or the anomalies
poorly for anomalous regions [38]. Auto-encoder (AE) [39
barely noticeable [2] [9. Therefore, feature reconstruction is proposed to construct features of pre-trained encoders instead of raw pixels To prevent the whole network from converging to trivial solution, the parameters of the encoders are frozen during training 9 In feature distillation [ 6 the student network is trained from scratch to mimic the output features of the pre-trained teacher network with the same input of normal images, also based on the similar hypothesis that the student trained on normal samples only
23
succeed in mimicking features of normal regions.
Pseudo-anomaly methods generate handcrafted defects on normal images to imitate anomalies, converting UAD to supervised classification [35] segmentation task [36]. Specifically, CutPaste simulates anomalous regions by randomly pasting cropped patches of normal images. DRAEM constructs abnormal regions using Perlin noise as the mask and another image as the additive anomaly. SimpleNet [15] introduces anomaly by injecting Gaussian noise in the pre- pre-trained feature space. These methods deeply rely on how well the pseudo anomalies match the real anomalies (2), which makes it hard to generalize to different datasets.
35
36
13 14 memorize all (or their modeled distribu- tion) normal features extracted by networks pre-trained on large-scale datasets and match them with test samples during inference. Since these methods require memorizing, processing, and matching nearly all features from training samples, they are computationally expensive in both training and inference, especially when the training set is large. There is considerable semantic gap between large-scale natural images on which the frozen networks were pre-trained and the target UAD images in various modalities. Therefore, methods based on pre-trained networks struggle in transferring as the feature encoders are not optimized in the target domain.
Feature memory & modeling methods [00 12
There are few studies addressing the adaptation problem in UAD settings. CFA 14 and SimpleNet[IS unanimously utilize learnable linear layer to transform the output features of the frozen encoder. However, the linear layer may be insufficient for adaption and can hardly recover the domain-specific information in pre-trained features, especially when the target domain, e.g. CT and MRI, is remote from natural image domain. In OCC settings, number of works focus on utilizing self-supervised pre-training approaches to learn a compact representation space for normal samples [43] 44 45). However, the performances far from satisfactory in UAD tasks (I-AUROC under 90% on MVTec AD).
B Datasets
MVTec AD 11 an industrial defect detection dataset, containing sub-datasets (5 texture categories and 10 object categories) with total of 3,629 normal images as the training set and 1,725 normal and anomalous images as the test set. Pixel-level annotations are available for anomalous images for evaluating anomaly segmentation. All images resized to 256x256.
14

---

VisA [20] is an industrial defect detection dataset, containing 12 sub- datasets with total of 9,621 normal and 1200 anomalous images. Pixel-level annotations are available for anomalous images for evaluating anomaly segmentation. We split it into training and test sets following the setting in [20] All images are resized to 256x256.
OCT2017 is an optical coherence tomography (OCT) dataset [21.. The dataset is labeled into four classes: normal, drusen, DME (diabetic macular edema), and CNV (choroidal neovascularization), The latter three classes are considered anomalous. The training set contains 26,315 normal images and the test set contains 1,000 images. All images are resized 256x256.
APTOS is a color fundus image dataset, available as the official training set of the 2019 APTOS blindness detection challenge [22]. The dataset contains 3,662 images with annotated grade 0-4 to indicate different severity of diabetic retinopathy, yielding 1,805 normal images (grade 0) and 1857 anomalous images (grade 1-4). We randomly selected 1,000 normal images as our training set and the rest 2,662 images as the test set. Images are preprocessed to crop the fundus region, and then resized 256 256.
ISIC2018 is a skin disease dataset, available as task ISIC2018 challenge 223 It contains seven classes. NV (nevus) is taken as the normal class and the rest of classes are taken as anomaly, following [32]. The training set contains 6705 normal images. The official validation set is used as our test set, which includes 193 images. Images resized to 256 256 and then center cropped into 224 x 224 to remove redundant background. In addition, because the normal and anomalous images of ISIC2018 different objects (lesions) instead of healthy objects and unhealthy objects, the task is more like one-class classification. Therefore, we take the mean (instead of maximum) value of Smop as the Stmg in ReContrast and other reproduced methods.
C Complete Implementation Details
WideResNet50 30 pre-trained on ImageNet M is utilized as the encoder by default. The decoder is
the upside-down version of the encoder, exactly the same as RD4AD 22, i.e., the 3x3 convolution with stride at the beginning of each layer is replaced by 222 transposed convolution with stride 2. The bottleneck is also the same as RD4AD 2 For each decoder layer, two 1x1 convolutions are used to project the feature map to reconstruct the frozen encoder and trained encoder, respectively. AdamW optimizer [3T] is utilized with (=(0.9,0..00) and weight decay=le-5. 'he learning rates of the new (decoder and bottleneck) and pre-trained (encoder) parameters are 2e-3 and le-5 respectively. The network is trained for 3,000 iterations each sub-dataset in VisA, 2,000 on each sub-dataset in MVTec AD and ISIC2018, and 1,000 on APTOS and OCT2017. The batch size is 16 for industrial datasets and 32 for medical datasets. The a in (5) linearly rises from -3 to I in the first one-tenth iterations and keeps for the rest of the training. By default, the batchnorm (BN) layers of encoder set to train mode during training. Because training instability and performance drop observed for some categories [ the BN of encoder is set to eval mode (use pre- pre-trained statistics) for such ( is used to plot the landscape in Figure 4.
datasets. The reason discussed in Appendix E. A library The distance and steps arguments are set to and 50 respectively.
On MVTec AD and VisA, the results of comparison methods are taken from the original papers or the benchmark paper (46) that report their performances. On medical datasets, we take the results from 32] if available; otherwise, we reproduce the method with official code. For RD4AD, we use their official code, in which Cylobat is already implemented instead of Lregion. as discussed in Section 2.1. Codes are implemented with Python 3.8 and PyTorch 1.00.0 cuda 11.3. Experiments are run on NVIDIA GeForce RTX3090 GPUs (24GB).
D Additional Experiments
D.1 Additional Experimental Results
In addition to the averaged segmentation performances in Table we present the P-AUROC and AUPRO of each categories in MVTec AD in Table Al The results in the main paper are reported
! Toothbrush, Leather, Grid, Tile, Wood, Scnew in MVTec AD, cashew, VisA, and OCT2017. wnttp:///gttuu..oom/// http:///hw..ommmmmmmmoeeeeettttd
15

---

with single random seed following our baseline 2 In Table A2 we report the mean and standard deviation of three runs with three different random seeds 11, and 111). In addition the averaged performances in Table 3, we present the I-AUROC, P-AUROC, and AUPRO of each category in VisA
Table A3
Table AI: Anomaly segmentation performance of 15 subset (categories) of MVTèc AD (%).
Carpet
Grid
Leather
Tile
Wood
Bottle
Cable
Capsule
P-AUROC
99.3
99.2
99.5
96.3
95.9
99.0
98.9
98.4 95.4
AUPRO
97.9
97.8
99.2
93.7
92.5
97.1
95.6
Hazelnut
MutalNut
Pill
Screw
Toothbrush
Transistor
Zipper
P-AUROC AUPRO
99.1
98.7
99.1
99.6 98.6
99.2 95.0
95.4 82.3
98.1
95.9
94.4
97.7
94.9
Table A2: Performance MV'èc AD over three runs (%).
I-AUROC
P-AUROC
AUPRO
Carpet
99.72 0.27
99.27 + 0.05
98.55 + 0.88
Grid
100 + 0.00
99.26 + 0.05
97.79 + 0.03
Leather
100 0.00
99.48 0.02
99.17 0.04
Tile
0.17
96.18
92.95 + 0.66
Wood
99.05 - 0.05
0.09
92.61 0.10
Bottle
100 + 0.00
99,00
97.13 1 0.08
Cable
99.58 = 0.15
9.02
95.63 + 0.02
Capsule
97.86 0.36
95.34 0.04 0.00
Hazelnut
100 + 0.00
99.08
05.87
MetalNut
100 - 0.00
0.00
94.49 0.09
Pill
98.94 = 0.24
0.00
97.75 0.05
Screw
97.80 0.14
9.02
98.50 + 0.07
Toothbrush
99.44 0.45
0.02
0.02
Transistor
99.65 0.11
0.00
82.61 + 0.22
Zipper
99.68 = 0.15
98.16 0.04
95.12 + 0.25
All Avg
99.42 0.05
98.39 + 0.01
95.21 + 0.06
Table A3: Performance of 12 subset (categories) of VisA (%).
candle
capsules
cashew
chewinggum
fryum
macaronil I
LAUROC P-AUROC
97.20
93.55
98. 14
99.28
97.56
98.84
99.15
99.46
97.41
97.36
91.96
99.01
AUPRO
94.77
94.45
94.28
86.63
79.11
93.63
macaroni2
peb1
pcb2
pcb3
peb4
pipuffryum
I-AUROC P-AUROC
91.79
97.86
97.84
98.18
99.82
99.96
99.04
99.79
98.99
99.05
98.70
98.30
AUPRO
97.36
96.76
92.47
95.14
91.29
95.57
D.2 Qualitative Visualization
The qualitative anomaly segmentation results of ReContrast trained by Leglobel and ReContrast trained by Legoont- fm are presented in Figure AT It is shown that the hard-normal regions are less activated with Cgiobal-hm- The results of VisA medical images are shown Figure A2 and Figure respectively, Each Smap is
to 0-1 for clearer visualization.
D.3 Additional Ablation Study
Ablation study is conducted on the value of in Cgiobal-mm. as shown in Table A4 Assuming the regional cosine distances R (h, w) conforms Gaussian distribution, the discarding rates of feature
16

---

point are 2.3%, 15.9%, 50%, 69.1%, 1,, 84.1%, 93.3% and 97.7% for a 22 -1, 0, 0.5, 1, 1.5 and 2. respectively. Lytohal-hrn is not sensitive the discarding rate within the range from 50% to 93%. Though a can be tuned for each dataset, we find
works just well for most cireumstances.
We test variety of encoder backhones in Table A5 i.e., ResNet18, ResNet50, and WideRessNet50 (default), and report their performances, model parameters, as well as multipyy-acumulate operations (MACs). The corresponding decoder is the reversed version of the encoder. Different encoders may favor different training nyper-parameters such as learning rate and iteration. Though we do not further tune each backbone, all backbones produce excellent results with default the generality of our method. With each backbone, our method outperforms the corresponding feature reconstruction counterpart RD4AD (2. Notably. our method with ResNet18 is comparable to RD4AD with WideResNet50.
suggesting
The utilization of two encoders (one frozen, one trained domain-specifie) in method function as an ensembling, which may also contribute to performances. We conduct ablation experiments using different encoder-decoder feature map pairs generate anomaly maps in inference. The results are shown in Table A6 On MVTec, using only the feature of domain-specific encoder produces an I-AUROC 99.38%, which comparable the 99.45% of the ReContrast default and outperforms the 98.86% of baseline Config. B. In addition, we train an ensembling version of Config. B using two different encoders (ResNet50 and WideResNet50)), producing an I-AUROC of 98. 98.94% which is nearly identical to the baseline. On APTOS, using only the feature of domain-specifer encoder produces I-AUROC of 97.73% which outperforms both 97.5 of the ReContrast default and 92.49 22.49% of the baseline Config. B. The results indicate that the improvement due to ensembling is small and the dopain-specific encoder is vital.
Table A4: Ablation the values of a in Liglobal - hm MVTTc AD (%).
inn (Cgiobal)
-2
-1
0
0.5
1
1.5
2
discard rate
0%
2.3%
15.9%
50%
69.1%
84.1%
93.3%
97.7%
T-AUROC P-AUROC
99.13
99.13
99.14
99.32
99.39
99.45
99.38
98.85
98.09
98.11
98.13
98.30
98.31
98.37
98.39
98.25
AUPRO
94.59
94.62
94.65
94.97
95.00
95.20
95.28
95.13
Table A5: Ablation on the encoder backbones on MVTec AD (%).
Backbone
ResNet18
ResNet50
WResNet50
Method
RD4AD
Ours
RD4AD
Ours
RD4AD
Ours
Params(M)
18.7
21.7
63.6
74.9
117
145
MACs(G)
4.95
10.1
19.4
42.0
36.0
75.2
I-AUROC
97.9
98.7
98.4
99.1
98.5
99.5
P-AUROC
97.1
97.9
97.7
98.3
97.8
98.4
AUPRO
91.2
94.2
93.1
95.0
93.9
95.2
Table A6: Ablation the of different encoder feature maps for caleulating Smop (%).
encoder feature maps for calculating Smap
MVTTe AD
APTOS
Config. B
3 frozen
98.86
92.49
Config. (R50+WR50)
6 frozen
98.94
92.14
Ours
3 frozen only
99.16
95.32
Ours
trained only
99.38
97.73
Ours
3 trained + 3 frozen
99.45
97.51
E Limitations
Scope of Application. this work, we mainly focus on UAD that detects regional defects (most
common in practical applications). which is distinguished from one-class classification (OCC, or
17

---

Semantic AD). In our UAD, normal and anomalous samples are semantically the same objects except local detects, e.g. good cable v.s. spoiled cable. In OCC, normal samples and anomalous samples are semantically different, e.g. cat v.s. other animals. The slight difference in task setting makes the focus of corresponding methods different. Most methods for UAD attempt to detect anomalies by segmenting anomalous local regions, while methods for OCC detect anomalies based on the representation deviation of the whole image [47 48). Though methods of these two tasks can be used for each other to a certain extent, we mainly focus on UAD in this work.
Logical Anomaly. MVTec LOCO is a recently released dataset for evaluating the detection perfor- mance of logical anomalies. It contains 5 sub-datasets, each comprised of both structural anomalies, e.g. dents and scratches as in MVTec AD, and logical anomalies, e.g. dislocation and missing parts. We find our method performing less favorably on logical anomalous images than GCAD [49] which specially designed for logical anomalies, as presented in Table [7 We still outperform other vanilla UAD methods that are not specially designed for such anomalies.
Table A7: Anomaly Detection Performance on MVTec LOCO (%). Structural I-AUROC is calculated
on normal images and structural anomalous images. Logical I-AUROC is calculated normal images and logical anomalous images. Mean I-AUROC is the average of the above two scores,
RD4AD 2
DRAEM 36
PaDiM 10
PatchCore
GCAD 49
Ours
struct. I-AUROC
88.0
74.4
70.5
82.0
80.6
90.7
logic. I-AUROC
69.4
72,8
63.7
69.0
86.0
73.4
mean I-AUROC
78.7
73.6
67.1
75.5
83.3
82.1
Batch Normalization and Training Instability. As discussed in Appendix C, training instability and performance drops are observed for a few categories when setting BN mode of the encoder to train. The phenomenon of training instability observed in many deep learning tasks. However, validation sets are often allowed, which counteracts training instability and helps the choice of BN mode.
On the one hand, it can be caused by the limited feature diversity of the one-class UAD dataset. First, we recall the formulation of BN layer. Giving a d-dimensional feature x = (x(11, ,.., x (())), each dimension (channel) is normalized in training:
(() E. [K)
(A)
Var [no(k]] +
where the expectation E and variance Var are computed over the batch, and c is small constant for numerical stability. In most computer vision datasets, an input batch contains a variety of categories. Each dimension is more or less activated by different image features so that V Var [x] is not zero. However, because a UAD dataset contains only one type of category, there is a chance that channel of pre-trained feature (4:) is barely activated or equally activated, e.g. channel sensitive
to animals is dead on industrial objects. Thus, the denominator / /Var ar [z(()] + is nearly zero, causing malfunction of batch normalization. This problem can be fixed by using pre-trained statistical running_meet meen and runningyvar var (eval mode), which loses the effect of BN.
On the other hand, this instability can be attributed to the feature of Adam optimizer. We found that number UAD methods 24 [5] are also subject to training instability when running their code. Whether the last epoch (for reporting results) is in the middle of a loss spike and performance dip is strongly related to random seed. In some works 51, the authors attribute the training instability to the historical estimation of squared gradients in Adam optimizer.
In our according explorations, we replace the batch variance Var [ao( smaller than min(5e-4, running_van by pre-trained running var and we reset the historical state buffer (gradient momentum and second-order gradient momentum) of Adam every 500 iters. Without manually selecting BN mode, such tricks enable training with more stability. It yields I-AUROC of 99.4 % on MVTec and 97.28% on VisA, comparable to the 95. and 97.5% of our reported result and still outperforms previous SOTAs. In the future, approaches to eliminating the problem of BN and optimizer can be further investigated.
18

---

ground
Smap by
Smop by Lggtobat-am hm
ground truth
S
Smap by gyoobal-km
image
global
image
global
Figure A1: Smap of our ReContrast trained by Lygabal Leglobal han Hard-normal regions less activated with Lyytobll
19

---

image
inago
Str
image
Figure A2: Smap of our method VisA.
nermal (bealthy)
anomalous (unhealthy)
anomalous (unhealiby)
APTOS
OCT2017
ISIC2018
Figure A3: mmap our method on medical image datasets.
20


Reviews:
[{"review_id": "vau6xKyop7", "reviewer": "Reviewer_Jpdb", "summary": "This paper introduces a feature reconstruction model, namely ReContrast, for anomaly detection. By incorporating a set of techniques including feature global similarity, stop gradient, contrastive pair optimizations, the authors shows good performance of ReContrast on both industrial and medical datasets.", "questions": "Please refer to question 2-4 in the weakness section.", "limitations": "The appendix discuses the limitations of this method.", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper is easy to follow. The logic behind some technical improvements is well motivated.\n\n2. Extensive experimentation on diverse anomaly detection datasets is conducted. The numerical results of ReContrast outperforms prior arts.\n\n3. The analysis of limitation of RD4AD, especially the Global Cosine Similarity, is impressive.", "weaknesses": "1. The paper presentation should be improved. The current version is in a lab report format, outlining the observed issues and employing trial-and-error strategies to address them. It would be preferable to present the methodology in a more cohesive manner.\n\n2. I find it unclear why Config. D would prevent model collapse. As the feature similarity loss can be back-propagated from the decoder to the encoder, the encoder and decoder may collaborate to achieve a trivial solution.\n\n3. The motivation behind Config. E is also unclear to me. If the decoder's objective is to reconstruct the pre-trained encoder, what is the purpose of the trainable encoder? What is the objective function in Config. E?\n\n4. Figure 4 illustrates the model landscape with respect to model's trainable parameters. A flatter surface indicates a more stable training process, but it does not necessarily indicate better generalization concerning input data.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank you for your constructive comments with great insights. The replies to specific remarks are as followed:\n\n__1. The paper presentation should be improved. The current version is in a lab report format, outlining the observed issues and employing trial-and-error strategies to address them. It would be preferable to present the methodology in a more cohesive manner.__\n\nThank you for this kind suggestion. Inspired by the narrative of a number of previous arts [a, b], we tried a writing style of “spot a challenge, analysis the mechanism, solve the problem”, instead of the conventional manner of “methods in Method, experiments in Experiment”. This writing logic involves bringing some primary experiments in the Method section. We believe such an exploration-like narrative illustrates the motivation and intuition of our proposed method better. Though three challenges and solutions (global cosine, stop-grad, contrastive pair) are presented sequentially, they are all inspired by and strongly correlated to a cohesive center, i.e., elements of contrastive representation learning. \n\n\n__2. I find it unclear why Config. D would prevent model collapse. As the feature similarity loss can be back-propagated from the decoder to the encoder, the encoder and decoder may collaborate to achieve a trivial solution.__\n\nA short and intuitive explanation of stop-gradient operation, which is wildly used to prevent model collapse in contrastive representation learning (CL) [a, c], is that the optimization goals of the encoder and decoder (a.k.a predictor in CL) are different though gradients are back-propagated together. The goal of the decoder is to reconstruct non-constant encoder features, while the goal of the encoder is to provide discriminative information so that the decoder can reconstruct non-constant informative objections. As long as the encoder is not initialized to generate constant features, the configuration will not collapse to a trivial solution. We call this explanation “mutual reinforcement” in line 169-171.\n\nThere are also a number of works that try to explain the deeper mechanism of stop gradient. In [a], the authors hypothesize this configuration as an implementation of an Expectation-Maximization (EM) algorithm that implicitly solves two underlying sub-problems with two sets of variables. In [d], the authors argue that there are flaws in the hypothesis of [a] and suggest that the decomposed gradient vector (center gradient) helps prevent collapse via the de-centering effect. We will refer to them as further explanations of the mechanism of stop-grad in the final version. \n\n\n__3. The motivation behind Config. E is also unclear to me. If the decoder's objective is to reconstruct the pre-trained encoder, what is the purpose of the trainable encoder? What is the objective function in Config. E?__\n\nWe apologize for the ambiguity caused. The decoder takes the representation from the trainable encoder to reconstruct the pre-trained encoder, while it also takes the representation from the pre-trained encoder to reconstruct the trainable encoder __at the same time__.\n\nThe objective function of Config. E is $L_{global}(E_{train}, D_{freeze})+ L_{global}(E_{freeze}, D_{train})$, where $E_{train}$ is the feature from trainable encoder, $E_{freeze}$ is the feature from frozen encoder,  $D_{freeze}$ is the decoder feature with the input of frozen encoder, $D_{train}$ is the decoder feature with the input of trainable encoder.\n\nIn the final version, we will make a clearer description in Sec.2.5 as: “The decoder and bottleneck take the features of the domain-specific encoder to reconstruct the frozen encoder; vice versa, they simultaneously take the features of the frozen encoder to reconstruct the domain-specific encoder, building a complete two-view contrastive paradigm.” We revise Figure.1 and Figure.2 to be more precise, as shown in the PDF in the global reply.\n\n\n__4. Figure 4 illustrates the model landscape with respect to model's trainable parameters. A flatter surface indicates a more stable training process, but it does not necessarily indicate better generalization concerning input data.__\n\nPrevious works of literature [25, 26] suggest that the model in flat minima generalizes better than the model in sharp minima, including the well-known Sharp Awareness Minimization (SAM) [26] that implements this theory to design optimizer. (Please note that it is the flatness of the area near the optimized local minimum, not the whole landscape). The generalization is actually unwanted in UAD settings because the model may also well reconstruct unseen anomalies. We refer to this theory to explain why global cosine performs better than regional cosine after converging to a sharper local minimum. We will make a clearer claim in the final version that it is the sharpness around local minima not global landscape that indicates the generalization ability.\n\n[a] X. Chen and K. He. “Exploring simple Siamese representation learning”. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2021. ([16] in paper)\n\n[b] X. Chen, S. Xie, and K. He, “An Empirical Study of Training Self-Supervised Vision Transformers,” in Proceedings of the IEEE International Conference on Computer Vision, 2021.\n\n[c] J. B. Grill et al., “Bootstrap your own latent a new approach to self-supervised learning,” in\nAdvances in Neural Information Processing Systems, 2020. ([17] in paper)\n\n[d] C Zhang et al., “How does simsiam avoid collapse without negative samples? a unified understanding with self-supervised contrastive learning”, in International Conference on Learning Representations, 2022.\n"}, {"review_id": "G9D5Lj9yrw", "reviewer": "Reviewer_dCnY", "summary": "This work proposes an unsupervised anomaly detection method using contrastive learning method to reduce the biases between the pre-trained image domain and the target domain. In the paper, different designs of reconstruction networks are introduced and compared in terms of detection performance, model coverage and feature diversity. Finally, authors adopt a contrastive learning method equipped with global cosine loss, hard example mining, as well as additional frozen pre-trained encoder branch. This work conducts extensive experiments across two industrial defect detection benchmarks and three medical image anomaly detection tasks to demonstrate the transfer ability on various image domains.", "questions": "- As the work follows the SimSiam’s stop-gradient operation to prevent model collapsing. How about using momentum encoder in Moco to train the domain-specific encoder?\n- In this framework, the encoder must be switched during training, why not merge the features from flexible encoder and frozen encoder and then fed to the neck and decoder? In this way a unified network can be achieved.", "limitations": "Authors have addressed limitations of the method on training instability and logical anomaly detection.", "rating": 7, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "+ This work analyzes the domain gap between the pretrained image domain and the dataset image domain and tries a variety of modifications on distillation network to reduce the domain gap.\n+ Extensive experiments are conducted on MVTec AD, VisA, and three medical image datasets to demonstrate the transfer ability on various image domains.", "weaknesses": "- During both training and inference, the feature loss is calculated using six feature pairs from two encoders. The utilization of two encoders in the methods leads to an increase in parameters and inference time. However, this approach functions as an ensemble, contributing to improved performance. It is important to note that the ablation study does not provide results solely generated by the domain-specific encoder.\n- The training process for configuration E lacks explicit description in the text. In accordance with the information provided in Figure 2, the encoders are switched and repeated during training. However, the text does not provide a detailed explanation of this process.\n- Figure 4 displays the landscapes of S_map, which allows for a comparison between the two losses. According to the description, S_map represents the average anomaly score map of the APTOS dataset. However, the paper or reference [27] does not provide an introduction on how to interpret or visualize the figures, nor does it explain why average anomaly score maps from different images are used, considering that they may have different anomaly locations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank you for your constructive comments with great insights. \n\n__1. During both training and inference, the feature loss is calculated using six feature pairs from two encoders. The utilization of two encoders in the methods leads to an increase in parameters and inference time. However, this approach functions as an ensemble, contributing to improved performance. It is important to note that the ablation study does not provide results solely generated by the domain-specific encoder.__\n\nThank you for the insightful suggestions. Following your advice, we have conducted ablation experiments using the domain-specific encoder only to generate anomaly maps in inference. The results are shown in the following table. On MVTec, using only the feature of domain-specific encoder produces an I-AUROC of 99.38%, which is comparable to the 99.45% of the ReContrast default and outperforms the 98.86% of baseline Config. B. In addition, we train an ensembled version of Config. B using two different encoders (ResNet50 and WRN50), producing an I-AUROC of 98.94% which is nearly identical to the baseline. On APTOS, using only the feature of domain-specific encoder produces an I-AUROC of 97.73%, which outperforms both 97.51% of the ReContrast default and 92.49% of the baseline Config. B. The results indicate that the improvement due to ensembling is small and the domain-specific encoder is vital. These ablation experiments will be added in the final version.\n\n__I-AUROC (%) on MVTec AD and APTOS.__\n|   |    |     |  |\n|:--------------------:|:----------------------------:|:------------:|:---------:|\n| **-**                | **encoder feature of $S_{map}$** | **MVTec AD** | **APTOS** |\n| Ours (default)        | 3 trained+3 frozen           | 99.45        | 97.51     |\n| Ours        | 3 trained only   | 99.38        | 97.73     |\n| Ours                 | 3 frozen only     | 99.16        | 95.32     |\n| Config.B             | 3 frozen      | 98.86        | 92.49     |\n| Config. B (R50+WR50) | 6 frozen   | 98.94        | 92.14     |\n\n__2. The training process for configuration E lacks explicit description in the text. In accordance with the information provided in Figure 2, the encoders are switched and repeated during training. However, the text does not provide a detailed explanation of this process.__\n\nThank you for your valuable comments. The “switch and repeat” means that the decoder takes the representation from the trainable encoder to reconstruct the pre-trained encoder, while it also takes the representation from the pre-trained encoder to reconstruct the trainable encoder at the same time (switch position). In the manuscript, it is described in line 190-192: “The decoder and bottleneck take the features of one encoder as the input to reconstruct the features of the other encoder and __vice versa__.” \n\nIn the final version, we will make a clearer description following your suggestion: “The decoder and bottleneck take the features of the domain-specific encoder to reconstruct the frozen encoder; vice versa, they simultaneously take the features of the frozen encoder to reconstruct the domain-specific encoder, building a complete double-view contrastive paradigm.” We will revise Figure.1 and Figure.2-Config. E to be more precise, as shown in the PDF in the global reply.\n\n__3. Figure 4 displays the landscapes of S_map. According to the description, S_map represents the average anomaly score map of the APTOS dataset. However, the paper or reference [27] does not provide an introduction on how to interpret or visualize the figures, nor does it explain why average anomaly score maps from different images are used, considering that they may have different anomaly locations.__\n\nThank you for your comments and we will add descriptions in the final version to make it clearer to readers. The x-y plane (horizontal) does not represent the location in images. Given a network architecture and its trained parameters, this tool [27] calculates and visualizes the loss (or metric, i.e. averaged S_map) surface along random directions near the given (optimal) model parameters. Therefore, the figures visualize the averaged S_map (a scaler, average of all locations of all samples, you can consider it as mean L_region) on z-axis, against the model parameter space (reduced to 2-dimension) on x-y plane (not image locations). This metric v.s. parameter landscape indicates the stability of training and the generalization ability. Axis labels are added in the revised Figure 4 as presented in the PDF in global reply.\n\n__4. As the work follows the SimSiam’s stop-gradient operation to prevent model collapsing. How about using momentum encoder in Moco to train the domain-specific encoder?__\n\nThank you for this insightful question. We have tried to use an EMA (exponential moving average momentum) version of the domain-specific encoder to substitute the frozen encoder, which makes it similar to BYOL [17]. It yields an I-AUROC of 99.43% on MVTec, which is comparable to the 99.45% of our default ReContrast. This result also complies with the conclusion of SimSiam that it is the implicit stop-grad operation in the momentum encoder that prevents the collapsing  .\n\n__5. In this framework, the encoder must be switched during training, why not merge the features from flexible encoder and frozen encoder and then fed to the neck and decoder? In this way a unified network can be achieved.__\n\nThank you for this valuable comment. The “switch and repeat” represents the simultaneous cross-reconstruction between two encoders. Therefore, if you mean merging the features of flexible encoder and frozen encoder on batch dimension, it is exactly what we mean and do. We concatenate two groups of features in a batch and then feed them to the neck and decoder. Then we split the output of decoder back into two groups. We have revised the architecture of Config. E in Figure 2 for clearer illustration, as shown in the PDF in the global reply.\n\n\n\n\n"}, {"review_id": "liaTQRPqKy", "reviewer": "Reviewer_xS4v", "summary": "This paper proposes a novel unsupervised anomaly detection (UAD) method called ReContrast. The method addresses the issue of poor transferability of pre-trained encoders from natural image domains to target UAD domains such as industrial inspection and medical imaging. ReContrast optimizes the entire network to reduce biases towards the pre-trained image domain and aligns the network with the target domain. It combines feature reconstruction and contrastive learning to prevent training instability and pattern collapse. The method is evaluated on industrial defect detection benchmarks and medical image UAD tasks, demonstrating its superiority over current state-of-the-art methods.", "questions": "I really appreciate the structure of this article. The author has done a lot of work in the demonstration experiment, but the method innovation and contribution are relatively limited, but the overall is worth learning. Here are some suggestions.\n1.Regarding the experiment of Global Cosine Similarity, the author conducted the experiment on the original RD4AD, namely Config A and Config B. It is suggested to add the experimental setting using Region Cosine Similarity to the experimental setting of Config E. To demonstrate the validity of Global Cosine Similarity on ReContrast, rather than a special case on RD4AD Settings.\n2.In the experimental part, I noticed that the P-AUROC/AUPRO index lacked two model indexes compared with I-AUROC, ADTR and CFlow AD, in the experiment conducted on MVTec AD dataset. I hope the author could provide completion to ensure the completeness of the experiment.\n3.It is hoped that the author can visualize how the model gets the Anomaly score map in the visual RD4AD in Figure 2 to ReContrast, so as to facilitate readers' understanding.", "limitations": "Limitations are clearly described and I don't expect negative societal impact of this work.", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is well-written and easy to read.\n2. This paper propose a new unsupervised anomaly detection method, ReContrast. According to the suggested setup, it is technically valid and should work well.\n3. Sufficient experiments are done to demonstrate the effectiveness of the method, and the robustness of the method is proved by experiments on multiple data sets\n4. The results were strong. Performance is excellent in most cases.", "weaknesses": "1. In the ablation experiments of MVTec AD and APTOS, for example, the experiment on Global Cosine Similarity, the author conducted experiments on the original RD4AD, namely Config A and Config B. This is not sufficient to prove the validity of Global Cosine Similarity on ReContrast, and may be a special case on RD4AD Settings.\n2. The realistic contribution of this paper is relatively weak. Compared with other methods, the reconstructed method does not rely on the pre-trained model or the memory library in anomaly detection. The method in this paper relies on the pre-trained model to maintain the feature extraction capability of the encoder, which I think may affect the practical application of the model.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "First and foremost, we are thrilled that you like our writing logic. We made a lot of considerations to use this exploration-like narrative instead of a conventional paper structure.\n\n__1. In the ablation experiments of MVTec AD and APTOS, for example, the experiment on Global Cosine Similarity, the author conducted experiments on the original RD4AD, namely Config A and Config B. This is not sufficient to prove the validity of Global Cosine Similarity on ReContrast, and may be a special case on RD4AD Settings.__\n\nThank you for your valuable suggestion. We have conducted an ablation experiment replacing the global cosine loss of Config.E by regional cosine loss, called Config.E-R. The results are shown in the following table.  On MVTec AD, Config. E-R produces a final I-AUROC of 98.93%. On APTOS, Config. E-R produces a final I-AUROC of 96.39%. The performance is inferior to Config.E with global cosine but superior to Config.A without other proposed contrastive elements, which demonstrates the validity of global cosine loss as well as stop-grad and contrastive pairs. The results will be added to Table 5 in the final version.\n\n__I-AUROC (%) on MVTec AD and APTOS.__\n| | | | | | | |\n|------|:------------:|:-----------------:|:-------------:|:------------------:|:------------:|:-----------:|\n| **-**       | **$L_{global}$** | **optim-encoder** | **stop-grad** | **contrast-pairs** | **MVTec AD** | **APTOS**   |\n| Config. A   |              |                   |               |                    | 95.31  | 90.12 |\n| Config. B   | √             |                   |               |                    | 98.86  | 92.49 |\n| Config. E-R |              | √                 | √             | √                  | 98.93  | 96.39 |\n| Config. E   | √            | √                 | √             | √                  | 99.13  | 97.32 |\n\n__2.The realistic contribution of this paper is relatively weak. Compared with other methods, the reconstructed method does not rely on the pre-trained model or the memory library in anomaly detection. The method in this paper relies on the pre-trained model to maintain the feature extraction capability of the encoder, which I think may affect the practical application of the model.__\n\nThank you for this comment. The reconstructed methods you refer to are the image-reconstruction methods that train Auto-Encoder-like and GAN-like networks to reconstruct raw pixels from scratch. In line 21-22 of Appendix, we review such methods and point out their insufficiency: “However, pixel reconstruction models may also succeed in restoring unseen anomalous regions if they resemble normal regions in pixel values or the anomalies are barely noticeable [10; 11].” As far as we know, the best approach in this paradigm is RIAD [a] which produces an I-AUROC of only 91.7% on MVTec AD, compared with the 96%+ of recent methods that utilize pre-trained encoders. In addition, we also compare two such methods (f-AnoGAN, GANomaly) on medical datasets in Table 4, which also produces undesirable results. \n\nWe totally agree with your comments on “methods relied on pre-trained model limits the application”. It is actually the motivation of our work. In line 34-35 of the manuscript, we mentioned that “In previous works, poor transfer ability is caused by the semantic gap between natural images on which the frozen networks were pre-trained and various UAD image modalities”. The main motivation of this paper is to reorient the large-scale pre-trained encoders to UAD datasets including medical images that are far from ImageNet, e.g. CT and MRI, which promotes the practical applications of UAD on various image domains and modalities.\n\n[a] Zavrtanik, V., Kristan, M., & Skočaj, D. (2021). Reconstruction by inpainting for visual anomaly detection. Pattern Recognition, 112, 107706.\n\n__3. In the experimental part, I noticed that the P-AUROC/AUPRO index lacked two model indexes compared with I-AUROC, ADTR and CFlow AD, in the experiment conducted on MVTec AD dataset.__\n\nDue to the limitation of page length and width of the initial submission, some comparison results are not put in. The I-AUROC and AUPRO of CFlow-AD are 98.62% and 94.60%. The I-AUROC of ADTR is 96.1%. We will add the results in the Appendix in final version.\n\n__4. It is hoped that the author can visualize how the model gets the Anomaly score map in the visual RD4AD in Figure 2 to ReContrast, so as to facilitate readers' understanding.__\n\nThank you for your valuable suggestion. We add the visualization of score map calculation in Figure 2, as shown in the PDF in the global reply.\n\n"}, {"review_id": "4GzE5zjGo8", "reviewer": "Reviewer_e2Vq", "summary": "This paper proposes an epistemic method, namely ReContrast, for unsupervised anomaly detection (UAD). The proposed method is deeply based on UniAD and RD4AD, with some technical improvement to adapt to the data from different domains. Experiments are conducted on two popular industrial defect detection benchmarks and three medical image UAD tasks. ", "questions": "1.\tUnclear contributions compared to existing methods: The paper does not clearly differentiate its contributions from those of existing methods. Both feature reconstruction [9] and contrastive learning [a,b] have been extensively explored in the anomaly detection literature. Although the authors mention the usage of stop-gradient as their main contribution (Lines 75-76), it has already been employed in [a,b].\n2.\tMissing important experimental comparisons: While this paper heavily relies on UniAD [9], it fails to provide any comparisons with UniAD in Table 1, Table 2, and Table 3. Including such comparisons would be crucial for evaluating the proposed method against the baseline.\n3.\tPoor organization: The organization of Section 2.2-2.6 is confusing and lacks coherence. These subsections appear to be assembled randomly without a clear structure, and there is no apparent connection between them. It seems that existing methods and the authors' proposed methods are randomly mixed together, leading to a lack of clarity and logical flow.\n4.\tDomain-specific claims and evaluation: The authors assert that their method is domain-specific, claiming that the proposed components could help adapt the model to industrial or medical domains. However, it is unclear whether the trained model can be used simultaneously on both industrial datasets, i.e, MVTec and Visa. The authors should clarify why they separately evaluate these datasets. They are suggested to follow the approach of UniAD [9] to demonstrate if one model can effectively detect anomalies across multiple categories within a single dataset.\n\nReferences: \n[a] Focus your distribution: Coarse-to-fine non-contrastive learning for anomaly detection and localization. ICME 2022.\n[b] Registration-based Few-Shot Anomaly Detection. ECCV 2022.\n", "limitations": "N/A", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The technical details are well illustrated in the paper.\n2. Ablation studies show the effectiveness of the proposed components.", "weaknesses": "1.\tThe paper lacks novelty and resembles more of a technical report than a research paper. The authors summarize their contributions in Lines 68-78, which primarily consist of detailed technical modifications. The proposed method heavily relies on existing methods UniAD and RD4AD, with some specific improvements such as generating contrastive pairs and utilizing a GAP-mimicking objective function.\n2.\tThis paper has room for improvement in terms of its writing. In Lines 42-59, the authors introduce several components aimed at enhancing performance. However, the relationships between these components are not clearly established, and the motivations behind their inclusion are not adequately explained.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for all your valuable suggestion and concerns.\n\n__1. (Weakness 1 & Question 1) The paper lacks novelty. The proposed method is deeply based on UniAD and RD4AD, with some technical improvement to adapt to the data from different domains. Unclear contributions compared to existing methods [a,b]__\n\nFirst, we would like to say that our method is not based on UniAD, where transformer decoder, neighbor masking, and feature jitter are proposed to address the unified UAD setting (all-class-one-model). The only thing in common is that they are both feature-reconstruction-based, which is a big paradigm of UAD. It is fair to say that our method is based on RD4AD, which serves as our baseline and starting point of exploration.\n\nThough [a] and [b] utilize stop-grad of contrastive learning (CL) to avoid pattern collapse, they are not contrastive methods (nor did the authors state they are), they did not analyze how stop-grad helps, and their motivation is not to optimize pre-trained encoder. In contrast, we further dive into how stop-grad helps to prevent pattern collapse by probing the feature diversity of the encoder, as discussed in Sec.2.3 and Sec.2.4. In addition, stop-grad is only one component of CL, but ReContrast introduces three cohesive elements of CL (global similarity, stop-grad, and contrastive pairs) to build a complete 2-D contrastive paradigm. In the final version, we will cite [a] and [b] to review the use of stop-grad in UAD tasks.\n\nSecond, we would like to claim that it is not the simple assembling of existing techniques that makes our method specific to a specific dataset. It is the motivation to optimize and reorient the feature encoder towards each target UAD training set that makes our method domain-specific to whatever it is trained on, instead of using a universal ImageNet-pretrained frozen encoder even if it is not discriminative on the target dataset.\n\nUnder this global motivation, we found it difficult to optimize the pre-trained encoder during feature reconstruction training accompanied by pattern collapse, instability, and identical shortcut, which is why all previous methods freeze the pre-trained encoder without any adaption during training. Inspired by the structural similarity between CL and feature reconstruction, we bring three essences of CL into feature reconstruction to enable smooth optimization of encoder. We would like to say that the three proposed components, i.e., global cosine distance, stop gradient, and augmentation-free contrastive pair, are NOT A+B+C-like combinations, but three cohesive components strongly correlated and inspired by CL. In addition, except stop-grad, the elements of CL cannot be directly utilized in feature reconstruction UAD. For example, because the global representation after global average pooling (GAP) in CL messes locality, we propose a novel global cosine distance that calculates the distance in 1-D while still maintaining the pixel-to-pixel correspondence. Because the image augmentation of CL cannot be used in UAD scenario, we propose an augmentation-free method to construct two views of one image.\n\n__2. (Weakness 2 & Question 3) This paper has room for improvement in terms of its writing. The paper resembles more of a technical report than a research paper.__\n\nInspired by the narrative of a number of previous arts (SimSiam, MoCo v3), we tried a writing logic of “spot a challenge, connection to CL, address the problem, analysis the mechanism”, instead of the conventional manner of “methods in Method, experiments in Experiment”. This writing logic involves bringing some preliminary experiments in the Method section and analysis of training dynamics (feature diversity, loss landscape). We believe such an exploration-like narrative illustrates the motivation and intuition of each proposed component better. \n\nTo be specific, in Introduction, we point out that the three proposed components are cohesively inspired by the elements of CL, which is the reason we name it ReContrast. In the Method section, we found it too intuitive and confusing to directly propose three components without explaining why to introduce and how they work. Therefore, in each section, we first present an obstacle of optimizing encoder in feature reconstruction, which is the motivation of the proposed components. Second, we target an element of CL that address a similar challenge in representation learning. Third, we adapt the elements to fit in the 2-D feature reconstruction paradigm. And last, we analyze the mechanism by inspecting feature diversity or loss-parameter landscape. \n\nFurthermore, some reviewers (especially #3 xS4v) acknowledge this narrative, and we will be very grateful if you feel comfortable with this paper structure. In the final version, we will further emphasize each method’s cohesiveness, motivation, and specific relation to contrastive learning inspired by your valuable comments.\n\n__3. (Question 2) While this paper heavily relies on UniAD [9], it fails to provide any comparisons.__\n\nSince our method is not based on UniAD (please refer to the reply of Question 1) and not evaluated under unified setting, we compared the separate setting (one-class-one-model) version of UniAD, i.e. ADTR [3]. ADTR is the work of the same authors of UniAD with exactly the same architecture but without the components addressing the unified setting (all-class-one-model), which is harmful to separate setting. ADTR (I-AUROC=97.5%) is better than UniAD (96.6%) under the separate setting, which we believe is much fairer to be compared with our method.\n\n__4. (Question 4) They are suggested to follow the unified-setting of UniAD [9] (all-class-one-model).__\n\n__Because of the character limitation of each response, we place the response in the global reply. Please check.__\n\n"}, {"review_id": "tVaBXn9UKv", "reviewer": "Reviewer_S3Cv", "summary": "The paper presents an approach, called ReContrast, for domain-specific anomaly detection and segmentation. The authors aim to overcome the transfer limitations of existing methods by addressing and mitigating the challenges associated with the use of pre-trained encoders from natural image domains. They propose a combination of the reverse distillation approach and contrastive learning to optimize the entire network and align it with the target domain.\n\nIn the paper, the authors systematically explain their approach, highlighting three key elements: the introduction of a new optimization objective called global cosine distance to enhance training stability, the utilization of stop-gradient operation to prevent pattern collapse, and the integration of contrastive learning (without the use of negative samples). By integrating these elements into their feature reconstruction framework, the authors improve the network's transfer ability for anomaly detection tasks.\n", "questions": "The paper lacks of reported results in the multi-class setting, where anomaly detection is performed across multiple target classes simultaneously. This setting ([9] in the manuscript), reflects a more realistic scenario of anomaly detection in real-world applications. By learning the data distribution of multiple classes jointly, the detection task becomes more complex and challenging. However, the paper does not provide insights into the performance of the proposed ReContrast approach in this specific setting. What is the performance of the proposed ReContrast approach in this setting?\n\nI am willing to improve my score once these points (including weaknesses) are clarified.", "limitations": "Yes.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The authors address and analyze the limitations of the reverse distillation approach by incorporating simple yet effective modifications. This results in an improved anomaly detection method.\n2. The paper is well-written, and provides a clear and concise description of the proposed ReContrast framework and its underlying principles.\n3. The experimental evaluations conducted across diverse datasets and models demonstrate the practicality and effectiveness of the approach.\n", "weaknesses": "1. The authors handle training instability by selectively changing the batch normalization (BN) mode of the encoder based on performance on specific datasets or categories. This raises concerns about the proposed method's reliability and generalizability. Using the test set to modify the training process of problematic categories can introduce unintended biases and compromise the evaluation integrity. It would be more desirable to provide a systematic and principled approach to utilizing BN layers without relying on a separate validation set. This would ensure consistent and unbiased training across all datasets and categories. Consequently, it is important to report the performance of the proposed approach when utilizing the train and eval modes consistently for all datasets. Such analysis would provide valuable insights into the observed performance drops that the authors mentioned and help assess the robustness of the proposed method.\n\n2. Another limitation of the paper is the narrow focus on structural anomalies, as evidenced by the results presented in Table A6. While the proposed method performs well at detecting structural anomalies, it falls short when it comes to detecting logical anomalies. It is important to highlight this limitation in the main paper since unsupervised anomaly detection encompasses various anomaly types, including global anomalies like those found in datasets such as CIFAR-10 ([24-26] in the appendix). Clarifying the scope and limitations of the method for detecting different anomaly types will provide a more comprehensive understanding of its applicability in real-world scenarios.\n\n\\#### Update #### \\\nI've increased my rating slightly in light of the response.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your insightful comments and suggestions.\n\n__1. The authors handle training instability by selectively changing the batch normalization (BN) mode of the encoder based on performance on specific datasets or categories. This raises concerns about the proposed method's reliability and generalizability.__\n\nThanks for this valuable question. This instability issue actually lies in the optimization of neural networks.\n\nAs discussed in Appendix E line 134-144, one possible reason is the denominator of BN layer: “Because a UAD dataset contains only one type of category, there is a chance that a channel is barely activated or equally activated, driving the variance of BN to zero.”. We have preliminarily shown that we can reach a comparable result 99.33% on MVTecAD by replacing the batch variance smaller than a constant to pre-trained running_var for each channel.\n\nIn addition, we found that a number of training-based UAD methods (RD4AD[2], CFA[14], SimpleNet[15]) also suffer from training instability when running their code. Whether the last epoch (for reporting results) is in the middle of a loss spike and performance dip is strongly related to random seed. The phenomenon of training instability is also observed in many other disciplines. However, validation sets are allowed which mitigate the effect of instability. In [a], the authors attribute the training instability on large-scale datasets to the historical gradient estimation of Adam optimizer. Accordingly, we tried a strategy to reset the running state (momentum and velocity) of Adam every 1000 iterations. Without manually selecting BN mode, it yields I-AUROC of 99.37% on MVTec and 97.3% on VisA, which is comparable to the 99.45% and 97.5% of our reported result and still outperforms previous SOTAs.\n\nSuch approach, analysis, and results will be added into the main paper for fair comparison and to provide insights for training UAD models under potential instability.\n\n[a] Molybog, Igor, et al. \"A theory on adam instability in large-scale machine learning.\" arXiv preprint arXiv:2304.09871 (2023).\n\n__2. Another limitation of the paper is the narrow focus on structural anomalies. It is important to highlight this limitation in the main paper since unsupervised anomaly detection encompasses various anomaly types, including logical anomalies and global anomalies like those found in datasets such as CIFAR-10.__\n\nThank you for this insightful comment.\n\nDue to the page limitation of the manuscript, we discuss the field of UAD in Appendix A lines 7-14. We distinguish UAD from one-class classification (OCC, or Semantic AD). In our UAD, normal samples and anomalous samples are semantically the objects except local detects, e.g. good cable v.s. spoiled cable, healthy eye v.s. diseased eye. In OCC, normal samples and anomalous samples are semantically in different categories, e.g. cat v.s. other animals (like in CIFAR-10). The slight difference in task setting makes the focus of corresponding methods different. Most methods for UAD attempt to detect anomalies by segmenting anomalous regions, while methods for OCC detect anomalies based on the representation deviation of the whole image. The cross-using of methods in different settings causes unsatisfactory performances because they address different problems with different anomaly assumptions, as discussed in Appendix lines 48-51. In this work, we mainly focus on UAD that detects regional defects (most common in industrial inspection and medical disease screening).  We will add these claims of scope at the beginning of the Method section in the main paper for clarity.\n\n__3. The unified setting (all-class-one-model) of UniAD, reflects a more realistic scenario of anomaly detection. What is the performance of the proposed ReContrast approach in this setting?__\n\nThank you for your valuable suggestion. We run ReContrast under the unified setting on MVTec-AD and VisA. The model is optimized on the whole dataset for 5000 iterations. On MVTec AD, our method produces an I-AUROC of __98.2%__, exceeding the previous SOTA 96.5% of UniAD [9] by a large 1.7%. The results on VisA are presented in global reply due to page limitation. The results under the unified setting will be added in the final version following your valuable suggestion.\n\n__I-AUROC (%) on MVTec AD under unified setting__\n|     |  |   |   |\n|:----------:|:-----:|:-----:|:----------:|\n| Method     | RD4AD | UniAD | ReContrast |\n| Carpet     | 98.3  | __99.8__  | 98.3 |\n| Grid       | __99.0__  | 98.2  | 98.9 |\n| Leather    | __100__   | __100__   | __100__ |\n| Tile       | 98.1  | 99.3  | __99.5__       |\n| Wood       | 99.3  | 98.6  | __99.7__       |\n| Bottle     | 79.9  | 99.7  | __100__        |\n| Cable      | 86.8  | 95.2  | __95.6__       |\n| Capsule    | 96.3  | 86.9  | __97.3__       |\n| Hazelnut   | __100__   | 99.8  | __100__        |\n| MetalNut   | 99.8  | 99.2  | __100__        |\n| Pill       | 92.8  | 93.7  | __96.3__       |\n| Screw      | 96.5  | 87.5  | __97.2__       |\n| Toothbrush | __97.5__  | 94.2  | 96.7       |\n| Transistor | 93.3  | __99.8__  | 94.5       |\n| Zipper     | 98.6  | 95.8  | __99.4__ |\n| Mean       | 95.8  | 96.5  | __98.2__       |\n\n\nThe reason why we did not include such results in the initial submission is that we consider this setting is actually less practical than the conventional separate setting (one-class-one-model). For example, in medical images, it is unreasonable to train a unified model for all modalities including CT, MRI, fundus images, etc, when the categories of images are easy to obtain. Similarly in industrial defect detection, the categories of the object are determined for each production line. The main benefit of a unified model is to save disk memory for model parameters, which is not the focus of this paper.\n"}]

Author rebuttal:
__Note: *Revised Figures are in the one-page PDF. Please check "pdf" below.*__
 
 
__*The following area is only the extension of response to Reviewer 4 (e2VQ) due to page limitation.*__

__Reply to Review 4 (e2VQ). 4. (Question 4) They are suggested to follow the unified setting of UniAD [9] (all-class-one-model).__

We run our ReContrast under the unified setting on MVTec AD and VisA. The model is optimized on each whole dataset for 5000 iterations. On MVTec AD, our method produces an I-AUROC of __98.2%__, exceeding the previous SOTA 96.5% of UniAD by a large 1.7%. On VisA, both our ReContrast (__95.1%__) and baseline RD4AD (92.7%) outperform the previous SOTA method UniAD (88.8%), while the proposed method exceeds RD4AD by 2.4%. Detailed results on each category are presented in the following table. The results under this unified setting will be added in the final version following your constructive suggestion.


__I-AUROC (%) on MVTec AD under unified setting__
|     |  |   |   |
|:----------:|:-----:|:-----:|:----------:|
| Method     | RD4AD | UniAD | ReContrast |
| Carpet     | 98.3  | __99.8__  | 98.3 |
| Grid       | __99.0__  | 98.2  | 98.9 |
| Leather    | __100__   | __100__   | __100__ |
| Tile       | 98.1  | 99.3  | __99.5__       |
| Wood       | 99.3  | 98.6  | __99.7__       |
| Bottle     | 79.9  | 99.7  | __100__        |
| Cable      | 86.8  | 95.2  | __95.6__       |
| Capsule    | 96.3  | 86.9  | __97.3__       |
| Hazelnut   | __100__   | 99.8  | __100__        |
| MetalNut   | 99.8  | 99.2  | __100__        |
| Pill       | 92.8  | 93.7  | __96.3__       |
| Screw      | 96.5  | 87.5  | __97.2__       |
| Toothbrush | __97.5__  | 94.2  | 96.7       |
| Transistor | 93.3  | __99.8__  | 94.5       |
| Zipper     | 98.6  | 95.8  | __99.4__ |
| Mean       | 95.8  | 96.5  | __98.2__       |

__I-AUROC (%) on VisA under unified setting__
|           |        |       |            |
|:---------:|:------:|:-----:|:----------:|
| Method    | RD4AD  | UniAD | ReContrast |
| candle    | 93.6   | 94.0    | __96.3__    |
| capsules  | 67.8   | 72.2  | __77.7__       |
| cashew    | __94.6__   | 94.4  | 94.5       |
| chewingum | 95.3   | 98.1  | __98.6__       |
| fryum     | 94.4   | 86.3  | __97.3__       |
| macaroni1 | 97.2   | 88.5  | __97.6__       |
| macaroni2 | 86.2   | 75.3  | __89.5__       |
| pcb1      | __96.5__   | 90.1  | __96.5__       |
| pcb2      | 93.0     | 88.4  | __96.8__       |
| pcb3      | 94.5   | 84.1  | __96.8__       |
| pcb4      | __100__    | 99.0    | 99.9       |
| pip_fryum | __99.7__   | 94.7  | 99.3       |
| Mean      | 92.7   | 88.8  | __95.1__       |


The reason why we did not include such setting in the initial submission is that we consider this setting is actually less practical than the conventional separate setting. For example, in medical images, it is unreasonable to train a unified model for all modalities including CT, MRI, and fundus images, when the categories of images are easy to obtain. Similarly in industrial defect detection, the categories of the object are determined for each production line. The main benefit of a unified model is to save disk memory for model parameters, which is not the focus of this paper.



Meta-review: This paper presents a novel contrastive learning paradigm, namely ReContrast, that aims to improve domain-specific unsupervised anomaly detection. It helps to overcome challenges such as pattern collapse, training instability, and the use of identical shortcuts that can hamper the learning process. The effectiveness of this approach has been validated through extensive experiments on various datasets, including MVTec AD, VisA, and three medical image datasets, showing superior performance compared to existing methods. 

Initially, there were several criticisms raised by all reviewers, due to the ambiguity and uncertainty in the results. It also affected the reviewers' confidence in the validity of the results. The rebuttal could address many of the issues and after careful consideration and discussions after the rebuttal, the AC finds merits in the paper. If accepted, the authors are encouraged to address all issues raised by the reviewers.
Decision: Accept (poster)