SlowGuess/NeurIPS_2023_OCR huggingface dataset:
In this section, we evaluate the proposed method on two popular UAD applications, i.e., industrial defect inspection and medical disease screening. We refer to our complete approach (ReContrast  $+\mathcal{L}_{global-hm}$ ) as ReContrast and Ours in the Experiments section for simplicity.
Datasets. MVtec AD is the most widely used industrial defect detection dataset, containing 15 categories of sub-datasets (5 textures and 10 objects). VisA is a challenging industrial defect detection dataset, containing 12 categories of sub-datasets. OCT2017 is an optical coherence tomography dataset [21]. APTOS is a color fundus image dataset, available as the training set of the 2019 APTOS blindness detection challenge [22]. ISIC2018 is a skin disease dataset, available as task 3 of ISIC2018 challenge [23]. Detailed information is presented in Appendix B.
Implementation. WideResNet50 [30] pre-trained on ImageNet [1] is utilized as the encoder by default. AdamW optimizer [31] is utilized with  $\beta = (0.9, 0.999)$  and weight decay=1e-5. The learning rates of new (decoder and bottleneck) and pre-trained (encoder) parameters are 2e-3 and 1e-5, respectively. The network is trained for 3,000 iterations on VisA, 2,000 on MVTec AD and ISIC2018, and 1,000 on APTOS and OCT2017. The  $\alpha$  in equation (5) linearly rises from -3 to 1 in the first
Table 1: Anomaly detection performance on MVTec AD, measured in I-AUROC  $(\%)$
ADTR [3]	RD4AD [2]	CFlowAD [34]	CutPaste [35]	DRAEM [36]	PaDiM [10]	PatchCore [11]	Ours
Carpet	98.7	98.9	98.7	93.9	97.0	99.8	98.7	99.8
Grid	95.0	100	99.6	100	99.9	96.7	98.2	100
Leather	98.1	100	100	100	100	100	100	100
Tile	93.8	99.3	99.9	94.6	99.6	98.1	98.7	99.8
Wood	91.2	99.2	99.1	99.1	99.1	99.2	99.2	99.0
Text. Avg.	95.4	99.5	99.5	97.5	99.1	98.9	99.0	99.7
Bottle	98.0	100	100	98.2	99.2	99.9	100	100
Cable	96.8	95	97.6	81.2	91.8	92.7	99.5	99.8
Capsule	99.1	96.3	97.7	98.2	98.5	91.3	98.1	97.7
Hazelnut	98.6	99.9	100	98.3	100	92.0	100	100
MetalNut	97.0	100	99.3	99.9	98.7	98.7	100	100
Pill	98.3	96.6	96.8	94.9	98.9	93.3	96.6	98.6
Screw	99.3	97.0	91.9	88.7	93.9	85.8	98.1	98.0
Toothbrush	98.5	99.5	100	99.4	100	96.1	100	100
Transistor	97.9	96.7	95.2	96.1	93.1	97.4	100	99.7
Zipper	97.2	98.5	98.5	99.9	100	90.3	99.4	99.5
Obj. Avg.	98.1	98.0	97.7	95.5	97.4	93.8	99.2	99.3
All Avg.	97.2	98.5	98.3	96.2	98.0	95.4	99.1	99.5
Table 2: Anomaly segmentation performance on MVTec AD, measured in P-AUROC/AUPRO (%).
RD4AD [2]	DRAEM [36]	SPADE [37]	PaDiM [10]	PatchCore [11]	Ours
Text. Avg.	97.7 / 95.0	97.5 / 92.1	92.9 / 88.4	96.9 / 93.1	97.5 / 93.6	98.0 / 96.2
Obj. Avg.	97.9 / 93.4	98.3 / 93.3	97.6 / 93.4	97.8 / 91.6	98.4 / 93.3	98.6 / 94.8
All Avg.	97.8 / 93.9	97.8 / 92.5	96.0 / 91.7	97.5 / 92.1	98.1 / 93.4	98.4 / 95.2
Table 3: Anomaly detection and segmentation performance on VisA (%)
RD4AD [2]	DRAEM [36]	SPADE [37]	PaDiM [10]	PatchCore [11]	Ours
I-AUROC Avg.	96.0	88.7	92.1	89.1	95.1	97.5
P-AUROC Avg.	90.1	93.5	85.6	98.1	98.8	98.2
AUPRO Avg.	70.9	72.4	65.9	85.9	91.2	92.6
one-tenth iterations and keeps 1 for the rest training. More experimental details and environments are presented in Appendix C.
Metrics. Image-level anomaly detection performance is measured by the Area Under the Receiver Operator Curve (I-AUROC). F1-score (F1) and accuracy (ACC) are adopted for medical datasets following [32]. The operating threshold of F1 and ACC is determined by the optimal value of F1. For anomaly segmentation, pixel-level AUROC (P-AUROC) and Area Under the Per-Region-Overlap (AUPRO) [33] are used as the evaluation metrics. AUPRO is more meaningful than P-AUROC because of the unbalanced amount of normal and anomalous pixels [20].
3.2 Anomaly Detection and Segmentation on Industrial Images
Anomaly detection results on MVtec AD are shown in Table 1. Our approach achieves a superior average I-AUROC of  $99.5\%$ . In the aspect of error (100%–I-AUROC), we achieve only  $0.5\%$ , reducing the previous SOTA error of PatchCore (0.9%) by relatively  $44\%$ . Anomaly segmentation results are shown in Table 2 (average of 15 categories). Our ReContrast achieves SOTA performance measured by both P-AUROC and AUPRO. In AUPRO, we produce  $95.2\%$ , exceeding the previous SOTA RD4AD by  $1.3\%$ . We also present SOTA performances on texture (Text. Avg.) and object (Obj. Avg.) categories, respectively.
Table 4: Anomaly detection performances under unified multi-class setting on MVTec AD and VisA, measured in I-AUROC(%)
MVTec AD	RD4AD [2]	UniAD [9]	Ours	VisA	RD4AD [2]	UniAD [9]	Ours
Carpet	98.3	99.8	98.3	candle	93.6	94.6	96.3
Grid	99.0	98.2	98.9	capsules	67.8	74.3	77.7
Leather	100	100	100	cashew	94.6	92.6	94.5
Tile	98.1	99.3	99.5	chewingum	95.3	98.7	98.6
Wood	99.3	98.6	99.7	fryum	94.4	90.2	97.3
Bottle	79.9	99.7	100	macaroni1	97.2	91.5	97.6
Cable	86.8	95.2	95.6	macaroni2	86.2	83.6	89.5
Capsule	96.3	86.9	97.3	pcb1	96.5	94.3	96.5
Hazelnut	100	99.8	100	pcb2	93.0	92.5	96.8
MetalNut	99.8	99.2	100	pcb3	94.5	89.8	96.8
Pill	92.8	93.7	96.3	pcb4	100	99.3	99.9
Screw	96.5	87.5	97.2	pip_fryum	99.7	97.3	99.3
Toothbrush	97.5	94.2	96.7
Transistor	93.3	99.8	94.5
Zipper	98.6	95.8	99.4
All Avg.	95.8	96.5	98.2	All Avg.	92.7	91.5	95.1
Anomaly detection and segmentation results on VisA are shown in Table 3 (average of 12 categories). Our approach achieves a superior average I-AUROC of  $97.5\%$ , exceeding previous SOTA RD4AD by  $1.5\%$ . In AUPRO, we produce  $92.6\%$ , outperforming previous SOTA PatchCore by  $1.4\%$ .
3.3 Multi-Class Anomaly Detection with A Unified Model
Conventional UAD settings train separate models for different object categories. In UniAD [9], the authors proposed to train a unified model for multiple classes. We evaluate our ReContrast under the unified setting on MVTec AD and VisA. Our model is trained for 5,000 iterations on all images containing all 15 MVTec AD categories (or 12 categories in VisA) and evaluated on each category following [9]. The results are presented in Table 4. On MVTec AD, our method produces an I-AUROC of  $98.2\%$ , exceeding the previous SOTA  $96.5\%$  of UniAD by a large  $1.7\%$ . On VisA, both our ReContrast  $(95.1\%)$  and baseline RD4AD  $(92.7\%)$  outperform the previous multi-class SOTA UniAD  $(91.5\%)$ , while the proposed method exceeds RD4AD by  $2.4\%$ .
3.4 Anomaly Detection on Medical Images
The experimental results on three medical image datasets are presented in Table 5. Apart from industrial UAD methods, we also compare our ReContrast with approaches designed for medical images (f-AnoGan [7], GANomaly [8], AE-flow [32]), and contemporaneous methods proposed for better domain adaption ability (CFA [14], SimpleNet [15]). We reproduce the methods that were not evaluated on the corresponding dataset if they are open-sourced. Our method achieves the best results in all metrics on APTOS and ISIC2018, and comparable best results on the relatively easy OCT2017, demonstrating that ReContrast can directly adapt to various medical image modalities.
To verify the effect of the proposed elements, we conduct thorough ablation studies on MVTec AD and APTOS. Because some configurations are unstable during training on some categories, we report the last iteration's performances and those of the best checkpoint during training (best by I-AUROC, evaluate every 250 iterations). The results are reported in Table 6. The use of  $\mathcal{L}_{global}$  (Config.B) boosts the performance of Config.A trained by  $\mathcal{L}_{region}$ . Directly optimizing the encoder and decoder together (Config.C) causes a great degeneration because of pattern collapse, while either stop-gradient (Config.D) or contrastive pairs (Config.C+cp) recovers some of it. By introducing both (Config.E), the performance is further improved to exceed the frozen encoder baseline. Finally, training with  $\mathcal{L}_{global-hm}$  yields SOTA performances (Ours). Meanwhile,  $\mathcal{L}_{global-hm}$  can also improve the baseline reconstruction method alone (Config.B+hm). More ablation studies, experimental results with error bars, limitations, and qualitative visualization are presented in Appendix.
Table 5: Anomaly detection performances on medical image datasets  $(\%)$
APTOS	OCT2017	ISIC2018
I-AUROC	F1	ACC	I-AUROC	F1	ACC	I-AUROC	F1	ACC
f-AnoGAN[7]	89.64	89.71	85.23	83.02	88.64	82.13	79.80	67.03	68.39
GANomaly[8]	83.72	85.85	79.37	90.52	91.09	86.16	72.54	60.95	56.99
PaDiM[10]	77.17	85.47	77.24	96.88	95.24	92.80	82.13	72.04	73.06
RD4AD[2]	92.43	90.65	86.44	99.25	97.79	96.70	85.09	74.53	78.76
PatchCore[11]	90.45	90.18	85.57	99.61	98.34	97.50	78.94	68.57	71.50
AE-flow[32]	N/A	N/A	N/A	98.15	96.36	94.42	87.79	80.56	84.97
CFA[14]	94.21	94.39	92.03	98.01	96.40	94.70	81.31	72.31	74.61
SimpleNet[15]	93.42	91.16	87.27	98.50	96.91	95.40	82.17	69.82	73.59
Ours	97.51	95.27	93.35	99.60	98.53	97.80	90.15	81.12	86.01
Table 6: Ablation study on MVtec AD and APTOS. Reported in last/best
\( \mathcal{L}_{global} \)	optim. encoder	stop grad.	contrast pairs	hard mining	MVTec AD	APTOS
I-AUROC	AUPRO	I-AUROC
Config.A	95.31/97.55	93.34/94.05	90.12/90.50
Config.B	✓	98.86/99.07	94.51/94.59	92.49/93.62
Config.B+hm	✓	✓	99.00/99.12	94.86/94.91	93.87/94.36
Config.C	✓	✓	91.54/95.96	88.24/92.14	90.71/91.06
Config.D	✓	✓	✓	94.64/97.07	84.11/87.38	93.06/95.66
Config.C+cp	✓	✓	✓	97.59/97.88	93.76/93.92	92.68/92.68
Config.E-gl	✓	✓	✓	98.93/99.26	94.65/94.71	96.39/96.39
Config.E	✓	✓	✓	✓	99.13/99.34	94.59/94.60	97.32/97.43
Ours	✓	✓	✓	✓	✓	99.45/99.52	95.20/95.29	97.51/97.51
In this study, we propose a novel contrastive learning paradigm, namely ReContrast, for domain-specific unsupervised anomaly detection. It addresses the transfer ability of the pre-trained encoder by jointly optimizing all parameters end-to-end. The key elements of contrastive learning are elegantly embedded in epistemic UAD method to avoid pattern collapse, training instability, and identical shortcut. Extensive experiments on MVTec AD, VisA, and three medical image datasets demonstrate our superiority. The idea of optimizing encoders can further boost the application of UAD methods on more image modalities that are far from natural image domain.
Scope of Application In this work, we mainly focus on UAD that detects regional defects (most common in practical applications like industrial inspection and medical disease screening), which is distinguished from one-class classification (OCC, or Semantic AD). In our UAD, normal and anomalous samples are semantically the same objects except local detects, e.g. good cable v.s. spoiled cable. In OCC, normal samples and anomalous samples are semantically different, e.g. cat v.s. other animals. More details are discussed in Appendix E.
Training Instability Our method still suffers some extent of training instability. Because of the absence of validation sets in UAD settings, whether the last epoch (for reporting results) is in the middle of a loss spike and performance dip is related to random seeds. We found that a number of UAD methods (RD4AD[2], CFA[14], SimpleNet[15]) are also subject to training instability when running their code. We discuss the reason and solutions to mitigate this effect in Appendix E.
This research work is supported by the National Natural Science Foundation of China (NSFC) (GrantNo. 82072007), Beijing Natural Science Foundation (Grant No. IS23112), and Beijing Institute of Technology Research Fund Program for Young Scholars.
[1] O. Russakovsky et al., "ImageNet Large Scale Visual Recognition Challenge," International Journal of Computer Vision, vol. 115, no. 3, 2015.
[2] H. Deng and X. Li, "Anomaly Detection via Reverse Distillation from One-Class Embedding," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9737-9746, 2022.
[3] Z. You, K. Yang, W. Luo, L. Cui, Y. Zheng, and X. Le, "ADTR: Anomaly Detection Transformer with Feature Reconstruction," arXiv preprint arXiv:2209.01816, 2022.
[4] Y. Shi, J. Yang, and Z. Qi, "Unsupervised anomaly segmentation via deep feature reconstruction," Neurocomputing, vol. 424, 2021.
[5] M. Salehi, N. Sadjadi, S. Baselizadeh, M. H. Rohban, and H. R. Rabiee, “Multiresolution knowledge distillation for anomaly detection,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2021.
[6] G. Wang, S. Han, E. Ding, and D. Huang, "Student-teacher feature pyramid matching for anomaly detection," in The British Machine Vision Conference (BMVC), 2021.
[7] T. Schlegl, P. Seeböck, S. M. Waldstein, G. Langs, and U. Schmidt-Erfurth, "f-AnoGAN: Fast unsupervised anomaly detection with generative adversarial networks," Medical Image Analysis, vol. 54, 2019.
[8] S. Akcay, A. Atapour-Abarghouei, and T. P. Breckon, "GANomaly: Semi-supervised Anomaly Detection via Adversarial Training," in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11363 LNCS, 2019.
[9] Z. You et al., "A unified model for multi-class anomaly detection," in Advances in Neural Information Processing Systems (S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, eds.), vol. 35, pp. 4571-4584, Curran Associates, Inc., 2022.
[10] T. Defard, A. Setkov, A. Loesch, and R. Audigier, "PaDiM: A Patch Distribution Modeling Framework for Anomaly Detection and Localization," in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12664 LNCS, 2021.
[11] K. Roth, L. Pemula, J. Zepeda, B. Schölkopf, T. Brox, and P. Gehler, “Towards total recall in industrial anomaly detection,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14318–14328, 2022.
[12] J. Yi and S. Yoon, "Patch SVDD: Patch-Level SVDD for Anomaly Detection and Segmentation," in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12627 LNCS, 2021.
[13] O. Rippel, P. Mertens, and D. Merhof, "Modeling the distribution of normal data in pre-trained deep features for anomaly detection," in Proceedings - International Conference on Pattern Recognition, 2020.
[14] S. Lee, S. Lee, and B. C. Song, "Cfa: Coupled-hypersphere-based feature adaptation for target-oriented anomaly localization," IEEE Access, vol. 10, pp. 78446-78454, 2022.
[15] Z. Liu, Y. Zhou, Y. Xu, and Z. Wang, "Simplenet: A simple network for image anomaly detection and localization," arXiv preprint arXiv:2303.15140, 2023.
First, we give the formulation of unsupervised anomaly detection (UAD). Let  $\mathcal{D}^{train} = \{(x_i,y_i)\}_{i=1}^N$ ,  $y_i \in \{-\}$  be the training set of normal samples, where  $x_i$  is the  $i^{th}$  sample with its label  $y_i$ . During test,  $\mathcal{D}^{test} = \{(x_i,y_i)\}_{i=1}^M$ ,  $y_i \in \{-, +\}$  ( $-$  for normal,  $+$  for anomalous) contains both normal and anomalous images. Our goal is to train a model using  $\mathcal{D}^{train}$  to identify  $(x, +)$  from  $(x, -)$  in  $\mathcal{D}^{test}$ , while localizing anomalous regions at the same time.
Epistemic methods are based on the assumption that the networks respond differently during inference between seen input and unseen input. Within this paradigm, pixel reconstruction methods assume that the networks trained on normal images can reconstruct anomaly-free regions well, but poorly for anomalous regions [38]. Auto-encoder (AE) [39; 38], variational auto-encoder (VAE) [40; 41], or generative adversarial network (GAN) [42; 7; 8] are used to restore normal pixels. However, pixel reconstruction models may also succeed in restoring unseen anomalous regions if they resemble normal regions in pixel values or the anomalies are barely noticeable [2; 9]. Therefore, feature reconstruction is proposed to construct features of pre-trained encoders instead of raw pixels [2; 3; 4]. To prevent the whole network from converging to a trivial solution, the parameters of the encoders are frozen during training [9]. In feature distillation [2; 5; 6], the student network is trained from scratch to mimic the output features of the pre-trained teacher network with the same input of normal images, also based on the similar hypothesis that the student trained on normal samples only succeed in mimicking features of normal regions.
Pseudo-anomaly methods generate handcrafted defects on normal images to imitate anomalies, converting UAD to supervised classification [35] or segmentation task [36]. Specifically, CutPaste [35] simulates anomalous regions by randomly pasting cropped patches of normal images. DRAEM [36] constructs abnormal regions using Perlin noise as the mask and another image as the additive anomaly. SimpleNet [15] introduces anomaly by injecting Gaussian noise in the pre-trained feature space. These methods deeply rely on how well the pseudo anomalies match the real anomalies [2], which makes it hard to generalize to different datasets.
Feature memory & modeling methods [10; 11; 12; 13; 14] memorize all (or their modeled distribution) normal features extracted by networks pre-trained on large-scale datasets and match them with test samples during inference. Since these methods require memorizing, processing, and matching nearly all features from training samples, they are computationally expensive in both training and inference, especially when the training set is large. There is a considerable semantic gap between large-scale natural images on which the frozen networks were pre-trained and the target UAD images in various modalities. Therefore, methods based on pre-trained networks struggle in transferring as the feature encoders are not optimized in the target domain.
There are a few studies addressing the adaptation problem in UAD settings. CFA [14] and SimpleNet[15] unanimously utilize a learnable linear layer to transform the output features of the frozen encoder. However, the linear layer may be insufficient for adaption and can hardly recover the domain-specific information in pre-trained features, especially when the target domain, e.g. CT and MRI, is remote from natural image domain. In OCC settings, a number of works focus on utilizing self-supervised pre-training approaches to learn a compact representation space for normal samples [43; 44; 45]. However, the performances are far from satisfactory in UAD tasks (I-AUROC under  $90\%$  on MVtec AD).
MVTec AD [19] is an industrial defect detection dataset, containing 15 sub-datasets (5 texture categories and 10 object categories) with a total of 3,629 normal images as the training set and 1,725 normal and anomalous images as the test set. Pixel-level annotations are available for anomalous images for evaluating anomaly segmentation. All images are resized to  $256 \times 256$ .
VisA [20] is an industrial defect detection dataset, containing 12 sub-datasets with a total of 9,621 normal and 1200 anomalous images. Pixel-level annotations are available for anomalous images for evaluating anomaly segmentation. We split it into training and test sets following the setting in [20]. All images are resized to  $256 \times 256$ .
OCT2017 is an optical coherence tomography (OCT) dataset [21]. The dataset is labeled into four classes: normal, drusen, DME (diabetic macular edema), and CNV (choroidal neovascularization). The latter three classes are considered anomalous. The training set contains 26,315 normal images and the test set contains 1,000 images. All images are resized to  $256 \times 256$ .
APTOS is a color fundus image dataset, available as the official training set of the 2019 APTOS blindness detection challenge [22]. The dataset contains 3,662 images with annotated grade 0-4 to indicate different severity of diabetic retinopathy, yielding 1,805 normal images (grade 0) and 1857 anomalous images (grade 1-4). We randomly selected 1,000 normal images as our training set and the rest 2,662 images as the test set. Images are preprocessed to crop the fundus region, and then resized to  $256 \times 256$ .
ISIC2018 is a skin disease dataset, available as task 3 of ISIC2018 challenge [23]. It contains seven classes. NV (nevus) is taken as the normal class and the rest of classes are taken as anomaly, following [32]. The training set contains 6705 normal images. The official validation set is used as our test set, which includes 193 images. Images are resized to  $256 \times 256$  and then center cropped into  $224 \times 224$  to remove redundant background. In addition, because the normal and anomalous images of ISIC2018 are different objects (lesions) instead of healthy objects and unhealthy objects, the task is more like one-class classification. Therefore, we take the mean (instead of maximum) value of  $S^{map}$  as the  $S^{img}$  in ReContrast and other reproduced methods.
C Complete Implementation Details
WideResNet50 [30] pre-trained on ImageNet [1] is utilized as the encoder by default. The decoder is the upside-down version of the encoder, exactly the same as RD4AD [2], i.e., the  $3 \times 3$  convolution with stride 2 at the beginning of each layer is replaced by a  $2 \times 2$  transposed convolution with stride 2. The bottleneck is also the same as RD4AD [2]. For each decoder layer, two 1x1 convolutions are used to project the feature map to reconstruct the frozen encoder and trained encoder, respectively. AdamW optimizer [31] is utilized with  $\beta = (0.9, 0.999)$  and weight decay=1e-5. The learning rates of the new (decoder and bottleneck) and pre-trained (encoder) parameters are 2e-3 and 1e-5 respectively. The network is trained for 3,000 iterations on each sub-dataset in VisA, 2,000 on each sub-dataset in MVTec AD and ISIC2018, and 1,000 on APTOS and OCT2017. The batch size is 16 for industrial datasets and 32 for medical datasets. The  $\alpha$  in (5) linearly rises from -3 to 1 in the first one-tenth iterations and keeps 1 for the rest of the training. By default, the batchnorm (BN) layers of encoder are set to train mode during training. Because training instability and performance drop are observed for some categories<sup>3</sup>, the BN of encoder is set to eval mode (use pre-trained statistics) for such datasets. The reason is discussed in Appendix E. A library<sup>4</sup> is used to plot the landscape in Figure 4. The distance and steps arguments are set to 1 and 50 respectively.
On MVtec AD and VisA, the results of comparison methods are taken from the original papers or the benchmark paper [46] that report their performances. On medical datasets, we take the results from [32] if available; otherwise, we reproduce the method with official code. For RD4AD, we use their official code, in which  $\mathcal{L}_{global}$  is already implemented instead of  $\mathcal{L}_{region}$ , as discussed in Section 2.1. Codes are implemented with Python 3.8 and PyTorch 1.12.0 CUDA 11.3. Experiments are run on NVIDIA GeForce RTX3090 GPUs (24GB).
D.1 Additional Experimental Results
In addition to the averaged segmentation performances in Table 2, we present the P-AUROC and AUPRO of each categories in MVtec AD in Table A1. The results in the main paper are reported
$^{3}$ Toothbrush, Leather, Grid, Tile, Wood, Screw in MVTec AD, cashew, pcb1 in VisA, and OCT2017.
<sup>4</sup>https://github.com/marcellodebernardi/loss-landscapes
with a single random seed following our baseline [2]. In Table A2, we report the mean and standard deviation of three runs with three different random seeds (1, 11, and 111). In addition to the averaged performances in Table 3, we present the I-AUROC, P-AUROC, and AUPRO of each category in VisA in Table A3.
Table A1: Anomaly segmentation performance of 15 subset (categories) of MVtec AD  $(\%)$ .
Carpet	Grid	Leather	Tile	Wood	Bottle	Cable	Capsule
P-AUROC	99.3	99.2	99.5	96.3	95.9	99.0	98.9	98.4
AUPRO	97.9	97.8	99.2	93.7	92.5	97.1	95.6	95.4
Hazelnut	MetalNut	Pill	Screw	Toothbrush	Transistor	Zipper
P-AUROC	99.1	98.7	99.1	99.6	99.2	95.4	98.1
AUPRO	95.9	94.4	97.7	98.6	95.0	82.3	94.9