SANFlow: Semantic-Aware Normalizing Flow for Anomaly Detection
Authors: Daehyun Kim
Sungyong Baik
Tae Hyun Kim
Venue: NeurIPS 2023 poster (2023)
Keywords: Anomaly Detection
Visual Anomaly Detection
Computer Vision
Normalizing Flow
Anomaly Localization

Abstract: Visual anomaly detection, the task of detecting abnormal characteristics in images, is challenging due to the rarity and unpredictability of anomalies. In order to reliably model the distribution of normality and detect anomalies, a few works have attempted to exploit the density estimation ability of normalizing flow (NF). However, previous NF-based methods have relied solely on the capability of NF and forcibly transformed the distribution of all features to a single distribution (e.g., unit normal distribution), when features can have different semantic information and thus follow different distributions. We claim that forcibly learning to transform such diverse distributions to a single distribution with a single network will cause the learning difficulty, limiting the capacity of a network to discriminate normal and abnormal data. As such, we propose to transform the distribution of features at each location of a given image to different distributions. In particular, we train NF to map normal data distribution to distributions with the same mean but different variances at each location of the given image. To enhance the discriminability, we also train NF to map abnormal data distribution to a distribution with a mean that is different from that of normal data, where abnormal data is synthesized with data augmentation. The experimental results outline the effectiveness of the proposed framework in improving the density modeling and thus anomaly detection performance.

SANFlow: Semantic-Aware Normalizing Flow for Anomaly Detection and Localization
Daehyun Kim!
Sungyong Baik² Tae Hyun Kim
Dept. of Artificial Intelligence',
Dept. of Data Science?, Dept. of Computer Science
Hanyang University [daehyun, dsybaik, taehyunkim//hanyaagg .c.krr
Abstract
Visual anomaly detection, the task of detecting abnormal characteristics in images, is challenging due to the rarity and unpredictability of anomalies. In order reliably model the distribution of normality and detect anomalies, few works have attempted exploit the density estimation ability of normalizing flow (NF). However, previous NF-based methods forcibly transform the distribution of all features into a single distribution (e.g., unit normal distribution), even when the features can have locally distinct semantic information and thus follow different distributions. We claim that foreibly learning to transform such diverse distributions to single distribution with single network will cause the learning difficulty, thereby limiting the capacity a network to discriminate between normal and abnormal data. As such, propose to transform the distribution of features at each location of a given input image to different distributions. Specifically, we train NF to map the feature distributions of normal data different distributions each location in the given image. Furthermore, to enhance the discriminability, we also train NF map the distribution of abnormal data to distribution significantly different from that of normal data. The experimental results highlight the efficacy of the proposed framework in improving the density modeling and thus anomaly detection performance.
1 Introduction
Abnormal events that deviate significantly from expected and typical characteristies are often un- wanted and are referred to as anomalies. The objective of anomaly detection is to identify such abnormal events. As such, anomaly detection applicable across domains (e.g., video-sure sureellluvllll,cce product defect detection, medical image analysis, etc.), where abnormal events can indicate or lead to critical issues. However, abnormal events occur rarely and can appear in various forms that cannot be known in advance, making it infeasible to collect large amount of abnormal data in many scenarios. The difficulty in collecting abnormal data has prevented anomaly detection from exploiting the recent breakthrough in supervised learning, leaving the field still challenging.
Acknowledging the difficulty of collecting abnormal data, many works have formulated anomaly detection unsupervised learning, one-class learning, or few-shot learning [ 45 49 g 47), in which the goal is to accurately model a distribution of normal data. Then, any data that deviates from the learned distribution of normal data is considered as anomaly. The approaches to unsupervised anomaly detection differ by how they recognize the differences between normal and abnormal data. Reconstruction-ben based methods leamn to reconstruct normal images and thus distinguish input images as abnormal if the reconstruction error is high [I 42 12 35). Mcanwhile, representation based approaches aim to learn feature space, where normal images are brought close to each other
"Correspondence to: Tae Hyun Kim <tachyunkim @ acc.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

complex region
Base distribution of normal data in simple region
Base distribution of abnormal region
Anomaly input
Estimated base distributions
Figure Motivation Overview: Different semantic features can be observed in different regions. Abnormal regions (red box) exhibit significantly distinct features compared to normal features (orange and blue boxes). Moreover, normal regions also display different semantic features based on the complexity of the region. Upon our observation, we propose Nemantic-Awand Normalizing Flow (SANFlow) that enables more accurate density estimation by adaptively embedding semantic feature distribution into corresponding but different base distributions.
Another line of works aims to simulate abnormalities through data augmentation [54] 26 On the other hand, recently emerging trend is employ normalizing flow (NF) for more reliable estimation of normal data
and abnormal images placed far away from normal images I
36
distribution [51] 20 37].
NF aims achieve better density estimation by learning sequence of reversible functions to map complex distribution of input data into a simple distribution (e.g.. normal distribution). The goal of NF aligns with that of anomaly detection in that reliable density estimation of normal samples leads to the accurate detection of anomalies. However, previous NF-based methods have solely relied on the capability of NF to model complex distributions simple ones. Although features different locations, scales and images can follow different distributions, previous works have attempted to map such multi-modal distribution simply a single normal distribution.
Considering that features at different locations and images may follow different distributions, we propose to adaptively embed the features each location of each image into different distributions accordingly. We make two contributions this work: First, we propose novel NF-based framework that transforms the feature distribution of normal data at each location into Gaussian distributions with zero mean but different variances. The variance at each location is estimated by an external network that is conditioned anomalies, we train NF to transform abnormal features into a distinct Gaussian distribution with a mean that is distant from the distributions of normal features. By embedding the features locally different distributions, we facilitate the training of NF. We demonstrate that the estimated variance is lower for simple regions such the background, while increases for more complex regions. A description of the proposed approach is shown in Figure T and evaluation examples by category are shown in Figure 2
image features and semanties. Second, enhance the detection of
The proposed framework demonstrates outstanding performance in both anomaly detection and localization. The strong empirical results underline the effectiveness of the proposed method in learning a more reliable density estimation, suggesting the importance of learning different target distributions for abnormal features and different normal features.
2
Related Work
Unsupervised anomaly detection tackles a challenging scenario, in which training images only consist of normal images that are free of anomalies. Under such challenging scenarios, standard supervised learning algorithms fail to work since training data available for only one class (normal data), therefore often referred to one-class learning. Methodologies that have emerged to overcome this issue largely differ by how they recognize the differences between normal data and abnormal data 9
Reconstruction based. Reconstruction based approaches are hinged on the motivation that reconstruc- tion models can reconstruct images well, if images are drawn from the same distribution as training images. Thus, these approaches distinguish images with high reconstruction error as abnormal. Sev- eral methods perform reconstruction by using auto-encoder architecture or the generator part of GAN models
57 6 55
[0 31 40).

---

Senr GU
carpet
Aipper
Figure 2: Anomaly localization results by our proposed framework MVTec benchmark dataset. From top to bottom: anomaly localization prediction, GT mask, anomaly score map and estimated variance map of input image. Variance maps are derived from mode values of and of each pixel and bright areas represent high variance and dark areas represent low variance values for corresponding distribution.
Data augmentation based. few methods have proposed to generate abnormal images through data augmentation that is based on either cut-out [13], random erasing 44), noise generation [54], or geometric transformations [17 16 21] In particular, CutPaste [26] introduces a new data augmenta- tion generate fine-grained and subtle abnormal local patches in normal images. Furthermore, other research areas use synthesized abnormal images to improve performance such as semantic
segmentation 23 19
or image classification [25]
Representation based. Another recent line research has focused on designing systems that can
leverage the strong representation power of pre-trained convolutional neural networks. [1] 36
46 56 34
Then, anomaly detection performed by measuring the distances between test image features and normal image features. The methods differ by how the features of normal image maintained or modeled and how the distance measured. SPADE [8] employs k-nearest neighbor (k-NN) for normal image features retrieval and feature correspondence for measuring the differences. To mitigate the complexity issue with nearest neighbor search, PatchCore [36] maintains a subset of features via coreset subsampling. Meanwhile, PaDiM [1]] models the normal features
multivariate Gaussian distribution and uses Mahalanobis distance 30 as a distance measure, instead of the k-NN algorithm, to improve the inference complexity.
Normalizing flow based. Recently, few works have shifted the attention to employing normalizing flow (NF) to better estimate the distribution of the features of normal patches. Through a sequence of learnable and invertible transformations, NF embeds the feature distribution into a simple normal distribution. DifferNet [37] is one of the first to employ NF to transform the distribution of CNN features to a normal distribution, where anomaly detection is performed at image level by measuring the likelihood of samples. Then, CFLOW-AD
20 extends NF-based methodology pixel-level anomaly detection. FastFlow [51] further improves the performance by utilizing normalizing flow in two-dimensional space, unlike previous mi-ddmmesioonl NF-based anomaly detection algorithms. In addition, CS-Flow (38) proposed new kind of NF model that jointly processes multiple feature maps across different scales. However, they forcefully embed different feature distributions of semantic patches to a single normal distribution, despite recent findings in classification that it is beneficial to transform images of different semanties into different distributions (2 3]. By contrast, we propose to train NF to transform the distributions at different locations to simple, but different distributions to better exploit the density estimation capability of NF by assuming locally varying base distributions.
3 Proposed method
In this section, we describe the overall pipeline of our framework, dubbed Semantic- Normal- izing Flow (SANFlow), as depicted in Figure Given an image æ, a pre-trained feature extractor is used to obtain features
V, at the i-th position. Then, we employ normalizing flow (NF) -- Vi + Z; to embed the feature distribution V, (a.k.a. target distribution) to simple and well- known distribution Z, (a.k.a. base distribution). To further guide NF to map feature distributions V; into different base distributions Z₂ upon semanties, we introduce an external network which allows to predict statistics of z; for each feature v. Moreover, similar to (26), employ data
3

---

augmentation to synthesize local anomalies within the input image in order to train NF to embed the distribution of anomaly features into a distribution Z that is distinct from the distributions of normal features 7
Then, we modify the standard objective function of NF account for different base distributions (Section[355 3.5). Finally, at test time, we use the trained NF to estimate the probability of given features to be normal, the negation of which is used as an anomaly score function detect anomalies in given image (Section 3.6
3.1 Background on normalizing flow
Normalizing flow (NF) 33 aims to learn function that transforms the latent variable z, which follows a simple and tractable base distribution Z, into the variable v, which follows a complex target distribution V. To do so, NF formulates g as the composition of invertible functions {gr d')_ ie.. g /" -f o.................... g², where L is the number of invertible functions. Such transformation of complex distribution into a simple one allows for efficient and exact density estimation, serving as a powerful tool in density estimation. By applying the change of variables theorem in sequence, the Log-likelihood of the target distribution pv (v) can be expressed in terms of the log-likelihood of the base distribution pz (z))
dg' dz'-1
py pv (v) = log p2)) Σ log det
(1)
where z' represents the resulting random variable after applying up to the 1-th function g' on z. Note that zd corresponds to z while =" corresponds to v. For efficient computation of the log-likelihood, functions {g' )/1 need to be invertible and have easy-to-compute Jacobian determinant. To meet such requirements, one of popular NF instances, RealNVP [14), formulated each function g' as an affine coupling block. In an affine coupling block, the input is first divided into two equal- equal-dimension parts. Then, each part undergoes affine transformation, whose parameters are generated by network conditioned on the other part. The transformed parts, in turn, are put back together via concatenation.
Then, the log- likelihood in Equation T is maximized (or the negative log-likelihood is minimized) to train the parameters of transformation function. A base distribution is commonly chosen as normal distribution (0, I) in previous NF-hased anomaly detection algorithms 20 51 37).
resulting in a loss function (i.e., negative log-likelihood) as follows:
|
log det. dg'
LNLL
dzl-l
3.2 Synthetic anomaly generation
In unsupervised anomaly detection scenarios, abnormal training images are typically not available. Therefore, we synthesize abnormal images facilitate training of NE. Abnormal images usually differ from normal images only in local regions, which are semantically or structurally similar to surrounding normal regions. To generate such abnormal data, CutPaste (26) introduces data augmentation technique that disturbs local regions of normal images with semantically similar patches. The extracted patches, in turn, undergo transformations (e.g.. random flip, rotate, blur, and color jittering) and replace the region of other normal images of the same category at random location. In this work, we use the CutPaste method to synthesize abnormal images. We modify this method slightly to generate more realistic abnormal images by blurring the borders of extracted patches and applying diverse color jittering values these patches. Also, we make abnormal patches by taking small rectangular patches like CutPaste or newly used circular patches. Size of patches are random and from normal image to retain diversity of abnormal regions and use the same ratio of all types of patches and normal data for training. Then, during training, our NF model handles normal and abnormal regions of each input image æ differently by assuming locally different base distributions. To achieve this, we introduce binary mask M as shown in Figure 3 for each synthetic anomaly image æ to denote whether each pixel pixel location (we h) belongs to the normal region (Mu, 1) or the abnormal region (M 0).

---

Fe (4f Canaeoo
4 ees
Gebiit
Horal iaae
Anorab lragy
Statinica Predicion
The overview of our proposed SANFlow framework. Firstly, with synthetic augmentation process, make binary mask M and anomaly image æ from normal image. Then, a pre-trained feature extractor f is applied to extract multi-scale feature maps (K=3, number of scales in this work) for æ. For each i-th position feature map of k-th level, each feature vector of is concatenated with position embedding vector P before being fed into NF, which is independently trained for each scale, To be specific, NF consists of 8 coupling blocks (i.e., L=8), which maps given feature distributions base distributions. Each base distribution is different in terms of statistics (variance in this work) which are dependent on semantic features of the corresponding location in the given image. In particular, we condition our statistics prediction network on the input image to generate parameters of inverse Gamma distribution such as a and B, to estimate the variances of base distributions, which are assumed follow inverse Gamma distributions.
Figure 3:
3.3 Feature extractor
Recently, features extracted by convolutional neural networks (CNN) pre-trained on ImageNet 39 have been shown be helpful in anomaly detection, by providing useful semantic information
8
20 48
In particular, several methods have employed multi-scale feature pyramid [27] to
handle anomalies of diverse sizes 20 48 g This because features of different scales capture information about regions of corresponding sizes (i.e., effective receptive field) in the image (29]
Motivated by previous findings, we also employ a pre-trained CNN [T1 20 36 to obtain a K-level feature pyramid. The feature map at the k-th scale is denoted by A E C X Ws where C₁ is the number of channels and H3 and We are the height and width of the feature maps, respectively.
3.4
Semantic-aware normalizing flow
3.4.1 Scale- and spatial-aware normalizing flow
We train NF map feature vectors computed by CNN to latents which follow simple base distributions. However, features can follow complex distributions at each scale and spatial location of feature map. Thus, using single NF model map such complex features a single base distribution can be difficult. Aiming to bridge the gap, CFLOW-AD 20 applied two modifications to a standard NF baseline model RealNVP 14:
independent NF models for each scale and conditioning NF on position embedding vector 15 44 similar to (2). Inspired by CFLOW-AD we also employ K independent NF models handle features across different scales we have K-level pyramid features and position embedding vector conditioning. Moreover, we train the NF model to map features to latents that follow spatially varying underlying distribution. This allows NF to consider semantic information, such as feature complexity, for corresponding locations in the image.
To be specific, we create set of vectors by taking feature vector oa at the i-th position (i.e., 2D coordinate) of the k-th scale feature map A as done in the previous work (20). Note that, omit the scale index & from this point on to avoid cluttered notation. To retain the spatial information, each feature vector ú; is concatenated with the corresponding position embedding vector p; IR whose values are composed of sinusoidal harmonics and unique to each i-th position [15] 44 The
resulting vector c; (v; p.) is now used to condition each transformation function g' in NE, making it process position-awas -awtre (2). Then, the log- likelihooho of feature distribution becomes:
I. log det
dg'
= log (z))
(3)
dz
1=1
5

---

where Z is simple, yet locally varying base distribution corresponding to feature vector
However, just making NF scale- and spatial-aware is insufficient to handle variations that exist within the same object category. This is because objects are not perfectly aligned across different images of the same category, resulting in images with different semantic information even at the identical spatial location and scale. Thus, in order to enhance the density estimation of our NF model, we propose embed feature distributions into different base distributions, leveraging the semantic information encoded in the features. Further details on this approach are provided in the following section.
3.4.2 Semantic-aware base distribution
In this study, instantiate semantic-dependent base distributions as Gaussian distributions with statistics that are estimated based on semantic information of features. Inspired by an algorithm that estimates the statistics of non-i..d. noise [52), we employ lightweight statistics prediction network h to estimate the statistics for the given feature U,. We condition the network, parameterized by On. on the semantic features to be able estimate appropriate statisties of corresponding base distributions. As we will discuss in our experiments, we empirically observe that estimating both mean (p²) and variance (o?) of the base distribution difficult and that estimating variance (af) alone is beneficial (Section 4.3) Therefore. we fix μ4 to be 0 for normal regions and 1 for abnormal regions. In doing so, we map abnormal features to a base distribution with minimal overlap with that of normal features during training, thereby aiding NF in mapping normal and abnormal regions into distinct distributions. When samples are non-i.i.d., the distribution of sample variance can be assumed to follow the inverse Gamma distribution [52). Since image pixels and semantic features are non-i.i.d., we formulate the variances of the corresponding base distributions to follow the inverse Gamma distribution as IG(-la, 8) with parameters a and B. We set a to be (%/- 11 and /3 to be e/s. where p² denotes the area of the corresponding receptive field of size p and & is the mode of the inverse Gamma distribution [52 [0]. In this work, is a hyperparameter, the value of which is empirically found to be 0.1. Upon the assumption, we train the network & to estimate a; and B; for each v, resulting in the following regularization loss as:
LKL
HaxWi (G(a,
= Σ ((a=aa)((a)) + (log l'(a) log)"((:)) + a(log 3: - log 8) + asl 3 Bi
- 1)},
(4)
where DKL (IG(a, 3:) IG(a, computes a KL divergence between the estimated distribution by the network h and the distribution of variance o². which we assume to follow IG(a, 3) as mentioned above; & is digamma function and I is gamma function. Notably, Lx7 with a and 3 acts as regularization to guide the statistics prediction network for stable training. The derivation details and detailed explanations can be found in the supplementary material.
3.5
Loss function
In this section, we delineate the overall objective function which our model is trained minimize. First, we need to modify the logelikel of z; corresponding to feature v; in Equation to account for handling both normal and abnormal features differently:
log (z.) m. log (1 - m.) log (Z;)
(5)
where the binary indicator m; is set to be 1 when the corresponding location in the binary mask M I, and O. otherwise. Note that, Z represents the corresponding base distribution when v; is a normal feature, while Zí represents the base distribution when v; is an abnormal feature. In practice, to compute the likelihood at the k-th scale, the binary mask M resized to match the size of the feature map A which has height H7 and width WR. The resizing is done using nearest-neighhor interpolation to preserve the binary nature of the mask. In turn, each log-likelihood term can be formulated as follows (detailed derivations can be found in the supplementary material):

---

1
1
log (zi) - log 2
[ 3: - (b(a)))
2
2,3,
|
(6)
and,
20p log 2T
1
1 (log Bi - (aa)))
0ii
|| - 1
2,,
Moreover, further aid NF in distinguishing normal and abnormal features, we can also train NF to perform binary classification using the loss function as follows:
Lacr = (s ms);
(8)
where BCE denotes the conventional binary cross-entropy loss, while s(z;) measures the probability
Pzr (2.)
of z; being classified as normal (i.e., s(z:) Pzn (p2 Note that the log-likelihood of z; being abnormal (Equation 7 can be used only during training such that our framework learns to map abnormal features to base distribution z" that has small overlap with the base distributions of normal features Z". This allows NF to transform normal and abnormal features to two distinct base distributions, thereby enhancing the discriminability.
Together with LKL. from Equation 4 the overall objective function Loverall to minimize is given by,
Loverall LNLI + A LRCR + A2 LKI.;
where d and A2 are hyperparameters to adjust the associated weights LECE and LKL.
3.6 Anomaly score functions
During inference, given test image æ E Hxx we first compute the log-likelihood of each feature v, v_ each k-th scale via Equation 3 where the log-likelihood of z; is computed with Equation 6 Then, the likelihood map pk #axWa for the k-th scale is obtained by taking exponential of the 6 We obtain the final likelihood map P R x by summing the likelihood map P from all scales. These likelihood maps first upsampled to the image resolution " using bilinear interpolation, following the approach described in (20). Finally, the anomaly score map computed as the negative of P, resulting in -P.
4 Experiments
In this section, we evaluate our framework to validate its capability in both pixel-Ievel anomaly localization and image-level anomaly detection. We will release our code and data upon acceptance. and more details and results can be found in the supplementary material.
4.1 Experimental settings
Dataset. The experiments are conducted on two commonly used datasets for unsupervised anomaly detection: STC (ShanghaiTechh Campus) dataset (28) and MV'Tec dataset 4 STC is a video surveillance dataset, which provides static videos of 13 different scenes of 856 x 480 resolution. contains 274, 515 frames for training and 42, 883 frames for evaluation. The training set consists of only normal sequences, while the evaluation consists of 300, 308 regular frames and 42, 883 irregular frames. MVTec is dataset that consists of images of industrial products categorized into 5 texture categories and 10 object categories. To evaluate the unsupervised anomaly detection performance, the training set includes only defect-free (e.g., normal) images: 3, 629 normal images are available for training while 1, 725 normal and abnormal images are used as test set. Among the test images, 1, 258 images contain defects (e.g., abnormal images). For data augmentation, abnormal patches undergo random vertical, horizontal flip and rotation during training, as described 3.2 We evaluate and compare algorithms in terms of area under the receiver operating characteristic curve (AUROC) and area under the per-region-overlay curve (AUPRO), as used can be found in the supplementary.
20 AUPRO scores
7

---

Table 1: Anomaly detection and localization results w.r.t. AUROC metric by various methods with WRN- 50 backbonn for feature extraction on STC
228
FramePred
X
MemAF
[8
SPADE 8
PaDIM
II
CFLOW-AD
20
PatchCore-10
SANFlow (Ours)
Image-wise
72.8
71,2
71.9
72.63
76.1
Pixel wise
89.9
91.2
94,48
91.8
94.8
Table Quantitative comparisons on anomaly localization performace in MVTec dataset 4 with respect to AUROC metric. For each category, the best and second best performance is bolded and underlined, respectively.
Category
AEsara WRN-50
7- --VA-grad WRN-50
PacciSVDD WRN-50
PaDDM WRN-50
CuiPPsti WRN-50 50
CHLOW-AD WRN-50
PateliCore-10 WRN-50
PatchCore-1 WRN-
SANFlow WRN-50
SANFlow WRN- 101
bottle
93.01
98.1
98.1
98.
98.66
cable
82.0
88.0
96.8
96.7
90.0
97.64
98.5
98.4
98.55
98.8
capsulle
94.0
91.7
95.8
98.5
97.4
98.
98.9
99.1
99.1
98.9
carpet
87.0
72.7
92.6
99.1
98.3
99.23
99.1 1
98.7
99.3
99.4
grid
bazelnut
97.0
98.8
97.5
98.2
97.3
98.82
98.7
93.8
99.2
99.0
leather
78.01
89.7
97.4
99.2
99.5
99.61
99.3
99.3
99.6
99.8
98.
91.0
93.5
95.1
95.7
95.7
98.95
17.66
97.8
screw
96.0
97.2
95.7
98.5
96.7
98.10
99.4
99.3
99.0
99.2
tile
59.0
58.1
91.4
94.1
90.5
97.71
95.9
96.1
98.9
99.1
toothhhush
98.1
98.8
mansistor
90.0
93.1
97.0
97.5
93.0
93.28
96.4
96.4
94.4
95.1
wood
73.0
80.9
90.8
94.9
95.5
94.49
95.1
95.1
96.4
97.9
zipper
average
87.0
88.8
95.7
97.5
96.0
97.9
98.1
98.2
98.5
98.8
Table 3: Quantitative comparisons on anomaly detection in MVTTec dataset 4 with respect to AUROC metric. For each category, the best and second best performance is bolded and underlined. respectively.
Canegocy
GANomaly WRN-50
DCSVM WRN-50
PachSVLD WRN-50
DiffecNet WRN.50
PADiM WRN-50
CHLOW-AD WRN-50
CuiPasse WRN-5I
PuschCare-Il 10 WRN-50
PatchCoore WRN-I0I
SANFlow WRN-50
SANFIow WRN-101
calile
75.7
90.3
97.59
99.6
capoule
73.2
54.4
76.7
86.9
97.68
18..2
97.8
08.2
97.7
98.9
corpet
69.9
62.7
92.9
92.9
98.73
03.9
98.7
98.4
99.8
99.9
gnd
70.8
41
94.5
$4.0
99.60
100
97.9
99.8
99.3
100
huzelout
78.5
91.1
92.0
99.3
99.95
58.3
100
100
1061
100
leather
84.2
90.9
97.1
100
100
100
100
100
scrww
74.6
96.3
31.89
94.0
tile
79.4
97.8
99.4
09.88
94.6
98.9
100
mothheush
03.3
61.9
100
98.6
99.65
99..4
99.2
100
96.7
100
mansistor
79.2
56.7
91.5
91.1
95.21
96.1
T08I
100
99.1
99.4
woed
83.4
95.3
96.5
99.8
99.12
99.
99.0
99.5
99.1
99.3
74.5
51.7
97.9
95.1
98.48
99.9
99.5
99.9
99.9
100
average
99.4
Implementation details. Following other works 4 20 36), our models are trained and evaluated
separately for each category. Similar to CFLOWWAD 220 we K=3 scales for feature pyramid; normalizing flow consists of L=8 transformation blocks; the dimension of position embedding vector D is 512; Adam optimizer with a learning rate of 5c-4 and 80 train epochs for training. The loss weight hyperparameters A and A2 are set to be 1.0 and 0.2, respectively. The statistics prediction network h consists of five convolutional layers with a skip connection as illustrated in Figure At last, we use NVIDIA RTX8000 for training and the model shows near real-time performance by running 13fps.
4.2 Experimental results
We provide the anomaly localization and detection performance on the STC dataset in Table while the results on MVTec dataset are reported in Table A and Table 3 All results presented in this study are obtained using the same backbone architecture, WRN-50 [53] while additional results on MV'Tec are reported for PatchCore (36] and our method with a larger backbone, WRN-101 [53]. The experimental results demonstrate that the performance of our proposed method is comparable with other of-te-oftth-arr methods. Table compare to other anomaly detection models that are not video targeted, method shows outstanding performances in both detection and localization.
8

---

tile
cable
Input image
(a)
(b)
Input image
(b)
Figure 4: Visualization of anomaly score map based on (a) th and (b) z on MVTèc dataset.
Table 4: Ablation study on loss functions in
Table 5: Ablation study on anomaly augmentation MVTec dataset.
MVTec and STC datasets.
De
Eacs
Cx.
STC:
MVTee
Augmentation
CacE
CxL
STC
MVToc
Model (1a)
x
8
72.6
98.3
Model (1b)
Bq431445)
x
x
73.5
98.2
Model (Id)
74.2
98.5
Model (1e)
By(33355
73.1
98.3
Model
73.6
Model (1d)
(q43)5555
74.2
SANFlow
76.1
98,7
SANFlow
Plus. Table and and Table 3 reports reports that our proposed method performs better than existing NF-based
anomaly detection algorithms (CFLOW-AD (20] and DifferNet (37] and other anomaly detection
algorithms such as AESSIM 4 VAE+grad [12], PatchSVDD 49 PaDiM [ and PatchCore- 1, 10 [36). In particular, the results demonstrate the effectiveness of our proposed semantic-aware base distribution mapping approach in enhancing the density estimation performance of the proposed NF model.
4.3 Ablation study
We conduct ablation experiments to evaluate the efficacy of each component of our framework. Specifically, perform ablation studies the loss function (Table synthetic anomaly generation (Table 5). and statistics estimated by our statistics prediction network h (Table 6 All ablation experiments are reported with respect AUROC, using the WRN-50 backbone, and all average AUROC values over all categories are reported for image-wise anomaly detection.
In Table analyze the efficacy of proposed loss functions. To do so, first compare against a baseline NF model trained using the standard negative log-likelihook loss of NF in Equation 2 (Model (1a)). Compared Model (1a), SANFlow brings substantial improvement, thereby validating the effectiveness of all three proposed loss functions (CNLL. in Equation 3 ussing Equation [ LkI., and LaCF). We validate the efficacy of each loss function separately by applying only LNU. using Equation 3 and 5 (Model (1b)) or disabling either LKı. (Model (1c) or LACF. (Model (1d)). While each variant leads performance improvement, using all three proposed loss functions leads the largest improvement, suggesting that all three proposed loss functions coordinate and play critical roles in our overall framework.
In Ta we evaluate the impact of synthetic anomaly augmentation (generation) the final performance of our framework. Training without anomaly augmenta- tion and thus without any anomaly data results in per- formance degradation (Model (2a)). Also, Model (1d) (Table which has only difference doing anomaly augmentation in training, performs better than Model (2a). Thus, the ablation result suggests that only embedding semantic features to different base distribu- tions but training NF with synthetic anomalies is also an important process. For better understanding. Fig- ure visualizes anomaly score maps based on estimated normal distribution 2" (a) and abnormal distribution Z" (b) for test images. As described in Section [36] Figure 4 (a) can be found with Equation 3 and agree on the abnormal region (highlighted with bright colors), validating that training NF to map normal and abnormal features to distant base distributions helps the model discriminate abnormal regions from normal regions.
Table 6: Ablation study statisties timation in MVTec and STC datasets. fixed/estimated refers whether statis- tics are manually fixed or estimated by our statisties prediction network.
Ai
Model (3a)
fixed estimated
fixed
75.9
97.6
Model (3b)
lixed
75.1
97.0
Model (3c)
estimated
SANElow (Ours)
fixed
estimated
76.1
98.7
Figure (b) can found with Equation 3ndd| 7 We note that both maps

---

Table repoots ablation results after fixing either mean (14), variance (dî), or both statistics of base distributions. If μ4 is fixed, it is set to be for normal features and 1 for abnormal features. If of is fixed, it is empirically set to be 0.1 for both normal and abnormal features. The performance observed to decrease noticeably when 14i is estimated alone (Model (3b)) or together with of (Model (3c)). On the other hand, estimating only of while fixing 14 brings significant performance improvement. The performance degradation from estimating 14, may be due to learning complexity. Furthermore, to enhance the persuasiveness of the mean margin, we conducted additional experiments on the MVTec dataset, comparing performance at margin values of 0.5, 1.0, and 1.5 in WRN-50 backbone. For margin values 0.5, 1.0, and 1.5, the proposing method shows 98.6, 98.7, and 98.1 AUROC for image-wist wise and 98.4, 98.5, and 98.3 AUROC for pixel-wise. As the margin of 1.0 yields the highest results in both image- and pixel- wise, we could confirm that the mean margin of 1.0 is persuasive choice.
4.4 Experiments other datasets
Although the MVTec dataset is a benchmark that is widely used for anomaly detection and lo- calization, there is difficulty in identifying the advantage of the proposing method with it due to performance saturation. Therefore, we provide a performance comparison in the VisA dataset [58]. Our proposed method with WRN-50 backbone achieves an image-wise AUROC of 93,4, 98.6, and PRO score of 89.4, while EFLOWWAD (0) and PaDiM 11] achieve 91.5, 59.8, and 89.1, 1, 85.9 of an image-wise AUROC and PRO score, respectively. The results show that approach demonstrates competitive performance against other methods, including other flow-based approaches, demonstrat- ing the effectiveness of our framework in greatly improving the density estimation capability of normalizing flow for anomaly detection and localization. Plus, while our approach excels at detecting anomalies within images, its effectiveness in tasks involving image- wise semantic outlier detection, like CutPaste [26 and CFLOW-AD, is limited. Nonetheless, we conducted an additional experiment with CIFAR-10 24, where our proposed method achieved an AUROC of 80.8, outperforming performances of CutPaste (69.4) and CFLOW-AD (79.32).
Conclusion
In this work, we propose to improve the density estimation of normalizing flow (NF) for anomaly de- tection, by training NF to dynamically embed given feature distribution different base distributions. In particular, base distributions are Gaussian distribution with statistics estimated by our statistics prediction network. As result, our NF framework learns to map not only diverse normal features but also abnormal features to corresponding yet different base distributions, enhancing the density estimation capability.
Limitation While SANFlow has the capability of mapping different semantic features to different base distributions, it requires anomaly augmentation to map anomaly features to a base distribution distant from base distributions of normal features for better anomaly detection. However, augmentations require domain knowledge and do not cover new anomalies that may occur. But, would like to note that our ablation study reveals that our framework brings improvements even without anomaly augmentation. Nonetheless, reducing the dependence anomaly augmentations in the proposed framework is an interesting future research direction.
Broader Impacts Finding anomalies in product during fabrication is important to ensure the quality of products. Anomaly detection algorithms can automate the process, which can offload the burden from human workers. This will allow human workers to focus other important aspects of fabrication, improving the overall process of fabrication and thus the quality of products. Furthermore, anomaly detection can be used to frequently check anomalies of objects that may be difficult for humans but crucial to human safety (e.g., tall buildings, bridges, nuclear power plants, etc.).
Acknowledgements This work was supported by Institute of Information & communications Tech- nology Planning & Evaluation (ITTP) grant funded by the Korea governmen (No. RS- 2023-00220628, Artificial intelligence for prediction of structure-based protein interaction reflecting physicochemical principles, 50%) and Samsung Electronics Co., Ltd, and Samsung Research Funding Center of Samsung Electronics (No. SRFCIT 1901- 06. 50%).
10

---

References
[1] Samet Akcay, Amir anomaly detection via adversarial training. In Proceedings of the Asian Conference Computer Vision (ACCV). Springer, 2018.
and Toby Breckon. Ganomaly: Semi-supervised
[2] Lynton Ardizzone, Carsten Luth, Jakob Kruse, Carsten Rother, and Ullrich Köthe. Guided image generation with conditional invertible neural networks. arXiv preprint arXiv:190..02392. 2019.
[3] Lynton Ardizzone, Radek Mackowiak, Carsten Rother, and Ullrich Köthe. Training normalizing flows with the information bottleneck for competitive generative classification. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
[4] Paul Bergmann, Michael Fauser, David Sattlegger, and Carsten Steger. Mytec ad-a compre- hensive real-world dataset for unsupervised anomaly detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
[5] Paul Bergmann, Michael Fauser, David Sattlegger, and Carsten Steger. Uninformed students: Student-teacher anomaly detection with discriminative latent embeddings. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
[6] Paul Bergmann, Sindy Löwe, Michael Fauser, David Sattlegger, and Carsten Steger. Improving unsupervised defect segmentation by applying structural similarity to autoencoders. arXiv preprint 18X1vv180..02011 2018.
[7] Victor Besnier, Andrei Bursuc, David Picard, and Alexandre Briot. Triggering failures: Out- of-distribution detection by learning from local adversarial attacks in semantic segmentation. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pages 15701-15710, 2021.
[8] Niv Cohen and Yedid Hoshen. Sub-image anomaly detection with deep pyramid correspon- dences. arXiv preprint arXiv:2005.02357, 2020.
[9] Yajie Cui, Zhaoxiang Liu, and Shiguo Lian. A survey on unsupervised industrial anomaly detection algorithms. arXiv preprint wxXiv:2204.11166 2022.
[10] Lucas Deecke, Robert Vandermeulen, Lukas Ruff, Stephan Mandt, and Marius Kloft. Image anomaly detection with generative adversarial networks. In Joint european conference on machine learning and knowledge discovery in databases. Springer, 2018
[11] Thomas Defard, Aleksandr Setkov, Angelique Loesch, and Romaric Audigier. Padim: patch distribution modeling framework for anomaly detection and localization. In Proceedings the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 475-489, 2021.
[12] David Dehaene. Oriel Frigo, Sébastien Combrexelle, and Pierre Eline. Iterative energy- based projection on a normal data manifold for anomaly localization. In Proceedings of the Interna- tional Conference Learning Representations (ICLR), 2020.
[13] Terrance DeVries and Graham W Taylor. Improved regularization of convolutional neural networks with cutout. arXiv preprint arXiv: 1708 04552, 2017.
[14] Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. Density estimation using real nvp. arXiv preprint 1rXiv:1605.08800 2016.
[15] Alexey Dosovitskiyy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In Proceedings of the International Conference on Learning Representations (ICLR), 2021.
[16) Spyros Gidaris, Praveer Singh, and Nikos Komodakis. Unsupervised representation learning by predicting image rotations. arXiv preprint 1803.1800007722 2018.
11

---

[17] Izhak Golan and Ran EI- Ea-Yaniv. Deep anomaly detection using geometric transformations. In Advances in Neural Information Processing Systems (NeurIPS). volume 31. 2018.
[18] Dong Gong, Lingqiao Liu, Vuong Le, Budhaditya Saha, Moussa Reda Mansour, Svetha Venkatesh, and Anton van den Hengel. Memorizing normality to detect anomaly: Memory- augmented deep autoencoder for unsupervised anomaly detection. Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2019.
[19] Matej Grcié., Petra Bevandić., and Sīniša Segvić. Dense open-set recognition with synthetic outliers generated by real nvp. In Proceedings of the International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISAPP), pages 133-143. INSTICC, SciTePress, 2021.
[20] Denis Gudovskiy, Shun Ishizaka, and Kazuki Kozuka. Cflow-ad: Real-time unsupervised anomaly detection with localization via conditional normalizing flows. In Proceedings of the IEEE Winter Conference on Applications of Computer Vision (WACV), 2022.
[21] Dan Hendrycks, Mantas Mazeika, Saurav Kadavath, and Dawn Song. Using self-supervised learning can improve model robustness and uncertainty. In Advances in Neural Information Processing Systems (NeurIPS), volume 32, 2019.
[22] Pavel Izmailov, Polina Kirichenko, Mare Finzi, and Andrew Gordon Wilson. Semi-supervised learning with normalizing flows. In International Conference on Machine Learning (ICML). PMLR, 2020.
[23] Shu Kong and Deva Ramanan. Opengan: Open-set recognition via open data generation. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pages 813-822, 2021.
[24] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
[25] Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. Training confidence-calibrate classifiers for detecting -uitt--diittibution samples. In Proceedings of the International Conference on Learning Representations (ICLR), 2018.
[26] Chun-Liang Li, Kihyuk Sohn, Jinsung Yoon, and Tomas Plister. Cutpaste: Self-supervised learning for anomaly detection and localization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
[27] Tsung-Yi Yi Lin, Piotr Dollar, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. Feature pyramid networks for object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR),
[28] Wen Liu, Weixin Luo, Dongze Lian, and Shenghua Gao. Future frame prediction for anomaly detection-a new baseline. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
[29] Wenjie Luo, Yujia Li, Raquel Urtasun, and Richard Zemel. Understanding the effective receptive field in deep convolutional neural networks. In Advances in Neural Information Processing Systems (NeurIPS), 2016.
[30] Prasanta Chandra Mahalanobis. On the generalized distance in statistics. Sankhyā: The Indian Journal of Statistics, Series A (2008-), 80:$1-S7, 2018.
[31] Stanislav Pidhorskyi, Ranya Almohsen, and Gianfranco Doretto. Generative probabilistic novelty detection with adversarial autoencoders. In Advances in Neural Information Processing Systems (NeurIPS), 2018
[32] Jonathan Pirnay and Keng Chai. Inpainting transformer for anomaly detection. In International Conference on Image Analysis and Processing, 2022.
[33] Danilo Jimenez Rezende and Shakir Mohamed. Variational inference with normalizing flows.
In International Conference on Machine Learning (ICML), 2015.
12

---

[34] Oliver Rippel, Patrick Mertens, and Dorit Merhof. Modeling the distribution of normal data in pre-trained deep features for anomaly detection. In International Conference on Pattern Recognition. IEFF, 2021.
[35] Nicolae-Cätälin Ristea, Neclu Madan, Radu Tudor Ionescu, Kamal Nasrollahi, Fahad Shahbaz Khan, Thomas B Moeslund, and Mubarak Shah. Self-supervised predictive convolutional attentive block for anomaly detection, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
[36] Karsten Roth, Latha Pemula, Joaquin Zepeda, Bernhard Schölkopf. Thomas Brox, and Peter Gehler. Towards total recall in industrial anomaly detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
[37] Marco Rudolph, Bastian Wandt, and Bodo Rosenhahn. Same same but differnet: Semi- supervised defect detection with normalizing flows. In Proceedings of the IEEE Winter Confer- ence on Applications of Computer Vision (WACV), 2021.
[38] Marco Rudolph, Tom Wehrbein, Bodo Rosenhahn, and Bastian Wandt. Fully convolutional cross- cross- scale-flows for image-based defect detection. In Proceedings of the IEEE Winter Conference on Applications of Computer Vision (WACV), 2022.
[39] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. Imagenet large scale visual recognition challenge. International Journal of Computer Vision (IJCV), 115(3)2211-252 2015.
[40] Mohammad Sabokrou, Mohammad Khalooei, Mahmood Fathy, and Ehsan Adeli. Adversarially learned one-class classifier for novelty detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
[41] Mayu Sakurada and Takehisa Yairi. Anomaly detection using autoencoders with nonlinear dimensionality reduction. In Proceedings of the MLSDA 2014 2nd workshop on machine learning for sensory data analysis, 2014.
[42] Thomas Schlegl, Philipp Seeböck, Sebastian M Waldstein, Ursula Schmidt-Erfurhh and Georg Langs. Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. In International conference on information processing in medical imaging. Springer, 2017.
[43] Tareq Tayeh, Sulaiman Aburakhia, Ryan Myers, and Abdallah Shami. Distance-basee anomaly detection for industrial surfaces using triplet networks. In 2020 11th IEEE Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON), pages 0372-0377. IEEE, 2020.
[44] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need, In Advances in Neural Informa- tion Processing Systems (NeurIPS), 2017.
[45] Shashanka Venkataramanan, Kuan-Chual Peng, Rajat Vikram Singh, and Abhijit Mahalanobis. Attention guided anomaly localization in images. In Proceedings of the European Conference on Computer Vision (ECCV). Springer, 2020.
[46] Shenzhi Wang. Liwei Wu, Lei Cui, and Yujun Shen. Glancing at the patch: Anomaly localization with global and local feature comparison. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition (CVPR), 2021.
[47] Ze Wang, Yipin Zhou, Rui Wang, Tsung- Yu Lin, Ashish Shah, and Ser Nam Lim. Few-shot fast- adaptive anomaly detection. Advances in Neural Information Processing Systems (NeurIPS), 2022.
[48] Jie Yang, Yong Shi, and Zhiquan Qi. Unsupervised anomaly segmentation via deep feature reconstruction. Neurocomputing, 2021.
13

---

[49] Jihun Yi and Sungroh Yoon. Patch svdd: Patch-level svdd for anomaly detection and segmenta- tion. In Proceedings of the Asian Conference on Computer Vision (ACCV), 2020.
[50] Hongwei Yong, Deyu Meng, Wangmeng Zuo, and Lei Zhang, Robust online matrix factorization for dynamic background subtraction. IEEE Trunsactions Pattern Analysis and Machine Intelligence, 40(7):1726-1740) 2018.
[51] Jiawei Yu, Ye Zheng, Xiang Wang, Wei Li, Yushuang Wu, Rui Zhao, and Liwei Wu. Fastflow: Unsupervised anomaly detection and localization via 2d normalizing flows. arXiv preprint arXiv:2111.07677, 2021.
[52] Zongsheng Yue, Hongwei Yong, Qian Zhao, Lei Zhang, and Deyu Meng. Variational denoising network: Toward blind noise modeling and removal. In Advances in Neural Information Processing Systems (NeurlPS), 2019.
[53] Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In Proceedings of the British Machine Vision Conference (BMVC), 2016.
[54] Vitjan Zavrtanik. Matej Kristan, and Danijel Skočaj. Draem a discriminatively trained recon- struction embedding for surface anomaly detection. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2021.
[55] Vitjan Zavrtanik, Matej Kristan, and Danijel Skočaj. Reconstruction by inpainting for visual anomaly detection. Pultern Recognition, 2021.
[56] Ye Zheng, Xiang Wang, Rui Deng, Tianpeng Bao, Rui Zhao, and Liwei Wu. Focus your distribution: Coarse-to-fine con-contrastive learning for anomaly detection and localization. In International Conference on Multimedia and Expo (ICME), 2022.
[57] Bo Zong, Qi Song, Martin Renqiang Min, Wei Cheng, Cristian Lumezanu, Dacki Cho, and Haifeng Chen. Deep autoencoding gaussian mixture model for unsupervised anomaly detection. In Proceedings of the International Conference on Learning Representations (ICLR), 2018.
[58] Yang Zou, Jongheon Jeong, Latha Pemula, Dongqing Zhang, and Onkar Dabeer. Spot-the- difference self- self-supervised pre-training for anomaly detection and segmentation. In Proceedings of the European Conference on Computer Vision (ECCV), pages 392-408. Springer, 2022.
14

---

A Loss Function Derivation
In this section, we show the detailed derivations for Equation 4 in Section 3.4.2 and Equation and
in Section 3.5
A.1 KI.-Divergence
We present the detailed derivations of our KL divergence loss LKL. where IG(a;, Be) and IG(o, .3) represent the probability density function of the inverse Gamma distribution with their corresponding parameters ag, 3; and a, 3. Furthermore, 8; and denote random variables that follow the cor- responding distributions IG(a,, B:) and IG(a, 8), respectively. The detailed formulations are as follows:
LKL (I
H Σ
Ellog [G B.) log [G(a,
MyXWL
Σ
[E]
- log IG(as, 3:]]]
Mx
Σ
(El-a + log (a 1) log + _
E1-0; 8 log (a: + 10 + "1
HxxW,
= Σ
log - (a + [a)( - Jay + log (,, - (a: + ();((;)))
1=1
HexW
- ((o=-a)b(a)) (log ((a) log ((:)) alog 3: -oog B) + arl 8
11)--
3:
(SI)
A.2 Log-likelihood Derivation
To calculate the semantic-aware variance o; for ligelikel computation, we approximate using the mode value (a,) from the inverse Gamma distribution IG(a,, 3.). This approximation is employed for computational efficiency, following the approach used in VDNet [52]. Therefore, in
Equation S2 and Equation S3 we use hoo x)) that predicts statistics of semantic-aware variances (i.e., o.) from input image æ as:
log Pzy (zi) Engo? a)
- h(of12)) log pl log
(2,00, (0) dd?
/ MoFlwa)
1 log 2x log o,
|2
}do
2
20
1
log 2T
1
/ h(o? xx) logo o,
マ
1
||
?
(S2)
2
2
o
1
log 2m
1
Fllog of -
1 ||
11
2
2
log 2T
(log B, (baa))
a
2 21
2
2
233
15

---

and
Pzp pzc (zi)
f h(0 logp( o,)
/ [z))
1|
1
log 2n
log o/
]do
2
2
20²
1
1
/
1
1
(S3)
log 2TT
h(o (x) log o do
|||
1112
h(oī|z))
oo do
2
1
1
1 Ellog o,3
1
log 2T
- - -1
2
2
log (log,3; 35 v((xs))
ai
|||
1
2
B Implementation Details
B.1 Hyperparameters
As described in the main manuscript, our overall framework comprises pre-trained feature extractor f. normalizing flow g, and statistics prediction network h. For the feature extraction process, we utilize a multi-scale feature pyramid approach. Specifically, we extract features from three different scales (K 3) using pre-trained feature extractor. Regarding the normalizing flow model, we employ L = 8 transformation blocks for each scale.
Similar to VDNet 522 our statistics prediction network takes an image as input and consists of five convolutional layers and skip-connection. Each convolutional layer comprises 64 filters wit size of 3 x 3 except for the last layer. In particular, the last layer of the statistics prediction network produces a two-channel output: one for a and the other for B3
During training phase on the MVTTe dataset 4, we fixed the learning rate to 5e-4. Moreover, we empirically determined the hyperparameters A and X2. Specifically,, we selected the values for λ and A2 from the set {0.1, 0.2, 0.3, 0.5, 1.0, 2.0} based on their impact on the validation results, and we use
1.0 and 22 0.2.
As for STC (Shanghai Tech Campus) dataset 28). all training details and hyperparameters are the same, except for a learning rate that is set to be (2e-
B.2 Data Augmentation
Small-sized rectangular shaped mask: 2 H S 32, 10 S W < 50
Medium-sized rectangular shaped mask: 16 < H < 64, 64 < W < 256
Circular shaped mask: 10 < radius S 25
To generate anomaly data, we first extracted patches based on the masks described above. These masks corresponded to small and medium-sized rectangular shapes, as well as circular shapes. Subsequently, the extracted patches underwent distortions to simulate anomalies, where we randomly perform color jittering between 0 and 0.1 (brightness, contrast, saturation, and huc). Additionally, we introduce random horizontal and vertical flips to further augment the dataset. These patches are then applied at the random locations of normal images from the same category. Figures STT and [S2 present categorical examples of synthetic anomaly images, with anomalies generated using rectangular patches in Figure SI and cireular patches in Figure S2]
We constructed a batch size 8 by including an equal number normal images and three types of abnormal images, with each type corresponding to a specific mask shape. This ensured balanced representation of normal and abnormal samples within the batch, allowing for effective training of model.
16

---

bottle
cable
Inseehuut
leather
incaal nut
pell
tife
toothbrush
trandster
Figure S1: Synthetic anomaly images generated with rectangular shaped patches.
buttle
cable
copoaale
carpet
grid
hazelnut
leather
metal not
pill
SETVW
tike
Coothbrash
transister
wood
sipper
Figure S2: Synthetic anomaly images generated with circular shaped patches.
Additional Experimental Results
C.1 Qualitative Results
In Figure $3 we give additional qualitative results of anomaly score maps similar to Figure in main manuscript. Specifically, we compare against Model (2a) from Table which a variant of our model that lacks anomaly augmentation. Figure S3 illustrates that even without anomaly augmentation, Model (2a) is still able to localize anomaly regions, highlighting that the major effectiveness of our framework does not come from anomaly augmentations but from the capability to transform the distribution of features at each location of given input image to different distributions. Regardless, the figure shows that our final model gives more accurate and clear anomaly score maps, owing to anomaly augmentation.
C.2 Quantitative Results
In this section, we provide additional and detailed quantitative experimental results for Table
and 6 in the main manuscript. In particular, provide the performance of each category for Table
5) and 6 in the corresponding tables: Table S2 S3 and [54] respectively. In Table S5 and S6 we
provide additional ablation results, which are obtained by applying the proposed semantic-aware normalizing flow (NF) schemes to other NF-based methods: CFLOW-AD [20] and FastFlow [51]. The
consistent performance improvement across CFLOW-AR and FastFlow demonstrates the flexibility and effectiveness of our proposed semantic-ane aware normalizing flow (NF) schemes.
C.3 Anomaly Score Function
Unlike the loss function, we do not use 8(2;) value in Equation g for computing anomaly score. However, s(2;) maybe helpful for defining better anomaly score function. To be sure that s(2,) is not helpful, we provide visualization examples Figure (S4 and [55] Figure S4 represents the results during training, while Figure S5 demonstrates the results during test. As it can be observed in Figure S5
17

---

during inference, there are instances where it tends to segment only localized regions produce high scores in unrelated areas. This may be due discrepancies between abnormal augmentation used during training and actual anomalies during test. Since, s(z;) is computed based on both negative log-likelihood (NLL) of normal and abnormal features, is necessary for the model to project normal and abnormal features precisely. However, Figure S5 illustrates that the projection of abnormal features is not as accurate as as normal features, showing unsatisfactory results with s(z;) as anomaly scoring function. Therefore, it does not exhibit suitability comparable to NLL for anomaly scoring. In the future, we have plans to utilize s(z;) for performance enhancement.
C.4 Additional Analysis for Table
While estimating both mean and variance to give higher performance due to more flexibility gives high expectation for the performances, we observe that giving too much flexibility and increasing training difficulty, thereby resulting in performance degradation as observed in Table the main paper and Figure S6 To better support the claim, we allow the model to estimate both mean and variance (i.e., Model (3c)) and plot the distribution of estimated mean by utilizing histogram in Figure S6 Graphs demonstrates that there are significant degree overlap between normal (blue) and abnormal (orange) distributions, when considering variance well.
18

---

Texture
Object
Ground
wood
carpet
grid
metal
pill
Figure S3: Visualization of anomaly score maps produced by our proposed framework and Model (2a) on texture and object categories of MVTec benchmark dataset. From top to bottom: GT mask, anomaly score map of proposed model and anomaly score map of Model (2a).
prediction
Figure $4: Ground truth, score map, and prediction for training data when using s(2) as the anomaly score,
prediction
Figure S5: Ground truth, score map, and prediction for test data when using s(2) as the anomaly score.
Table S1: Results by category in WRN-101 backbone.
Category
bottle
cable
coppale
carpet
grid
toothbouhh
wood
pipper
overae
Imuge- Wise
98.9
97.9
Pro-Score
96.6
98.7
96.9
98.0
91.5
97.3
99.3
96.3
97.6
95.8
97.4
94.2
80.6
96.3
98.3
96.1
19

---

GT
260
3007
score map
-2.4
-0.
523
539
05
15
bottle
prediction
cable
normal mean
1750
1109
abnormal mean
-050
-2.25
0.0
6.5
075
1.0
15
zipper
bottle
cable
zipper
Figure S6: Results of Model (3c) (model that estimates both mean and variance), including ground truth, score map, prediction, normal histogram. For histogram plots, each displays estimated mean distribution normal and abnormal features for each category: bottle, cable, and zipper. Orange part of histogram represents estimated abnormal means and blue part of histogram represents estimated normal means.
estimated map, abnormal mean estimated map and
Table S2: Results by category in Table
Categury
cupsule
curpet
motal
xipper
averuge
97.7
Image
96.7
97.5
09.1
08.6
98.5
SANFlow
100
98.7
97.7
100
99.8
94.0
100
99.1
09.9
98.7
(1b)
97.9
98.9
99.
95.0
99.1
96.1
98.3
Pixel- Wise
(1d)
98.6
98.3
97.1
99.0
99.0
99.3
98.6
99.2
99.6
98.3
98.1
99.1
98.9
09.0
99.0
98.3
93.6
96.0
96.5
99.1
98.4
98.3
SANFlow
98.6
98.5
99.1
99.3
98.5
99.2
99.6
98.5
99.2
99.0
93.9
98.9
98.4
96.4
98.9
98.55
(1b)
92.8
95.8
92.9
96.7
94.6
00.3
98.1
$7.8
94.7
94.4
93.19
87.5
88.1
90.9
95.5
93.3
Pro-Scure
(1d)
(1c)
93.1
94.1
04.8
96.5
93.9
03.0
96.5
96.5
94.6
04.2
06.6
96.5
08.5
98.5
91.9
91.7
94.9
05.1
04.9
94.7
94.7
04.4
89.3
80.2
88.4
87.8
91.4
91.2
96.2
04.6
94.1
93.8
SANTlww
92.7
96.7
91.1
96.4
91.3
96.8
98.3
92.8
95.6
95.3
91.2
87.9
87.6
91.1
95.7
94.0
20

---

Table S3: Results by category in Table
Calquory
copsollc
cappet
Pipper
averge
Image-
92.6
96.7
04.1
92.8
91.1
94.0
Table S4: Results by category in Table 6.
Categury
bottle
copsole
carpet
metol
sipper
averape
Image
98.0
96.6
99.5
SANFlow
100
98.7
97.7
99.3
100
09.5
94.0
100
09.1
98.7
97.1
96.2
Pixel- Wise
(3h)
(3c)
95.6
98.0
97.1
98.9
99.0
99.3
98.5
99.1
98.4
99.0
95.8
99.0
99.0
93.1
96.0
99.0
98..3
97.0
SANFlow
98.6
98.5
99.1
99.3
98.5
99.2
99.5
98.5
99.2
99.0
98.9
93.4
96.4
98.5
(3al
94.8
92.5
94.3
95.4
88.4
96.8
98.5
92.8
95.4
93.9
91.9
90.8
80.9
92.1
91.2
92.7
Pro-Score
(30)
(3c)
05.2
03.3
96.0
90.7
93.5
93.2
96.5
95.4
94.6
91.8
96.0
97.0
97.5
97.9
90.6
90.9
95.
94.5
92.6
95.1
93.1
92.9
87.7
90.5
86.2
84.6
91.2
90.8
93.9
96.2
92.7
93.7
SANFlow
92.7
96.7
94.1
96.4
94.3
96.8
98.3
92.8
95.6
95.3
94.2
87.9
87.6
91.1
95.7
94.0
Table S5: Comparison for other normalizing flow models with statistics prediction (SP v) and without statistics prediction(SP x). Experiments are conducted with the same backbone network WRN-50 on MV'Tec and results image-wise anomaly detection and pixel-wise anomaly localization. For convenience, we represent statistic prediction SP and NF based methods NF in this table. Since there is official code for FastFlow [5]], we conduct experiments with a third party code (https://gh) openvinotoolkic t/ anomalib).
reported with respect AUROC metric for two scenarios:
com/
NF
SP
bottle
calle
capstle
corpet
.2id
huzeinit
leuther
nettl .itt
piil
Sc/ee
tile
toothbruch
wood
zipper
Everage
Image-Wise
Fadi
100
100
96.8
93.5
59.4
99.0
180
99.6
97.7
99.9
100
99.5
99.6
94.1
92.4
150
53.6
99.9
94.4
96.5
95.1
100
97.6
Phods Wine
CFLOWWAD
18.70
98.75
07.66
97.73
68 08
98.99
00.35
09.25
96.59
95. 15
98.83
98.99
08.0
69
99.54
98.33
08 46
08.05
98.97
98.10
98.51
98.02
07.71
95.56
95.52
98.0
05.26
95.53
94.49
05.
98
08
07:87
96.25
95.7
97.4
98.6
98.5
97.6
Table S6: Detection and localization performances on STC dataset. CFLOW-AD+SP represents the
results of CFLOW-AD with statistic prediction (SP) in Table S5
(Ib)
(Ic)
(1d)
(2a)
(3a)
(3b)
(3c)
CFLOW-AD
CFLOW-AD+SP
SANFlow
Image-Wise
73.5
73.1
74.2
73.6
75.9
75.1
76.1
72.63
73.6
76.1
Pixel-Wise
91.5
91.5
94.6
94.5
94.6
94.1
93.7
94.48
94.5
94.8
21


Reviews:
[{"review_id": "tws6UaDOzT", "reviewer": "Reviewer_7gCM", "summary": "This paper presents an enhanced flow-based anomaly detection method by utilizing density estimation on normalizing flow (NF). The proposed approach involves predicting the inverse Gamma distribution using a statistics prediction network, which facilitates dynamic training of normalized flow distribution. Additionally, the training process benefits from the introduction of anomaly feature distribution and incorporates data augmentation techniques to enhance performance. By learning to map diverse normal features and abnormal features to distinct base distributions, the proposed NF framework enhances its capability for density estimation. The experimental results show that the improved density modeling achieved by the proposed framework, resulting in enhanced anomaly detection performance.", "questions": "Please see the weaknesses section. The comprehensive results on image-based anomaly detection are expected.", "limitations": "Authors have addressed the limitations of the method.", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "+ In contrast to previous flow-based anomaly detection methods that transform the distribution of all features into a single distribution, this work introduces a statistical prediction network to guide the training of different semantic features according to their respective distributions.\n+ By incorporating the data augmentation, both normal and anomaly distribution are predicted and employed to derive to the anomaly score. The qualitive results demonstrate that the localization performance of the proposed method is improved by data augmentation.\n", "weaknesses": "- The paper does not provide the detection performance of the PRO metric. The flow-based method tends to produce oversized defect regions, particularly for small-sized defects. Therefore, it is crucial to evaluate the location performance using the PRO metric.\n- For image based anomaly detection, several recent AD methods achieves promising the image level and pixel AUROC on MVTEC AD, which make it difficult to identify the advantage while comparison. Therefore, it is necessary to compare the performance on other datasets, e.g. VisA. In VisA, images exhibit complex structures, objects placed in sporadic locations, and different objects, which could differentiate the detection performance of different methods.\n- The paper utilizes a self-defined data augmentation method to synthesize anomalies. However, several existing data augmentation methods, such as Draem, nsa, and cutpaste, have been proposed for anomaly detection. It is important to consider whether the choice of data augmentation method impacts the detection performance.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for thoroughly reviewing our paper. \nWe appreciate your feedback. To address the concerns you raised, we are providing several experimental results and our perspectives.\n\n**W1: PRO metric**\n* In response to the question, we have taken the initiative to provide the PRO scores with WRN50 backbone in Supplementary **Table** **S1**. \nIt is noteworthy that the method utilizing Wide-ResNet50 as a backbone exhibits a commendable PRO score of 94.0 and also with an additional experiment with Wide-ResNet101 as a backbone we kindly provide a PRO-score of 96.0. \n* This performance not only stands out when compared to alternative methodologies that could serve as reference points, but also underscores its inherent merit by showcasing its immunity to the challenges associated with flow-based models.\n\n**W2: VisA dataset**\n* Another weakness Reviewer 7gCM (R1) mentioned was about the difficulty in identifying the advantage with the MVTec-AD dataset. \nFurthermore, R1 thankfully advised that a performance comparison in the VisA dataset was necessary. \nTo instill confidence in our approach, we conducted additional experiments on the VisA. \nWe actually conducted experiment for not only ours, but also DRAEM, CFLOW-AD, and PaDiM. \nIn addition, since our method is a flow-based network, we added results of FastFlow in paperwithcode for comparison.\n\n| VisA      | Image AUROC | Pixel PRO |\n|-----------|-------------|-------------|\n| Ours (NF-based)      | **93.4**        | **89.4**        |\n| CFLOW-AD (NF-based) | 91.5        | 59.8        |\n| FastFlow (NF-based) | 82.2        | 59.8        |\n| DRAEM     | 88.7        | 73.1        |\n| PaDiM     | 89.1        | 85.9        |\n\n* Our proposed method, with a WRN50 backbone, achieved an image, pixel-wise AUROC of 93.4, 98.6, and a PRO score of 89.4. \nThis shows that our approach demonstrates competitive performance across all methods, including other flow-based approaches, demonstrating the effectiveness of our framework in greatly improving the density estimation capability of Normalizing Flow for anomaly detection. \n\n**W3: Data augmentation ablation**\n* We employ the CutPaste technique with blurred boundaries to create more realistic abnormal images. \n* However, in response to the Reviewer's concerns, we undertook an additional performance comparison by incorporating the augmentation approach of DRAEM. \nWhen applying the DRAEM augmentation method for image-wise anomaly detection, we achieved an AUROC score of 97.1 while for pixel-wise anomaly detection, the AUROC score was 96.4. \nThese results indeed suggest that the augmentation method from DRAEM provides very complex abnormal regions and does not appear to yield positive contributions within the framework of our methodology, as indicated by the observed performance disparities.\n* While we need appropriate anomaly augmentation to properly map abnormal features to different base distributions, we show that our model can still work and perform better than a baseline (**Model (1a)** in Table 4) when we do not use anomaly augmentation at all (**Model (2a)**) in Table 5.\nWhile baseline **Model (1a)** gives 72.6 image-wise AUROC for STC dataset and 98.3 for MVTec dataset, our framework without anomaly augmentation **Model (2a)** gives 73.6 image-wise AUROC for STC dataset and 98.4 for MVTec dataset.\nThe performance improvement without anomaly augmentation suggests the effectiveness and importance of dynamically mapping features to its own respective base distributions based on semantic information.\nRegardless, learning to map abnormal features to different base distributions from normal features without anomaly augmentations or while being robust to choice of anomaly augmentations is important yet challenging research topic, which we leave as our future work."}, {"review_id": "JdDxObrFlR", "reviewer": "Reviewer_VpXm", "summary": "This paper propose a NF-based framework to model the normal feature distribution as Gaussian distributions with zero mean but different variances, and the abnormal features as distinct Gaussian distribution. The proposed model demonstrates relatively good performances in both anomaly detection and localization.", "questions": "Is the performance gain of 0.1% brought by anomaly augmentation? It seems the simulated anomalies are meaningless.", "limitations": "Yes", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The writing of this paper is clear and concise. It's easy to follow and understand each component of the proposed model. The motivation to model various distributions according to different regoins are reasonable.", "weaknesses": "1. The novalty of this paper seems to be increamental. It's based on / combanition of the reference papers [34] [22] [19] [48].\n2. The incorporation of semantic information seems to be naive. All the pixels have the same base distributions following the inverse\nGamma distribution as IG(·|α, β).  I wonder if the semantic information can be incorparated into the corresponding base distributions (The same regoin share the same base distribution).\n3.  I don't understand why you fix µi to be 0 for normal regions and 1 for abnormal regions. As the normal pixels have various characteristics, maybe corresponding to various µi, at least fot different regoins.\n4.  s(zi) maybe helpful for computing anomaly score. I'd like to see the correlations between s(zi)  and  GT location, which can reflect the gap between simulated training  anomalies and real testing anomalies.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are glad to hear that you found the paper to be clear and concise. We have diligently examined your comments and concerns as a reviewer, and have prepared responses addressing the raised concerns.\n\n**W1: Novelty**\n* We cautiously wish to assert the novelty of our approach. Contrary to previous Normalizing Flow (NF)-based frameworks, which use the same base distribution for all features and thus suffer from learning difficulty, our framework addresses this non-trivial issue by making full use of semantic features to dynamically map each feature to their respective base distributions, thereby increasing anomaly detection performance (as also highlighted by Reviewer dSr9).\n* Training NF to map each feature to its own respective base distributions is non-trivial. We introduce a statistical prediction network and our proposed loss functions (**Eq. (4) and (8)**) to guide the training of different semantic features according to their respective distributions (as also highlighted by Reviewer 7gCM).\n* Furthermore, we would kindly refer the reviewer to **L49-55** in the main paper for novelty of our method. In the context of vision anomaly detection, we firmly believe that estimating spatially and semantically varying base distributions based on the semantic information of features, enhancing discriminability and thereby anomaly detection performance, constitutes meaningful contributions.\n\n**W2: Naive incorporation of semantic information, all pixels share the same base distribution?** \n* Please allow us to provide a clarification of the proposed model: Each feature (or pixel) is mapped to its own respective base distribution with estimated variances based on the semantic characteristics of individual features, as illustrated in **Figure 1** of the main paper. \nIn essence, *all pixels **do not** follow the same base distribution* but follow its own respective base distributions, each with its own statistics (i.e., variance in our case) estimated by statistics prediction network, conditioned on semantic information.\n* The inverse Gamma distribution, IG($\\cdot$|$\\alpha$,$\\beta$), is used to guide the generation of $\\alpha_i$,$\\beta_i$ by the statistics prediction network for regularization and stable training, as mentioned in **L209-213** of our main paper.\nBut, $\\alpha_i$,$\\beta_i$ will not be equal to $\\alpha$,$\\beta$, as the overall framework, including the statistics prediction network, is trained to optimize our overall loss function (**Eq. (9)**), which also includes NLL. Since features are mapped to base distributions according to variances following IG($\\cdot$|$\\alpha_i$,$\\beta_i$) and $\\alpha_i$,$\\beta_i$ vary depending on semantic features, features are not mapped to the same base distribution.\n* In addition, the large performance gap between only using same base distribution Model **(1a)** and using spatial varying distributions (**SANFlow**) shown in Table 4 demonstrates the effectiveness of our framework.\n\n**W3: Why the means were fixed** \n* While one might expect estimating both mean and variance to give higher performance due to more flexibility, we observe that this gives too much flexibility and increases training difficulty, thereby resulting in performance degradation, as observed in Table 6 of the main paper and **Figure C** in PDF attached above.\nTo better support our claim, we allow the model to estimate both mean and variance (i.e., **Model (3c)**) and plot the distribution of estimated mean in **Figure D** in PDF attached above. \nThe figure demonstrates that there are significant degree of overlap between normal (blue) and abnormal (orange) distributions, when considering variance as well.\n\n**W4: Using $s(z)$ for anomaly score calculation**\n* We have provided visualization examples in the attached PDF. **Figure A** represents represent the results during training, while **Figure B** demonstrates the results during test. \n* As it can be observed in **Figure B**, **during inference, there are instances where it tends to segment only localized regions or produce high scores in unrelated areas**. \nThis may be due to discrepancies between abnormal augmentation used during training and actual anomalies during test.\nSince, $s(z)$ is computed based on both NLL of normal and abnormal features, it is necessary for the model to project normal and abnormal features precisely. However, **Figure B** illustrates that the projection of abnormal features is not as accurate as as normal features, showing unsatisfactory results with $s(z)$ as a anomaly scoring function.\nTherefore, it does not exhibit suitability comparable to NLL for anomaly scoring. Furthermore, we explored various attempts, such as multiplying or adding $s(z)$ to the existing anomaly score, yet the results favored using NLL alone. In the future, we have plans to utilize $s(z)$ for performance enhancement.\n\n**Q1: The performance improvement due to anomaly augmentation**\n* Before answering the question, we would like to emphasize that our major contribution involves finding appropriate distribution of different semantic features and also mapping anomaly feature to completely different base distribution. \n* To train model to map anomaly feature to a distant distribution, an augmentation process is necessary. \n* For the answer, we agree that 0.1 seems a small improvement at MVTec-AD dataset. \nAs the performance of MVTec-AD is saturated, we considered the STC dataset results for models (Model **(2a)**) and (Model **(1d)**) to show assistance effect of anomaly augmentation. \nFor STC, we observed a performance improvement from 73.6 (Model **(2a)**) to 74.2 (Model **(1d)**). \n* Anomaly augmentation allows us to use BCE loss for support mapping anomaly features to the distribution distant from the normal base distribution. \n**Table 5** and **Fig. S3** in supplementary include find qualitative and quantitative difference with and without BCE loss, highlighting the importance of using anomaly augmentation."}, {"review_id": "d5NN965T7U", "reviewer": "Reviewer_dSr9", "summary": "This paper presents a novel NF-based framework for anomaly detection (AD) and anomaly localization (AL) task. The proposed Semantic-Aware Normalizing Flow (SANFlow) framework map the distributions of normal data to different distributions at each location in the given image instead of mapping a distribution of whole image and makes distributions of abnormal data distant from normal data. The proposed method achieves the state-of-the-art in both AD and AL.", "questions": "1.\tCan the authors provide more details on the implementation of the SANFlow model, including the multi-scale FN?\n2.\tCan you explain why no code implementation is provided, since open-source is necessary for the community?\n3.\tHave the authors considered comparing the performance of the proposed model on other datasets, such as CIFAR10?", "limitations": "Yes.", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "[1. The proposed SANFlow model addresses the problem of learning difficulty in anomaly detection, which is conventional in NF-based methods. This innovation makes full use of semantic information and enhances the accuracy of anomaly detection.\n2. This paper provides a detailed description of the proposed SANFlow model. The methodology is well-developed and supported by theoretical analysis.\n3. The experimental results on the Mvtec-AD demonstrate the superiority of SANFlow.", "weaknesses": "1. This paper lacks the implementation details of the multi-scale FN, making it difficult to reproduce the study. And no code is provided to reproduce the proposed method.\n2. This paper needs ablation studies about the margin between normal and abnormal means.\n3. More experiments should be conducted. The paper lacks experiments on other standard AD datasets like CIFAR10.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate Reviewer dSr9 (R3) acknowledgment of the well-developed methodology of our paper and the recognition of the experimental results showcasing the superiority of the proposed method. \nIn order to facilitate a deeper understanding, we have also prepared responses addressing the points the Reviewer mentioned weaknesses of our paper.\n\n**W1: Multi-scale NF details and code** \n* To implement multi-scale NF, we utilized the FrEIA library. \n* For reproducibility, along with implementation details provided in **L263-269** in our main paper and **Section B** in the supplementary document, we will release our code upon acceptance.\n\n**W2: Mean margin ablation** \n* To enhance the persuasiveness of the mean margin, we conducted additional experiments on the MVTec-AD dataset, comparing performance at margin values of 0.5, 1.0, and 1.5 in  WRN-50 backbone. \n* As the margin of 1.0 yielded the highest results, we could confirm a mean margin of 1.0 is a persuasive choice.\n* We believe this is due to trade-off between discriminability and representation. The anomaly and normal features have subtle differences. If the margin difference is too small, the model will have difficulty discriminating normal and abnormal features. If the margin difference is too large, the model is forced to impose large differences on normal and abnormal features that have very small and subtle differences, increasing training difficulty.\n| Margin     | 0.5  | 1.0  | 1.5  |\n|------------|------|------|------|\n| Image wise | 98.6 | 98.7 | 98.1 |\n| Pixel wise | 98.4 | 98.5 | 98.3 |\n\n**W3: Experiments on other datasets**\n* While our approach excels at detecting anomalies within images, its effectiveness in tasks involving image-wise semantic outlier detection, like CutPaste and CFLOW-AD, is limited. As a result, specific results for these tasks were not included in the paper. Nonetheless, we conducted an additional experiment with CIFAR-10, where our proposed method achieved an AUROC of 80.8, outperforming CutPaste (69.4) and CFLOW-AD (79.32).\n* In the paper, there are results on MVTec-AD and STC dataset and in response to **7gCM W2** we provide results on the VisA dataset. "}, {"review_id": "rNwfeYT3Cs", "reviewer": "Reviewer_cBPv", "summary": "The manuscript proposes a new method for dense anomaly detection in industrial applications. The proposed method first extracts dense features with a frozen pretrained model. Then, a normalizing flow learns the distribution of extracted features at every spatial location. While the flow parameters are shared across the spatial locations, the parameters of flow prior are independently computed by an additional network h conditioned on the input image. In particular, the network h outputs variances of flow priors while means remain fixed. During the training, the input samples are augmented with synthetic negatives. The optimization objective is formed of the likelihood maximization for inlier features,  regularization of network h outputs, and discrimination between inliers and synthetic negatives. During the inference, the method detects anomalies according to the aggregated likelihood of features across different resolutions. The developed method achieves competitive results on two considered benchmarks.", "questions": "Adding a colorbar to Fig. 4 might improve clarity.\n", "limitations": "The manuscript adequately addressed the limitations of the method. ", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 1, "strengths": "S1. The manuscript is well-written and easy to follow.\n\nS2. Modelling different normalizing flow prior at every spatial location appears to be novel.\n\nS3. The proposed method achieves performance gains over related works.", "weaknesses": "W1. The difference between CFlow-AD [19] and the proposed SANFlow i) is modelling independent flow priors for each spatial location and ii) mapping anomalous and non-anomalous features to flow priors with different means. These contributions may be rather incremental.\n\nW2. Intuition on why estimating both mean and variance is inferior to estimating variance only (L197) should be comprehensively elaborated beyond L323-328. Especially since predicting parameters of flow prior is the main contribution of the manuscript.\n\nW3. Missing related works which augment datasets with synthetic negatives [a,b,c,d].\n\n[a] Shu Kong, Deva Ramanan: OpenGAN: Open-Set Recognition via Open Data Generation. ICCV 2021.\n\n[b] Matej Grcic, Petra Bevandic, Sinisa Segvic:\nDense Open-set Recognition with Synthetic Outliers Generated by Real NVP.  VISAPP 2021.\n\n[c] Victor Besnier, Andrei Bursuc, David Picard, Alexandre Briot: Triggering Failures: Out-Of-Distribution detection by learning from local adversarial attacks in Semantic Segmentation. ICCV 2021.\n\n[d] Kimin Lee, Honglak Lee, Kibok Lee, Jinwoo Shin:\nTraining Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samples. ICLR 2018", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for providing a thoughtful review. \nFor enhancing our paper, we have diligently reviewed the weaknesses and questions raised regarding our paper and have prepared additional experiments and answers.\n\n**W1: Novelty**\n* We cautiously wish to assert the novelty of our approach. Contrary to previous Normalizing Flow (NF)-based frameworks, which use the same base distribution for all features and thus suffer from learning difficulty, our framework addresses this non-trivial issue by making full use of semantic features to dynamically map each feature to their respective base distributions, thereby increasing anomaly detection performance (as also highlighted by Reviewer dSr9).\n* Training NF to map each feature to its own respective base distributions is non-trivial. We introduce a statistical prediction network and our proposed loss functions (**Eq. (4) and (8)**) to guide the training of different semantic features according to their respective distributions (as also highlighted by Reviewer 7gCM).\n* Furthermore, we would kindly refer the reviewer to **L49-55** in the main paper for novelty of our method. In the context of vision anomaly detection, we firmly believe that estimating spatially and semantically varying base distributions based on the semantic information of features, enhancing discriminability and thereby anomaly detection performance, constitutes meaningful contributions.\n\n**W2: Why estimating both mean and variance is inferior to estimating only variance** \n* We agree that we need more explanation or intuition of **L323-328**. \nWhile one might expect estimating both mean and variance to give higher performance due to more flexibility, we observe that this gives too much flexibility and increases training difficulty, as observed in Table 6 of the main paper and **Figure C** in PDF attached above. \nTo better support our claim, we allow the model to estimate both mean and variance (i.e., **Model (3c)**) and plot the distribution of estimated mean in **Figure D** in PDF attached above. \nThe figure demonstrates that there are significant degree of overlap between normal (blue) and abnormal (orange) distributions, when considering variance as well.\nWe will include the discussions in the revised paper.\n\n**W3: Related works on data augmentation with synthetic negatives**\n* Thanks to your kind suggestion, we believe we can incorporate missing references. We will make sure to include the papers [a, b, c, d].\n\n**Q1: Adding a color bar**\n* Lastly, we appreciate the advice mentioned in the question. We will also add a color bar to Figure 4 to enhance clarity."}]

Author rebuttal:
To the reviewers.

First of all, we are grateful for the reviewers' efforts and we have prepared responses to comments and tables for additional experiments. For better explanations, we include visualizations in PDF attached below. We provide brief explanations for each figure:

There are a total of four figures labeled as A, B, C, and D. 
* **Figure A**: 
    * First three rows 
    * Depicting ground truth, score map, and prediction for ***training data*** when using s(z) as the anomaly score. 
    * Referred in response to **Reviewer** **VpXm** **W4**
* **Figure B**: 
    * Next three rows 
    * Depicting ground truth, score map, and prediction for ***test data*** when using s(z) as the anomaly score. 
    * Referred in response to **Reviewer** **VpXm** **W4**
* **Figure C**:
    * Last five rows of images on the left, where each column corresponds to different examples.
    * Results of **Model (3c)** (model that estimates both mean and variance) , including ground truth, score map, prediction, normal mean estimated map, and abnormal mean estimated map
    * Referred in response to **Reviewer** **VpXm** **W3** and **Reviewer** **cBPv** **W2**
* **Figure D**:
    * Three figures in the bottom right corner
    * Each histogram displays estimated mean distribution of normal and abnormal features for each category: bottle, cable, and zipper.
    * Orange part of histogram represents estimated abnormal means and blue part of histogram represents estimated normal means.
    * Referred in response to **Reviewer** **VpXm** **W3** and **Reviewer** **cBPv** **W2**

Meta-review: This paper got slightly positive but reserved reviews: one "weak accept" and three "borderline accept".

The authors addressed the reviewers' concerns by elaborating on the novelty and providing additional results, including evaluating with PRO metric, testing on the VisA dataset, adopting the augmentation approach of DRAEM, and experimenting with different margin values. The authors had further discussions with **Reviewer VpXm** on the issue of incorporating semantic information. After the rebuttal, **Reviewer VpXm** and **Reviewer cBPv** increased their ratings. 

In more detail, the authors' rebuttal has addressed the following concerns raised in the reviews:
1. Novelty: The authors emphasize that previous Normalizing Flow (NF)-based methods on anomaly detection use the same base distribution for all features while their method maps semantic features to their respective base distributions and thus achieves better performance. The authors introduce a statistical prediction network with tailored loss functions to guide the training. The authors' discussions with **Reviewer VpXm** further clarify that it is challenging to find semantic-dependent prior distributions for variances. **Reviewer VpXm** has increased the rating to weak accept after discussions.
2. Experiments on the detection performance with the PRO metric: The supplementary material of the initial submission has presented PRO scores using Wide-ResNet50 as a backbone. The authors further present in the rebuttal an additional result with Wide-ResNet101 as the backbone, achieving a PRO-score of 96.0.
3.  VisA dataset: The authors provide additional experiments on the VisA dataset in the rebuttal. With a WRN50 backbone, the proposed method achieves AUROC scores of 93.4 (image-level), 98.6 (pixel-level), and a PRO score of 89.4, which are better than those of CFLOW-AD, FastFlow,
DRAEM, and PaDiM.
4. Augmentation: The authors present an additional comparison to show that adopting the augmentation approach of DRAEM does not improve the proposed method: with DREAM's augmentation, it achieves an AUROC score of 97.1 for image-wise anomaly detection and 96.4 for pixel-wise anomaly detection.
5. Ablation: The rebuttal also provides additional ablation studies about the margin (0.5, 1.0, and 1.5) between normal and abnormal means.

After reading the paper, reviews, author feedback, and discussions, the AC does not notice significant flaws that lead to overruling the reviewers' assessment. Furthermore, the AC acknowledges that the quality and contribution of this paper will be significantly improved if the authors elaborately revise the paper with respect to the reviews and discussions. 
The AC thus recommends accepting the paper and trusts that the authors, in good faith, will improve the paper to address the reviewers' comments and incorporate the additional results when preparing the final version.
Decision: Accept (poster)