AE-FLOW: Autoencoders with Normalizing Flows  for  Medical Images Anomaly Detection 
Authors: Yuzhong Zhao
Qiaoqiao Ding
Xiaoqun Zhang
Venue: ICLR 2023 poster (2023)
Keywords: Anomaly Detection
Normalizing Flow
Auto-encoder.

Abstract: Anomaly detection from medical images is an important task for clinical screening and diagnosis. In general, a large dataset of normal images are available while only few abnormal images can be collected in clinical practice. By mimicking the diagnosis process of radiologists, we attempt to tackle this problem by learning a tractable distribution of normal images and identify anomalies by differentiating the original image and the reconstructed normal image. More specifically, we propose a normalizing flow-based autoencoder for an efficient and tractable representation of normal medical images. The anomaly score consists of the likelihood originated from the normalizing  flow  and the reconstruction error of the autoencoder, which allows to identify the abnormality and provide an interpretability at both image and pixel levels. Experimental evaluation on two  medical images datasets showed that the proposed model outperformed the other approaches by a large margin, which validated the  effectiveness and robustness of the proposed method.

Published as a conference paper at ICLR 2023
AE-FLOW: AUTOENCODERS WITH NORMALIZING FLOWS FOR MEDICAL IMAGES ANOMALY DETECTION
Yuzhong Zhao, Qiaoqiao Ding, Xiaoqun Zhang School of Mathematical Sciences, MOE-LSC and Institute of Natural Sciences Shanghai Jiao Tong University (zhaoyuzhong, dinggiaoqian x jtt.. edu.cm
ABSTRACT
Anomaly detection from medical images is important task for clinical screen- ing and diagnosis. general, large dataset of normal images are available while only few abnormal images can be collected in clinical practice. By mimicking the diagnosis process of radiologists, we attempt to tackle this problem by learning tractable distribution of normal images and identify anomalies by differentiat- ing the original image and the reconstructed normal image. More specifically, we propose normalizing flow-based autoencoder for efficient and tractable repre- sentation of normal medical images. The anomaly score consists of the likelihood originated from the normalizing flow and the reconstruction coder, which allows to identify the abnormality and provide an interpretability at both image and pixel levels. Experimental evaluation on four medical and one non-medical images datasets showed that the proposed model outperformed the other approaches by large margin, which validated the effectiveness and robust- ness of the proposed method.
of the autoen-
INTRODUCTION
Medical anomaly detection (Taboada-Crisp) et al. 2009 Fernando et al. 2021 is an important task in clinical screening and diagnosis by capturing distinctive features in collected biomedical data, such as medical images, electrical biomedical signals other laboratory results. Anomaly detection aims to detect data that significantly deviates from the majority of data instances, arising in clinical applications due to imbalance between normal and abnormal data and variability of anomaly in real world scenario. Different to usual classification models used for computed aided diagnosis, anomaly detection is usually considered in an unsupervised or semi-supervised paradigm. In this paper, we mainly focus on anomaly detection from medical images, to mimic the diagnosis process of radiologists.
The traditional techniques for finding anomalies are divided into several categories, such as statistics-based methods (Hido al. 2011 Rousseeuw & Hubert 2011), distance-hased meth- ods (Knorr et al.] 2000) Angiulli et al. 2005). ), density- based methods (Breunig et al., (2000). and clustering-basee based methods Yang al., |2009] Al-Zoubi 2009). etc. Deep learning for anomaly de- tection (Wang et al. 2019 Chalapathy & Chawla 2019] Pang et al., 2021)). also known as deep anomaly detection, typically consists of learning feature representation model of normal images and constructs anomaly score function for abnormal images by neural networks. There are two main types of approaches for image anomaly detection, one is reconstruction baaeed model and the other is likelihood- -based model in the literature.
The standard procedure of reconstruction- based methods is to first learn an auto-encoder (AE) (Kramer) 1991) or generative models (Goodfellow et al., (014) for normal images and the difference between the test and reconstructed (generated) images through the representation neural networks can be used to characterize the level of anomaly. For example, AnoGAN (Schlegl et al. 2017) is a generative adversarial networks (GAN) based model utilizing a generator for image reconstruc- tion and an anomaly score using weighted sum of residual socre and discrimination score. In Akcay et al. (2018), GANomaly considers the distance in the latent feature space to distinguish the anomaly data. F-anoGAN (Schlegl et al. 2019) is improved version of anoGAN, which si-

---

Published as a conference paper at ICLR 2023
multaneously guides encoder training in both image and latent spaces. Even reconstruction based methods could give the dissimilarity in pixel level, many limitations exists for anomaly detection tasks. The limited capabilities of AEs in modelling high-dimensional data distribution often lead to inaccurate approximations and erroneous reconstructions, such as features are smoothed out in the reconstructed images (Kingma & Welling 2013 [Schleg] et al.] 2019 Ravanbakhsh et al.] (017)) and image boundaries are predicted as anomaly pixels and appeared in the difference (Schlegl et al. 2017 2019).
The second type of anomaly detection method is to construct a likelihood function of extracted im- age features. For example, normalizing flow-based anomaly detection models have been proposed for industry anomaly datasets (Rudolph et al., 20222 Gudovskiy et al.] [022]] Yu et 20.1).021.. Nor- malizing flow (Dinh et 2014: Rezende & Mohamed 2015] Dinh et al., 2016] Kingma & Dhari- wal. 2018) is a popular method that transform the observed data to tractable distribution. The idea behind normalizing flows deploying a sequence of invertible and differentiable mappings to transform complex distribution into simple probability distribution (e.g.. standard normal dis- tribution). NICE (Dinh et al. 2014) was introduced for modeling complex high-dimensional densi- ties. In Dinh et al. (2016), V was proposed to improve the coupling layer and multi-scale architecture was applied to enhance the representative ability of the framework. Recently, Glow proposed in Kingma& & Dhariwal (2018) utilizes actnorm and 1 convolution simulate more realistic images. Normalizing flows have been applied in many fields such as image generation, noise modeling. video generation (Papamakarios
al. 2021). Recently, some anomaly detection methods based on normalizing flows have also emerged and have achieved very high performance for industrial datasets. Differnet (Rudolph et al. 202T) used a backbone network extract multi- scale features of the input and a normalizing flow network to maximize the likelihood. The anomaly score is defined as the average of negative log- ligeliiiss.oos. CFLOW-AD (Gudovskiy et al., 2022) added positional embedding layers and utilizes conditional flows to complete the location problems. FastFlow (Yu et al. (202T) proposed a lightweight network and further enhances the accuracy. How- ever, these NF-based anomaly detection methods make decisions by estimating the likelihood of the extracted features, and the structural information of images are missing. From this perspective, an auto-encoder architecture can construct mapping between the hidden feature space and the data space, and enforce the model to learn structural information of the original data.
In this paper, we propose to construct loss function and anomaly score funetion by an auto-encoder with the normalizing flow bottleneck, namely AE-FLOW. This model combines the benefits of nor- malizing flow methods for computing the anomaly likelihood of extracted features image level, and the interpretability of reconstruction-bant based methods at pixel-level. The proposed score function takes both computable probability density in feature features and visual structures consistency in image domain into consideration. The model is in a self-supervised paradigm that uses only normal data, which is more adaptive to real world applications. The experiments on four public medical datasets and one mon-medical dataset demonstrated the effectiveness and robustness of the proposed method, with a detail comparison to other existing methods.
METHODS
We propose method that integrates the difference on image structures of the original and recon- structed image and the likelihood of image features distribution. The proposed pipeline consists of three important components, i.e., -lweeer-foowddeoderr. Intuitively, normal data will be mapped to the high density area of the standard Gaussian distribution as normalizing flows are trained with normal data. In contrast, those abnormal data will be mapped to the tail of the distribution, making it difficult for the decoder network to effectively reconstruct the original image. The loss function of the three-block pipeline takes into account of the dissimilarity between the reconstructed image and the original image in pixel-level and the likelihood of data distribution in feature-level. The anomaly score is composed of the reconstruction error and the flow likelihood for the inference stage. The pipeline is illustrated in Fig. [ compared to the usual autoencoder neural network. The detailed illustration of the overall network structure is shown in Fig. 2 In the following, we will explain the three blocks in detail.
Encoder. The model starts with a pretrained encoder network for extracting features of the input im- age. Each input image will be downsampled four times and the extracted feature is low-dimensional
2

---

Published as a conference paper at ICLR 2023
g(Z)
8(z)
Encoder
Decoder
Encoder
Decoder
NF
Z
Z
(O.D)
Figure 1: Illustration of the vanilla autoencoder (left) and the skeleton of the proposed AE-FLOW (right). For the vanilla autoencoder, the encoder block extracts the feature of the input and the decoder transforms the intermediate feature back to the image. For the AE-FLOW model, NFs map the extracted feature to the standard Gaussian distribution and the normalized feature is decoded as the reconstructed image.
frcco
Pretrained
TransConv
ConvBlock
Pretrained ConvBlock
Conv TransConv
Pretrained ConvBlock
Conv TransConv
Encoder
Decoder
Pretrained
Conv TransConv
Bottleneck
ConvBlock
Feature
Flow Model
Normalized Feature
koow
Figure 2: The overview of the AE-FLOW model. Firstly, the encoder block extracts low- dimensional features z of the input images x. Then, the normalizing flows transform the feature vector standard Gaussian distribution. At last, the decoder reconstructs the normalized feature to the image x'. The likelihood of the normalized feature is used as the flow loss and the residual between and x' is used as the reconstruction loss.
and suitable for the flow model training. More specifically, the encoder f X - Z transforms the input image x E 33 HxW to feature z E RCx 18 W. where H and W are the height and width of the original image and C is the number of channels.
Normalizing flow bottleneck.
order to explicitly obtain the probability density Pz(z) of the ex- tracted feature vector z, we use normalizing flow model + z' to transform it to the feature space ~ (z'; 0, I) which follows a standard normal distribution. This flow model trained by maximizing log-likes likeliholoo for the normal image data, which implying high probability for normal data, while a low probability for abnormal data in the inference stage. The test data can divided into two categories by applying threshold on the likelihood. In particular, the normalizing flow map is composed bijective transformation blocks. For each block, one coupling layer, Softplus activation and output vector permutations used. The total flow model contains eight flow blocks.
3

---

Published as a conference paper at ICLR 2023
By denoting the input and the output y each flow block, it can be formulated as KKiggma
& Dhariwal 2018 Ardizzone et al. 2018-22 :
y R ( (Sglobal)
@ Coupling (x) Egllobal
where R is 1 x 1 invertible convolution that reverses the ordering of the channels, (-) is the Softplus activation function
log(1 + exp(3 * -)) with / 0.5, Sglobal and Egiobal are learning parameters corresponding to the scale and bias in the actnorm step, ④ point-wise product. For the
coupling layer, we split the input x into two compositions (x1xx2) by the channels, and denote the corresponding output as u (u,, u2) = Coupling (x) and:
u
exp (2 tanh (s (x2))) +x2)
U2 X2
where s (x2) and (x2) represents the scale and bias which are predicted by one NN, i.c. sxx2), t(xg) NN(x)). Two subnet architectures are considered for this NN: one follows the network designed by Yu et al. (202T) for which contains two convolutional layers. The second one based on ResNets (He et al. 2016) structure for better image structure representation and easier training. Details can be found in the later experiments section.
Decoder. The decoder y Z' X' is used reconstruct the image x' from normal distributed data by minimizing the distance between the input image x and reconstructed image x'. In the decoder, we adopt four modules and each module contains two convolution layers. In each module, we use 3 x 3 kernel transposed convolution layer for upsampling operation and another 3 x 3 kernel convolution layer to adjust the number of channels and enhance the reconstruction ability.
Loss Function.
The loss function consists of two components, the flow loss and the reconstruction loss. The negative logarithm probability likelihood pz(z') is taken as the flow loss. The reconstruc- tion loss is defined as the mean square error (MSE) between the input image and the reconstructed image. We choose MSE order to force the model to recover main image structures. The weighted sum of two losses will represent the total loss. The loss function with parameter or is formulated as:
L = a * LftOw + (1 - a)) * Lrecons
(1)
where
Lioo log (2)((() log Pz/ (z') log det Oz'
Oz
N
Lreron MSE(x,
Xi.j - x(.) 1=1
N2
j=1
We note that the reconstruction loss can be also replaced with other measures, such as Structural Similarity Index Measure (SSIM) (Wang et al., 2004).
Anomaly Score. Similar to the loss function, we use two components for the anomaly score in the inference step. The negative probability density of the normalized feature is used as the flow score. For the reconstruction loss, we use the negative SSIM between the input image and reconstructed image as its range of [-1, 1] and it mainly measure the structure difference between images. The window size for calculating SSIM score is set to be 11. The total anomaly score is formulated as weighted summation the two scores by a parameter 3
S = 3 Snow + (1 (3) Syecons
(2)
where
Snaw -P (z');
Specn = -SSIM(X,X) x').
3 EXPERIMENTS
3.1 DATASETS
The proposed method was performed on five datasets with image-level annotations: the OCT (Opti- cal Coherence Tomography) dataset, the Chest X-ray dataset (Kermany et al.) 2018), the skin image
4

---

Published as a conference paper at ICLR 2023
ISIC2018 International Skin Imaging Collaboration) dataset (Tschandl al. 2018 Codella al. 2019). the brain tumor BraTS2021 (Brain Tumor Segmentation) dataset (Menze et al. 2014 Bakas
et al. 2017 Baid al. 2021) and the microscopic images MIIC (Microscopic Images of Integrated Circuits) dataset (Huang et al. 2021).
The OCT dataset consists of 26, 315 normal images for training. In the test phase, there four categories: normal, drusen, CNV and DME. The later three categories are regarded as abnormal data and each category has 242 images. The image number of test data for the four categories is chosen to be the same to avoid the data imbalance problem.
OCT:
Chest X-ray: The X-ray chest dataset contains 1, 1,349 cases of normal images for training. There are 234 normal images and 390 abnormal images which are diagnosed as pneumonia for testing.
ISIC2018: The ISIC2018 challenge dataset (task three) contains seven categories and we refer to Lu & Xu 2018) for classifying the NV (nevus) category as normal samples and the rest category as abnormal data. The training set includes 6705 healthy images, and the original validation set are used as the test data, which includes 123 normal images and 70 abnormal images.
BS2021: The Brain Tumor Segmentation (BraTS) Challenge 2021 dataset consists of 3D mul- timodal magnetic resonance imaging (MRI) scans. We slice the central portion of each 3D data set to make up the 2D dataset for our model. There are 11, 739 healthy slices in our training set, and 1, 051 slices with tumors and 459 slices without tumors are used as the test set.
The MIIC dataset first appeared in (Huang et al., 2021 contains real microscopic images of integrated circuits (ICs), with 23,888 normal images for training, and 1, 272 normal and 116 abnormal images for testing. We directly compare our methods with the proposed method in the original paper (Huang et al. 2021).
MIIC:
3.2 IMPLEMENTATION DETAILS
We note that only normal data are used for the training and only the parameters in the flow model and the decoder are updated for the stability of training process. For the encoder, we chose ImageNet- pretrained Wide ResNet-50-2 (Zagoruyko & Komodakis) 2016) as the feature extractor as it shows to be effective and and provide sufficient reception field. In the following, we provide the network details of the three blocks and the optimizer.
The encoder contains four modules, and the feature map of the fourth block were chosen as the extracted features. In more detail, each image is resized to 256 x 256 and the size of the extracted feature map is 16 x 16 with 1024 channels.
Encoder:
Flow: The flow architecture was constructed using FrEIA library (Ardizzone et al. 2018- 2022 Two types of subnets are considered for the coupling layer. One is based on Yu et al. 2021). for which each block contains two convolutional layers with ReLU activation function, and the corresponding kernel size is 3 and 1 x 1 respectively. The other is ResNet-type network with one 3 x 3 convolution layer with batch normalization and ReLU function, and shortcut connection with 1 convolution will be added the output. We find the ResNet-type network performs better in most situations, especially when it comes to large-scale datasets. The total flow steps is set be S.
Decoder: The decoder consists of four blocks. We choose transposed convolution with 2x2 kernel and stride 2 for the upsampling operation, and the channel number is symmetric with the encoder (1024 512 + 256 + 64). For the former three blocks, we employ another 3 x 3 convolutional layer with the same channel number to enhance the ability of reconstructing the image. The final transposed convolution layer with 3 channels will directly output the reconstructed image x' E F
The network parameters were acquired via minimizing the loss function using the Adam method (Kingma & Ba 2014). The momentum, batch size, learning rate and weight decay hyperparameters were set to be 0.9, 64, 2 10-4 and 10-5 respectively. The last two hyperparameters were set be 10 and 0 for the Chest X-ray dataset. The model was trained for 100 epochs.
Optimizer:
Weight parameters: For the loss function, we find the model attains the best results when weight parameters 0.5 and B 0.9, which will elaborate more in the later section.
5

---

Published as a conference paper at ICLR 2023
3.3 RESULTS
The proposed methods were compared with five deep learning-based approaches: one AE-based
method (Kramer 1991), two representative GAN-based methods: f-AnoGAN (Schlegl et al. 2019) and GANomaly (Akcay et al., 2018): and two recently proposed NF-based methods: DifferNet (Rudolph et al., 202 Fastflow (Yu et al. 2021). We refer to Zhao et al. (2021) for the quan-
titative metries. All results are reported based on the best performance on the test set. See Tab.
for the quantitative comparison of the different methods on the four medical datasets. We adopted Area Under Curves (AUC), fl score (FI), accuracy (ACC), sensitivity (SEN) and specificity (SPE) as the evaluation metries and the threshold of ACC, SEN and SPE is determined based on the best F1-score. The threshold used for evaluation is determined based on the best value of F1-score. The corresponding AUC curves for all models are also illustrated in Fig. 3 for better illustration.
For the OCT dataset, AE-FLOW achieves the best results among all methods in all metrics: AUC, F1, ACC, SEN and SPE. GAN-based methods are superior to AE method and GANomaly outper- forms f-AnoGAN. Normalizing flow-based methods, i.e. DifferNet and Fastflow, have shown to be superior while the proposed AE-FLOW method outperforms Fastflow by noticeably margin with an additional reconstruction block. Thus adding the reconstruction part in AE-FLOW is beneficial promote the detection accuracy. The Chest X-ray datasets contains significantly less data than the OCT dataset, thus the accuracy are less high than the OCT data for all the methods. We find the performance of GAN-based method can not exceed that of autoencoder on this dataset, which may be due to the insufficiency of training data. However, we find that even with a small dataset, normalizing flow-based techniques may still provide satisfactory results. By taking the advantage of normalizing flows and the reconstruction error measure, our model still achieves the best results in terms of AUC, Fl, accuracy and specificity. For the ISIC2018 dataset, the given normal category (nevus) can be easily mistaken with other categories, which makes anomaly detection challenging. For this case, our method still performs significantly superior to other methods, and the impact of and B the AUC (provided in supplementary material) also demonstrates that the flow score is a robust index. The experimental results of each method for the BraTS2021 dataset are compara- tively similar, while our method is still competitive and produces marginally improved results than the others. For the non-medical dataset MIIC, we compare with the method in Huang et al. (2021), and the results are shown in Tab 2 Although the performance of the approach in the literature is impressive, our method is still quite competitive and demonstrates its effectiveness.
Table 1: Quantitative comparison for different methods.
Dataset Metric
OCT (%)
Chest X-ray
(%)
AUC
Fl
ACC
SEN
SPE
AUC
F1
ACC
SEN
SPE
Autocncoder
83.02
88.64
82.13
92.29
51.65
81.02
81.63
73.08
94.87
36.75
F-AnoGAN (Schlegl et al.
2019
85.09
86.70
79.24
89.67
47.93
71.44
79.96
69.39
96.92
23.50
GANomaly (Akcay et al.
2018
90.52
91.09
86.16
93.53
64.05
76.14
81.18
74.36
87.69
52.14
DitferNet (Rudolph et al. 2021]
94.27
92.85
89.05
93.80
74.79
86.41
84.62
79.17
90.77
59.83
Fastllow Yuu el al. [022
94.08
92.60
88.43
95.45
67.36
90.25
86.02
80.77
93.39
59.40
Ours
98.15
96.36
94,42
96.56
88.02
92.00
88.92
85,58
91.28
76.07
Dataset
ISIC2018
(%)
BraTS2021
(5)
Metric
AUC
Fl
ACC
SEN
SPE
AUC
F1
ACC
SEN
SPE
Autocncoder
73.97
62.57
64.77
78.57
56.91
70.38
82.47
70.79
98.38
07.63
f-AnoGAN (Schlegl et al.
2019
79.80
67.03
68.39
85.71
58.54
78.06
84.89
76.42
94.77
34.42
GANomaly (Akcay el al.
2018
72.54
60.95
56.99
90.00
38.21
74.50
82.33
70.86
97.24
10.46
DitferNet (Rudolph et al.
2021]
76.52
64.58
64.25
87.14
51.22
78.94
83.78
74.70
93.53
31.59
Fastllow YYu el al. 2021
80.81
69.95
70.98
90.00
60.16
80.94
85.37
78.21
90.96
49.02
Ours
87.79
80,56
84.97
81.43
86,99
81,58
86.06
78.74
93.91
44.01
Table 2: Quantitative comparison for MIIC dataset.
Method
AUC
Fl
TPR
FPR
f-anoGAN(Schleg) et al. 2019)
56.38
31.65
18.96
00.08
GANomaly (Akeay et al. 2018)
93.34
74.64
67.24
01.18
Proposed in Huang et (2021)
97.28
83.48
78.45
00.86
Ours
98.69
89.62
81.03
00.07
6

---

Published as a conference paper at ICLR 2023
OCT
Chest X-ray
1.0
1.0
True Positive Rate
Asto-enzoder (AUC 0.5350)
Auto-encode: (AUC 0.8102)
F-AnGGAN (AUC 6.bb6))
FAnoGAN (ALC 0.114)
GANomaly (AUC 0.9052)
GANomaly (AUC 0.7014)
DiFerNee (AUC 0.9427)
DiffoNee (AUC 0.8611)
Fostflw (AUC (.900))
Fastlow (AUC 0.9025)
Ours (AUC 0.9815)
0.0
(AUC 0.9200)
0.0
0.5
1.0
0.0
0.5
1.0
ISIC2018
Bra TS2021
1.0
1.0
True Positive Rate
0.5
ancodencoder 0.7307)
0.5
Auto-ancoder 0.7058)
LAncCAN (AUC 0.7980)
FAncCAN (ALC 0.7806)
GANomaly
Difieiter (AUC 0.7652)
Difieriee (AUC 0.7864)
Fastflow c.808))
0.3094)
0.0
Our (AUI 0.8774)
0.0
Our (AUC = 0.8158)
0.0
0.5
1.0
0.0
False Positive Rate
False Positive Rate
Figure 3: AUC curves for different methods. Our method attains the best AUC score for all four medical datasets.
To further validate the effectiivnesss, the distribution curves of the computed anomaly score for the two categories, i.c. normal and anomaly are shown Fig. The horizontal axis of the figure represents the anomaly score and the vertical axis represents the probability density. It is shown that the predicted anomaly score of anomaly data is commonly larger than that of normal data. Moreover, the overlapping takes small percentage of the total area, and each curve hits its own unique peak, which demonstrates that the proposed anomaly score function is a valid and effective measure for the classification of normal and abnormal samples.
Compared to flow-hased method, the proposed AE-FLOW model allows to provide some visual interpretability of the anomaly detection model. In Fig. and Fig. 6 we show the original, the reconstructed and the residual images for some normal and abnormal samples from the OCT and Chest X-ray datasets respectively. For the OCT dataset, the original images x reconstructed images x' and residual images x x'I are shown in Fig. 5 It shows that the reconstruction image smooth out the zig-zag shape structure, which is identified as the lesion part the original image. In contrast, the normal data do not contain these structures and the corresponding reconstructed image are much closer the input images with negligible reconstruction error, and this difference helps to distinguish the anomaly and provide visual interpretability of the detection model. For the chest X-ray dataset, the difference between the original and the reconstructed image for anomaly data more obvious. Most of X-ray images of pneumonia contain transparent white areas while there less such structures in normal data. These structures are not present in the reconstructed images based the AE-FLOW learned from normal data. Thus the differences between the original image and the reconstructed image are in a higher level than that of normal data.
3.4 ABLATION STUDY
As the proposed model composed of the reconstruction emro and the flow likelihood for both loss function and anomaly score, investigated the performance of difference choices of the two weights parameters a and 3 in the loss (equation []] and anomaly score (equation ( respectively. Fig gives the AUC value of the proposed model with different and (3 ranged in [0. 1]. When

---

Published as a conference paper at ICLR 2023
OCT
Chest X-ray
ISIC2018
Bra TS2021
11.63
11.00
-1.53
-1.30
-0.50
-0.25
6.000
-IL90 -0.55
-I1.75
-1..0
-1.25
Anomaly Score
Anomaly Score
Anomaly Score
Anomaly Score
Anamaly
Normal
Figure 4: Distribution curves of the anomaly score the anomaly and normal data. We can see two distributions have different peaks and the ratio of overlapping areas is relatively small, especially in the OCT dataset.
Abnormal
Normal
Figure 5:
The comparison of normal and abnormal samples from the OCT dataset. For each sample, show the original, reconstructed and residual images from top to bottom respectively.
a= 3 = 1.0, only the flow loss used for the training and only the flow score is used for anomaly detection. By letting 0.0 and we can see that the AUC get improved (96 - 97 for the OCT dataset, 90 - 92 for the Chest X-ray dataset) even if only the flow score is used the anomaly score (B 1.0). This shows that the reconstruction loss (Croron) positively affects the flow model. Specifically, for both datasets, = 0.5, 8 0.9, the best results be achieved. Furthermore, it can be seen that the proposed model stably attains high AUC for a relatively large range of parameters a and 3, which implies the robustness of the proposed model.
< 1.0, the reconstruction error is introduced in the loss function
In Tab. we show the AUC score under different settings for the subnet of the NF model and the reconstruction loss function using MSE SSIM. The results show that the model with these four dif- ferent combinations still lead to stably high perforamnce, compared to other methods. ResNets with MSE loss achieves the best AUC for the OCT 15), ISIC2018 (87.79) and BraTS2021 (81.58) dataset, while 2-Layer CNN with SSIM achievest the best for Chest X-ray (92.00) dataset, which may due to the facts that the anomaly in OCT datasets are mainly due to local lesions while the anomaly in Chest X-ray dataset more spread out.
Table 3: AUC Comparison under different types of reconstruction loss and NF subnet architecture.
OCT (%)
Chest X-ray
(%)
2-Layer CNN
ResNets
2-Layer CNN
ResNets
MSE 96.43
98.15
91.69
89.92
SSIM
94.89
95.31
92.00
90.47
ISIC2018
(%)
BraTS2021
(%)
2-Layer CNN
ResNets
2-Layer CNN
ResNets
MSE 81.19
87.79
81.55
81.58
SSIM
81.22
87.09
78.95
80.33
8

---

Published as a conference paper at ICLR 2023
Abnormal
Normal
Figure 6: The comparison of the original, reconstructed and residual images (from top bottom) of normal and abnormal samples from the Chest X-ray dataset.
OCT
Chest
X-ray
86
87
86
84
79
66
59
80
80
79
80
79
80
80
80
79
50
83
84
87
89
91
93
95
96
96
96
95
84
86
88
89
90
90
91
91
91
91
91
85
89
93
96
97
97
95
82
86
88
90
91
92
85
90
94
97
83
87
89
Loss ratio
83
86
88
90
93
94
96
96
96
96
95
Loss ratio
0.4
83
86
88
89
89
90
90
90
90
90
90
84
85
87
89
91
93
95
97
98
98
97
81
85
88
89
91
91
92
92
92
92
92
9.6
86
90
94
0.6
77
82
91
84
85
89
93
95
96
96
96
95
77
82
86
89
90
91
91
91
91
83
85
88
90
92
94
95
96
97
96
95
0.8
82
86
88
90
90
91
91
91
91
91
91
84
89
93
96
96
85
88
90
90
91
96
96
96
96
1.0
38
90
90
90
0.0
0.2
0.4
0.6
0.6
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Anomaly score ratio
Anomaly score ratio 8
Figure 7: AUC scores under different loss ratio and score ratio for the OCT (left) and Chest X-ray (right) datasets respectively.
4 CONCLUSION
In this paper, we proposed AE-FLOW, self-supervised model that integrates the tractable likelihood of the NF model and the image dissimilarity between the input and the reconstructed images for the representation of normal data and detecting medical anomalies, The proposed anomaly score function takes into account both the image reconstruction error and the likelihood in feature space for classifying normal and abnormal data, by learning a representation model only from normal data. The method provides not only image-level tractability of normal data but also pixel-level interpretability of anomaly data. Experiments on different medical image datasets demonstrated the effectiveness and robustness of the proposed approach, with large margin of improvement over other related and representative approaches for anomaly detection.

---

Published as a conference paper at ICLR 2023
ACKNOWLEDGMENTS
This work was supported by Shanghai Municipal Science and Technology Major Project (2011SH2DZX1002) and NSFC (No. 12090024 and 12201402). We thank the Student Innova- tion Center at Shanghai Jiao Tong University for providing us the computing services.
REFERENCES
Samet Akcay, Amir Abapou-Abaaghoueii, and Toby P Breckon. Ganomaly: Semi-supervised anomaly detection via adversarial training. In Asian conference on computer vision, pp. 622- 637. Springer, 2018.
Mohd Belal Al-Zoubi. An effective clustering- based approach for outlier detection. European Journal of Scientific Research, 28(2):310-316 2009.
Fabrizio Angiulli, Stefano Basta, and Clara Pizzuti. Distance-based detection and prediction of outliers. IEEE transactions knowledge and data engineering, 18(2):145-160, 2005.
Lynton Ardizzone, Till Bungert, Felix Draxler, Ullrich Köthe, Jakob Kruse, Robert Schmier, and Peter Sorrenson. Framework for Easily Invertible Architectures (FrEIA), 2018-2022. URL
Ujjwal Baid, Satyam Ghodasara, Suyash Mohan, Michel Bilello, Evan Calabrese, Errol Colak, Key- van Farahani, Jayashree Caapathy-Cramed Felipe C Kitamura, Sarthak Pati, et al. The rsna-asnr- miccai brats 2021 benchmark on brain tumor segmentation and radiogenomic classification. arXiv preprint arXiv:2107.02311 2021.
Spyridon Bakas, Hamed Akbari, Aristeidis Sotiras, Michel Bilello, Martin Rozycki, Justin Kirby, John B Freymann, Keyvan Farahani, and Christos Davatzikos. Advancing the cancer genome atlas glioma mri collections with expert segmentation labels and radiomic features. Scientific data, (1):1-13, 2017.
Markus M Breunig, Hans-Peter Kriegel, Raymond T Ng, and Jörg Sander. Lof: identifying density- based local outliers. In Proceedings of the 2000 ACM SIGMOD international conference Management of data, pp. 93-104, 2000.
Raghavendra Chalapathy and Sanjay Chawla. Deep learning for anomaly detection: A survey, arXiv preprint arXiv: 00340777 2019.
Noel Codella, Veronica Rotemberg, Philipp Tschandl, Emre Celebi, Stephen Dusza, David Gut- man, Brian Helba, Aadi Kalloo. Konstantinos Liopyris. Michael Marchetti, et al. Skin lesion analysis toward melanoma detection 2018: A challenge hosted by the international skin imaging collaboration (isic). arXiv preprint arXiv:1902.0333 2019.
Laurent Dinh, David Krueger, and Yoshua Bengio. Nice: Non-linear independent components esti- mation. arXiv preprint
2014.
Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. Density estimation using real nvp. arXiv preprint arXiv:1605. N0X.:0600500803, 2016.
Tharindu Fernando, Harshala Gammulle, Simon Denman, Sridha Sridharan, and Clinton Fookes. Deep learning for medical anomaly detection a survey, ACM Computing Surveys (CSUR), 54(7): 1-37, 2021.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neurul information processing systems, 27, 2014.
Denis Gudovskiy, Shun Ishizaka, and Kazuki Kozuka. Cflow-ad: Real-time unsupervised anomaly detection with localization via conditional normalizing flows. In Proceedings of the IEEE/CVF Winter Conference Applications of Computer Vision, pp. 98-107 2022.
10

---

Published as a conference paper at ICLR 2023
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Shohei Hido, Yuta Tsuboi, Hisashi Kashima, Masashi Sugiyama, and Takafumi Kanamori. Statis- tical outlier detection using direct density ratio estimation. Knowledge and information systems, 26(2):309-333 2011.
Ling Huang, Deruo Cheng, Yang Xulei, Lin Tong, Shi Yiqiong, Kaiyi Yang, Gwee Bah-Hwee, and Wen Bihan. Joint anomaly detection and inpainting for microscopy images via deep self- supervised learning. In IEEE International Conference on Image processing (ICIP), 2021.
Daniel S Kermany, Michael Goldbaum, Wenjia Cai, Carolina CS Valentim, Huiying Liang, Sally Baxter, Alex McKeown, Ge Yang, Xiaokang Wu, Fangbing Yan, et al. Identifying medical diag- noses and treatable diseases by image-based deep learning. Cell, 1222-1111, 2018.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint 1412.690, 2014.
Diederik Kingma and Max Welling. Auto-encoding variational bayes. arXiv: 14, 2013.
arXiv preprint
Durk P Kingma and Prafulla Dhariwal. Glow: Generative flow with invertible 1xl convolutions. Advances in neural information processing systems, 31, 2018.
Edwin M Knorr, Raymond Ng, and Vladimir Tucakov. Distance- outliers: algorithms and applications. The VLDB Journal, 8(3):237-253) 2000.
Mark A Kramer. Nonlinear principal component analysis using autoassociative neural networks.
AIChE journal, 37(2):333-243, 1991.
Yuchen Lu and Peng Xu. Anomaly detection for skin disease images using variational autoencoder. arXiv preprini arXiv: 1807.01349 2018
Bjoern H Menze, Andras Jakab, Stefan Bauer, Jayashree Kalpathy- Crammr,, Keyvan Farahani, Justin Kirby, Yuliya Burren, Nicole Porz, Johannes Slotboom, Roland Wiest, et al. The multimodal brain tumor image segmentation benchmark (brats). IEEE transactions on medical imaging, 34 (10):1993-2024, 2014.
Guansong Pang, Chunhua Shen, Longbing Cao, and Anton Van Den Hengel. Deep learning for anomaly detection: A review. ACM Computing Surveys (CSUR), 54(2):1-38, 2021.
George Papamakarios, Eric Nalisnick, Danilo Jimenez Rezende, Shakir Mohamed, and Balaji Lak- shminarayanan. Normalizing flows for probabilistic modeling and inference. Journal of Ma- chine Learning Research, (2(5)):1-64, 2021. URL lttt::// /m/r.. orq/papers/v22/
119-108..0 html
M. Ravanbakhsh, M. Nabi, E. Sangineto, L. Marcenaro, C. Regazzoni, and N. Sebe. Abnormal event detection in videos using generative adversarial nets. IEEE, 2017.
Danilo Rezende and Shakir Mohamed. Variational inference with normalizing flows In Interna- tional conference on machine learning, pp. 1530-1538. PMLR, 2015.
Peter Rousseeuw and Mia Hubert. Robust statistics for outlier detection. Wiley interdisciplinary reviews: Data mining and knowledge discovery, 1(1):73-79, 2011.
Marco Rudolph, Bastian Wandt, and Bodo Rosenhahn. Same same but differnet: Semi-supervised defect detection with normalizing flows. In Proceedings of the IEEF/CVF winter conference on applications of computer vision, pp. 1907-1916, 2021.
Thomas Schlegl, Philipp Seeböck, Sebastian M Waldstein, Ursula Schmidt- -rfuuth,, and Georg Langs. Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. In International conference on information processing in medical imaging, pp. 146- 157. Springer, 2017.
11

---

Published as a conference paper at ICLR 2023
Thomas Schlegl, Philipp Seeböck, Sebastian M Waldstein, Georg Langs, and Ursula Schmidt- Erfurth. F-anogam: Fast unsupervised anomaly detection with generative adversarial networks. Medical image analysis, 54:30 44, 2019.
Alberto Taboada-Crispi, Hichem Sahli, Denis Hernandez-Pachecoo, and Alexander Falcon-Ruiz. Anomaly detection in medical image analysis. In Handbook of research on advanced techniques in diagnostic imaging and biomedical applications, pp. 426-446. IGI Global, 2009.
Philipp Tschandl, Cliff Rosendahl, and Harald Kittler. The ham 1000 dataset, large collection of multi-source dermatoscopic images of common pigmented skin lesions. Scientific data, 5(1):1-9, 2018.
Hongzhi Wang, Mohamed Jaward Bah, and Mohamed Hammad. Progress in outlier detection tech- niques: A survey. leee Access, 7:107964-108000, 2019.
Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Ecro Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4):600- 612, 2004.
Xingwei Yang, Longin Jan Latecki, and Dragoljub Pokrajac. Outlier detection with globally opti- mal exemplar-based gmm. In Proceedings the 2009 SIAM International Conference Data Mining, pp. 145-154. SIAM, 2009.
Jiawei Yu, Ye Zheng, Xiang Wang, Wei Li, Yushuang Wu, Rui Zhao and Liwei Wu. Fast- flow: Unsupervised anomaly detection and localization via 2d normalizing flows. arXiv preprint arXiv:2111.07677, 2021.
Sergey Zagoruyko and Nikos Komodakis. 1rXiv:1605.07146, 2016.
Wide residual networks.
arXiv preprint
He Zhao, Yuexiang Li, Nanjun He, Kai Ma, Leyuan Fang, Huiqi Li, and Yefeng Zheng. Anomaly de- tection for medical images using self supersupe and translation-consie features. IEEE Trans- actions on Medical Imaging, (111)):641-36511 2021.
12

---

Published as a conference paper at ICLR 2023
A CROSS VALIDATION
We performed five-fold cross-validation experiments for all four medical datasets to further evaluate the robustness and effectiveness of our proposed method. We randomly split the initial test into validation and test sets in the ratio of 2:8. In order to maintain the balance of each category, we keep the original percentage of normal and abnormal data in each set. The training settings are the same as Section 3.2. We list each metric's s means and standard deviations for the five rotations in the Tab.
Table 4: Cross Validation Results for different medical datasets. ACC
Metric
AUC
SEN
SPE
OCT
96.7840.46
95. 95.1310.38
92.5240.61
95.5641.25
83.40.4..58
Chest X-ray
89.77+1.68
86.6242.35
81.8044.14
92.1842.51
64.55+114.20
ISIC2018
87.37
78.4343.20
81.4243.32
87.8641.75
77.78.5..1
Bra22
80.22+0.27
85.9940.15
78.6040.61
93.89+1.90
43.54+4.53
As shown in the table, our approach continues to perform well across all datasets. Due to the size of the dataset, the Chest X-ray and ISIC2018 datasets have more fluctuating outcomes than the larger OCT and BraTS2021 datasets.
B
MORE RESULTS FOR THE RATIOS
ISIC2018
Bra SS021
42
41
42
42
43
44
46
47
48
48
47
0
61
59
56
53
49
47
44
42
38
35
27
54
57
61
65
70
75
79
83
85
86
86
68
70
71
73
74
75
76
77
78
78
79
02
59
63
67
71
76
79
82
84
84
84
84
0.2
70
72
74
76
77
78
79
80
80
80
81
49
50
53
56
60
66
72
78
83
85
85
71
74
76
77
78
79
79
80
80
80
80
0.4
69
70
71
73
74
76
78
80
82
84
83
04
70
73
75
77
78
79
80
80
81
81
81
Loss ratio
62
64
66
69
72
76
80
83
86
88
88
68
73
76
78
79
80
81
81
81
82
82
Loss 0.6
74
75
76
76
77
79
80
82
84
84
06
30
55
66
72
76
78
80
81
81
82
60
64
68
75
82
87
88
69
74
79
80
80
81
81
0,8
59
62
65
74
82
87
86
08
70
75
78
80
81
76
82
85
85
69
73
76
80
80
1.0
87
87
87
87
1.0
0.0
0.2
0.4
0.6
0.6
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Anomaly score ratio
B
Anomaly score ratio 8
Figure 8: AUC scores under different loss ratio and score ratio 3 for ISIC2018 and BraTS2021 datasets.
The variations of AUC scores with different loss ratios and score ratios 3 for the other two medical datasets are displayed in the Fig. Similar to the pattern in Fig. 7 we see that the flow part's accuracy higher than the reconstruction part's, 's, but after adding the reconstruction part for training
(a < 1), the performance of the likelihood-oni 80 + 82 for BraTS2021). Specifically, we can still achieve best results for = 0.5 and 8 = 0.9, and we report our experiments under this setting.
anomaly score rises (87 + 88 for ISIC2018 and
C VISUALIZATION FOR MORE DATASETS
13

---

Published as a conference paper at ICLR 2023
Abnormal
Normal
Figure 9: The comparison of normal and abnormal samples from the ISIC2018 dataset. For each sample, we show the original, reconstructed and residual images from left right respectively.
Abnormal
Normal
Figure 10: The comparison of normal and abnormal samples from the BraTS2021 dataset. For each sample, we show the original, reconstructed and residual images from left to right respectively.
Abnormal
Normal
Figure 11: The comparison of normal and abnormal samples from the MIIC dataset. For each sample, we show the original, reconstructed and residual images from left right respectively.
14


Reviews:
[{"review_id": "gJPlpOlgqr", "reviewer": "Reviewer_Ngn6", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a normalizing flow-based autoencoder (AE-Flow) to address medical image anomaly detection.  The AE-Flow model contains three components, an encoder to extract features of the input images, a flow model to convert extracted feature vector to the specific feature space, which follows a standard normal distribution, and a decoder to reconstruct the image from normally distributed data. In the inference stage, a flow score and a reconstruction loss are combined into the final anomaly score. Its contribution to medical image anomaly detection is to adopt the normalizing flow-based method to supervise the model training deeply. ", "summary_of_the_review": "This paper wants to address medical image anomaly detection. The proposed AE-Flow method combines autoencoder and normalizing flows into a multi-task architecture. It is more common for anomaly detection, and could not convince me to accept this paper. ", "strength_and_weaknesses": "Strengths:\n\n1. This paper adopts a normalizing flow for anomaly detection. The final anomaly score consists of a negative probability density of the normalized feature and reconstruction error, which supervises the model training at both image and pixel levels. \n2. Two different datasets (OCT and CT) are used to conduct some comparison experiments to evaluate the effectiveness.\n\nWeaknesses:\n1. Direct anomaly detection is little help for medical image diagnosis.  It would be more meaningful to mark the abnormal areas, especially small lesions. \n2. It is a standard solution to combine auto-encoder with the normalizing flow for anomaly detection. \n3. It'd be better to explain why α and β values are sensitive for the final performance. If we can unify and fix α and β parameters, it will greatly reduce the model training time and will be more available for users. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "fsJNGiyTsOj", "reviewer": "Reviewer_hWE4", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper introduces an auto-encoder with the normalizing flow bottleneck called AE-FLOW for anomaly detection in medical images. The model combines the benefits of normalizing flow methods for computing the anomaly likelihood of extracted features at image level, and the interpretability of reconstruction-based methods at pixel-level.", "summary_of_the_review": "The study sounds interesting. I think this is a promising mechanism for detecting anomalies in medical images. The problem is that the proposed approach has not been validated with enough experiments and diverse datasets. \n\nIt would be nice if the authors could cite a few relevant papers (page 1, second para, first sentence) for their assertions. In my opinion, it is not acceptable by asserting only “One may refer to the review paper (Wang et al., 2019) for more references.” (page1, second para)\n\nWe cannot draw conclusions about the robustness of a model based on limited experiments. Thus, I have some reservations on the robustness, quality, and novelty of the study. Overall, I am not fully convinced of the contribution of the paper that would lead me to suggest it for ‘acceptance’.\n\n", "strength_and_weaknesses": "Strength:\n- A promising mechanism focusing on anomaly detection from medical images;\n- propose to construct a loss function and anomaly score function;\n- a representation model learns only from normal data;\n- provides image-level tractability of normal data as well as pixel-level interpretability of anomaly data.\n\nWeaknesses:\n- Experiments are conducted in a limited dataset.\n- limited technical contribution.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "5QQD4hpi-z", "reviewer": "Reviewer_XDoM", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors present AE-FLOW, an autoencoder-based method for anomaly detection using normalizing flows in the latent space. Anomaly detection is an important problem in medical imaging, becuase in some domains, screening for abnormal diagnoses is critical. The authors propose using normalizing flows to learn an efficient and tractable distribution of medical images. An anomaly score is calculated using both the negative probability density and the negative reconstruction error. The encoder and decoder consist of conventional convolutional layers with downsampling and upsampling, respectively. The normalizing flow bottleneck consists of eight flow blocks. The model contains two major hyperparameters: one that interpolates between the flow and reconstruction losses and one that interpolates between the flow and reconstruction scores. The authors use two medical image dataset, OCT images and chest xrays. They were able to show that AE-FLOW outperforms other anomaly detection baselines on several metrics. An ablation study was performed that that demonstrated both the flow and reconstruction losses are important for overall performance. The authors show qualitative results using residual images generated from the original and reconstruction images. Abnormal images show much large residual values compared to normal images. ", "summary_of_the_review": "The authors present a novel application of normalizing flows for medical image anomaly detection with improved performance over previously benchmarked models. ", "strength_and_weaknesses": " Strengths:\n1) The paper is very well written, clear, and easy to follow. \n2) The figures are well done and informative. \n3) The paper addresses a generally important problem for screening of diagnosis images.\n4) This is a good use of normalizing flows. \n\nWeakness:\n1) I would ask that the authors provide some additional explanation or evidence for the performance boost compared to other benchmarked models. The authors do a good job presenting the results and show good performance over the other baselines; but I am still unsure as why the model is outperforming the others. \n\n2) If one is to use this model for inference on a new example, how would you select a threshold for the anomaly score? The authors seem to have selected an anomaly score based on the testing set F1 scores. For a better model evaluation, identifying an anomaly score using a validation set, then testing on a true held out testing set seems to be the best way to determine the accuracy, sensitivity, specificity, etc. of AE-FLOW. This cross-validation experiment can be added to an appendix. \n\n3) The method applies in general for anomaly detection. Providing results in a non-medical dataset would strengthen the experimental results for AE-FLOW.  This could also be supplemental data. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster