Original: AE-FLOW: A UTOENCODERS WITH N ORMALIZING F LOWS FOR M EDICAL I MAGES A NOMALY D ETECTION
Revised: AE-FLOW: A UTOENCODERS WITH N ORMALIZING F LOWS FOR M EDICAL I MAGES A NOMALY D ETECTION

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Anomaly detection from medical images is an important task for clinical screening and diagnosis.
Revised: Anomaly detection from medical images is an important task for clinical screening and diagnosis.

Original: In general, a large dataset of normal images are available while only few abnormal images can be collected in clinical practice.
Revised: In general, a large dataset of normal images are available while only few abnormal images can be collected in clinical practice.

Original: By mimicking the diagnosis process of radiologists, we attempt to tackle this problem by learning a tractable distribution of normal images and identify anomalies by differentiating the original image and the reconstructed normal image.
Revised: By mimicking the diagnosis process of radiologists, we attempt to tackle this problem by learning a tractable distribution of normal images and identify anomalies by differentiating the original image and the reconstructed normal image.

Original: More specifically, we propose a normalizing flow based autoencoder for an efficient and tractable representation of normal medical images.
Revised: More specifically, we propose a normalizing flow based autoencoder for an efficient and tractable representation of normal medical images.

Original: The anomaly score consists of the likelihood originated from the normalizing flow and the reconstruction error of the autoencoder, which allows to identify the abnormality and provide an interpretability at both image and pixel levels.
Revised: The anomaly score consists of the likelihood originated from the normalizing flow and the reconstruction error of the autoencoder, which allows to identify the abnormality and provide an interpretability at both image and pixel levels.

Original: Experimental evaluation on four medical and one non-medical images datasets showed that the proposed model outperformed the other approaches by a large margin, which validated the effectiveness and robustness of the proposed method.
Revised: Experimental evaluation on four medical and one non-medical images datasets showed that the proposed model outperformed the other approaches by a large margin, which validated the effectiveness and robustness of the proposed method.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Medical anomaly detection (Taboada-Crispi et al., 2009; Fernando et al., 2021) is an important task in clinical screening and diagnosis by capturing distinctive features in collected biomedical data, such as medical images, electrical biomedical signals or other laboratory results.
Revised: Medical anomaly detection (Taboada-Crispi et al., 2009; Fernando et al., 2021) is an important task in clinical screening and diagnosis by capturing distinctive features in collected biomedical data, such as medical images, electrical biomedical signals or other laboratory results.

Original: Anomaly detection aims to detect data that significantly deviates from the majority of data instances, arising in clinical applications due to imbalance between normal and abnormal data and variability of anomaly in real world scenario.
Revised: Anomaly detection aims to detect data that significantly deviates from the majority of data instances, arising in clinical applications due to imbalance between normal and abnormal data and variability of anomaly in real world scenario.

Original: Different to usual classification models used for computed aided diagnosis, anomaly detection is usually considered in an unsupervised or semi-supervised paradigm.
Revised: Different to usual classification models used for computed aided diagnosis, anomaly detection is usually considered in an unsupervised or semi-supervised paradigm.

Original: In this paper, we mainly focus on anomaly detection from medical images, to mimic the diagnosis process of radiologists.
Revised: In this paper, we mainly focus on anomaly detection from medical images, to mimic the diagnosis process of radiologists.

Original: The traditional techniques for finding anomalies are divided into several categories, such as statistics-based methods (Hido et al., 2011; Rousseeuw & Hubert, 2011), distance-based methods (Knorr et al., 2000; Angiulli et al., 2005), density-based methods (Breunig et al., 2000), and clustering-based methods(Yang et al., 2009; Al-Zoubi, 2009), etc.
Revised: The traditional techniques for finding anomalies are divided into several categories, such as statistics-based methods (Hido et al., 2011; Rousseeuw & Hubert, 2011), distance-based methods (Knorr et al., 2000; Angiulli et al., 2005), density-based methods (Breunig et al., 2000), and clustering-based methods(Yang et al., 2009; Al-Zoubi, 2009), etc.

Original: Deep learning for anomaly detection (Wang et al., 2019; Chalapathy & Chawla, 2019; Pang et al., 2021), also known as deep anomaly detection, typically consists of learning a feature representation model of normal images and constructs an anomaly score function for abnormal images by neural networks.
Revised: Deep learning for anomaly detection (Wang et al., 2019; Chalapathy & Chawla, 2019; Pang et al., 2021), also known as deep anomaly detection, typically consists of learning a feature representation model of normal images and constructs an anomaly score function for abnormal images by neural networks.

Original: There are two main types of approaches for image anomaly detection, one is reconstruction based model and the other is likelihood based model in the literature.
Revised: There are two main types of approaches for image anomaly detection, one is reconstruction based model and the other is likelihood based model in the literature.

Original: The standard procedure of reconstruction based methods is to first learn an auto-encoder (AE) (Kramer, 1991) or generative models (Goodfellow et al., 2014) for normal images and the difference between the test and reconstructed (generated) images through the representation neural networks can be used to characterize the level of anomaly.
Revised: The standard procedure of reconstruction based methods is to first learn an auto-encoder (AE) (Kramer, 1991) or generative models (Goodfellow et al., 2014) for normal images and the difference between the test and reconstructed (generated) images through the representation neural networks can be used to characterize the level of anomaly.

Original: For example, AnoGAN (Schlegl et al., 2017) is a generative adversarial networks (GAN) based model utilizing a generator for image reconstruction and an anomaly score using a weighted sum of residual socre and discrimination score.
Revised: For example, AnoGAN (Schlegl et al., 2017) is a generative adversarial networks (GAN) based model utilizing a generator for image reconstruction and an anomaly score using a weighted sum of residual socre and discrimination score.

Original: In
Revised: In

Original: Akcay et al.
Revised: Akcay et al.

Original: , GANomaly considers the distance in the latent feature space to distinguish the anomaly data.
Revised: , GANomaly considers the distance in the latent feature space to distinguish the anomaly data.

Original: F-anoGAN (Schlegl et al., 2019) is an improved version of anoGAN, which simultaneously guides encoder training in both image and latent spaces.
Revised: F-anoGAN (Schlegl et al., 2019) is an improved version of anoGAN, which si- multaneously guides encoder training in both image and latent spaces.

Original: Even reconstruction base methods could give the dissimilarity in pixel level, many limitations exists for anomaly detection tasks.
Revised: Even reconstruction base methods could give the dissimilarity in pixel level, many limitations exists for anomaly detection tasks.

Original: The limited capabilities of AEs in modelling high-dimensional data distribution often lead to inaccurate approximations and erroneous reconstructions, such as features are smoothed out in the reconstructed images (Kingma & Welling, 2013; Schlegl et al., 2019; Ravanbakhsh et al., 2017) and image boundaries are predicted as anomaly pixels and appeared in the difference (Schlegl et al., 2017; 2019).
Revised: The limited capabilities of AEs in modelling high-dimensional data distribution often lead to inaccurate approximations and erroneous reconstructions, such as features are smoothed out in the reconstructed images (Kingma & Welling, 2013; Schlegl et al., 2019; Ravanbakhsh et al., 2017) and image boundaries are predicted as anomaly pixels and appeared in the difference (Schlegl et al., 2017; 2019).

Original: The second type of anomaly detection method is to construct a likelihood function of extracted image features.
Revised: The second type of anomaly detection method is to construct a likelihood function of extracted image features.

Original: For example, normalizing flow based anomaly detection models have been proposed for industry anomaly datasets (Rudolph et al., 2021; Gudovskiy et al., 2022; Yu et al., 2021).
Revised: For example, normalizing flow based anomaly detection models have been proposed for industry anomaly datasets (Rudolph et al., 2021; Gudovskiy et al., 2022; Yu et al., 2021).

Original: Normalizing flow (Dinh et al., 2014; Rezende & Mohamed, 2015; Dinh et al., 2016; Kingma & Dhariwal, 2018) is a popular method that transform the observed data to a tractable distribution.
Revised: Normalizing flow (Dinh et al., 2014; Rezende & Mohamed, 2015; Dinh et al., 2016; Kingma & Dhariwal, 2018) is a popular method that transform the observed data to a tractable distribution.

Original: The idea behind normalizing flows is deploying a sequence of invertible and differentiable mappings to transform a complex distribution into a simple probability distribution (e.g., a standard normal distribution).
Revised: The idea behind normalizing flows is deploying a sequence of invertible and differentiable mappings to transform a complex distribution into a simple probability distribution (e.g., a standard normal distribution).

Original: NICE (Dinh et al., 2014) was introduced for modeling complex high-dimensional densities.
Revised: NICE (Dinh et al., 2014) was introduced for modeling complex high-dimensional densities.

Original: In Dinh et al.
Revised: In Dinh et al.

Original: (2016), RealNVP was proposed to improve the coupling layer and a multi-scale architecture was applied to enhance the representative ability of the framework.
Revised: (2016), RealNVP was proposed to improve the coupling layer and a multi-scale architecture was applied to enhance the representative ability of the framework.

Original: Recently, Glow proposed in Kingma & Dhariwal (2018) utilizes actnorm and 1 × 1 convolution to simulate more realistic images.
Revised: Recently, Glow proposed in Kingma & Dhariwal (2018) utilizes actnorm and 1 × 1 convolution to simulate more realistic images.

Original: Normalizing flows have been applied in many fields such as image generation, noise modeling, video generation (Papamakarios et al., 2021).
Revised: Normalizing flows have been applied in many fields such as image generation, noise modeling, video generation (Papamakarios et al., 2021).

Original: Recently, some anomaly detection methods based on normalizing flows have also emerged and have achieved very high performance for industrial datasets.
Revised: Recently, some anomaly detection methods based on normalizing flows have also emerged and have achieved very high performance for industrial datasets.

Original: Differnet (Rudolph et al., 2021) used a backbone network to extract multi-scale features of the input and a normalizing flow network to maximize the likelihood.
Revised: Differnet (Rudolph et al., 2021) used a backbone network to extract multi-scale features of the input and a normalizing flow network to maximize the likelihood.

Original: The anomaly score is defined as the average of negative log-likelihoods.
Revised: The anomaly score is defined as the average of negative log-likelihoods.

Original: CFLOW-AD (Gudovskiy et al., 2022) added positional embedding layers and utilizes conditional flows to complete the location problems.
Revised: CFLOW-AD (Gudovskiy et al., 2022) added positional embedding layers and utilizes conditional flows to complete the location problems.

Original: FastFlow (Yu et al., 2021) proposed a lightweight network and further enhances the accuracy.
Revised: FastFlow (Yu et al., 2021) proposed a lightweight network and further enhances the accuracy.

Original: However, these NF-based anomaly detection methods make decisions by estimating the likelihood of the extracted features, and the structural information of images are missing.
Revised: However, these NF-based anomaly detection methods make decisions by estimating the likelihood of the extracted features, and the structural information of images are missing.

Original: From this perspective, an auto-encoder architecture can construct a mapping between the hidden feature space and the data space, and enforce the model to learn structural information of the original data.
Revised: From this perspective, an auto-encoder architecture can construct a mapping between the hidden feature space and the data space, and enforce the model to learn structural information of the original data.

Original: In this paper, we propose to construct a loss function and anomaly score function by an auto-encoder with the normalizing flow bottleneck, namely AE-FLOW.
Revised: In this paper, we propose to construct a loss function and anomaly score function by an auto-encoder with the normalizing flow bottleneck, namely AE-FLOW.

Original: This model combines the benefits of normalizing flow methods for computing the anomaly likelihood of extracted features at image level, and the interpretability of reconstruction-based methods at pixel-level.
Revised: This model combines the benefits of normalizing flow methods for computing the anomaly likelihood of extracted features at image level, and the interpretability of reconstruction-based methods at pixel-level.

Original: The proposed score function takes both computable probability density in feature features and visual structures consistency in image domain into consideration.
Revised: The proposed score function takes both computable probability density in feature features and visual structures consistency in image domain into consideration.

Original: The model is in a self-supervised paradigm that uses only normal data, which is more adaptive to real world applications.
Revised: The model is in a self-supervised paradigm that uses only normal data, which is more adaptive to real world applications.

Original: The experiments on four public medical datasets and one non-medical dataset demonstrated the effectiveness and robustness of the proposed method, with a detail comparison to other existing methods.
Revised: The experiments on four public medical datasets and one non-medical dataset demonstrated the effectiveness and robustness of the proposed method, with a detail comparison to other existing methods.

Original:  M ETHODS
Revised:  M ETHODS

Original: We propose a method that integrates the difference on image structures of the original and reconstructed image and the likelihood of image features distribution.
Revised: We propose a method that integrates the difference on image structures of the original and reconstructed image and the likelihood of image features distribution.

Original: The proposed pipeline consists of three important components, i.e., encoder-flow-decoder.
Revised: The proposed pipeline consists of three important components, i.e., encoder-flow-decoder.

Original: Intuitively, normal data will be mapped to the high density area of the standard Gaussian distribution as normalizing flows are trained with normal data.
Revised: Intuitively, normal data will be mapped to the high density area of the standard Gaussian distribution as normalizing flows are trained with normal data.

Original: In contrast, those abnormal data will be mapped to the tail of the distribution, making it difficult for the decoder network to effectively reconstruct the original image.
Revised: In contrast, those abnormal data will be mapped to the tail of the distribution, making it difficult for the decoder network to effectively reconstruct the original image.

Original: The loss function of the three-block pipeline takes into account of the dissimilarity between the reconstructed image and the original image in pixel-level and the likelihood of data distribution in feature-level.
Revised: The loss function of the three-block pipeline takes into account of the dissimilarity between the reconstructed image and the original image in pixel-level and the likelihood of data distribution in feature-level.

Original: The anomaly score is composed of the reconstruction error and the flow likelihood for the inference stage.
Revised: The anomaly score is composed of the reconstruction error and the flow likelihood for the inference stage.

Original: The pipeline is illustrated in Fig.
Revised: The pipeline is illustrated in Fig.

Original: 1, compared to the usual autoencoder neural network.
Revised: 1, compared to the usual autoencoder neural network.

Original: The detailed illustration of the overall network structure is shown in Fig.
Revised: The detailed illustration of the overall network structure is shown in Fig.

Original: In the following, we will explain the three blocks in detail.
Revised: In the following, we will explain the three blocks in detail.

Revised: Encoder.

Revised: The model starts with a pretrained encoder network for extracting features of the input image.

Revised: Each input image will be downsampled four times and the extracted feature is low-dimensional

Original: Firstly, the encoder block extracts lowdimensional features z of the input images x .
Revised: Firstly, the encoder block extracts lowdimensional features z of the input images x .

Original: Then, the normalizing flows transform the feature vector to a standard Gaussian distribution.
Revised: Then, the normalizing flows transform the feature vector to a standard Gaussian distribution.

Original: At last, the decoder reconstructs the normalized feature z ′ to the image x ′ .
Revised: At last, the decoder reconstructs the normalized feature z ′ to the image x ′ .

Original: The likelihood of the normalized feature is used as the flow loss and the residual between x and x ′ is used as the reconstruction loss.
Revised: The likelihood of the normalized feature is used as the flow loss and the residual between x and x ′ is used as the reconstruction loss.

Original: Encoder. The model starts with a pretrained encoder network for extracting features of the input image. Each input image will be downsampled four times and the extracted feature is low-dimensional and suitable for the flow model training.
Revised: and suitable for the flow model training.

Original: More specifically, the encoder f : X → Z transforms the input image x ∈ R 3 × H × W to a feature z ∈ R C × H 16 × W 16 , where H and W are the height and width of the original image and C is the number of channels.
Revised: More specifically, the encoder f : X → Z transforms the input image x ∈ R 3 × H × W to a feature z ∈ R C × H 16 × W 16 , where H and W are the height and width of the original image and C is the number of channels.

Original: Normalizing flow bottleneck.
Revised: Normalizing flow bottleneck.

Original: In order to explicitly obtain the probability density p Z ( z ) of the extracted feature vector z , we use a normalizing flow model Φ : Z → Z ′ to transform it to the feature space z ′ ∼ N ( z ′ ; 0 , I ) which follows a standard normal distribution.
Revised: In order to explicitly obtain the probability density p Z ( z ) of the extracted feature vector z , we use a normalizing flow model Φ : Z → Z ′ to transform it to the feature space z ′ ∼ N ( z ′ ; 0 , I ) which follows a standard normal distribution.

Original: This flow model is trained by maximizing log-likelihood for the normal image data, which implying a high probability for normal data, while a low probability for abnormal data in the inference stage.
Revised: This flow model is trained by maximizing log-likelihood for the normal image data, which implying a high probability for normal data, while a low probability for abnormal data in the inference stage.

Original: The test data can be divided into two categories by applying a threshold on the likelihood.
Revised: The test data can be divided into two categories by applying a threshold on the likelihood.

Original: In particular, the normalizing flow map is composed of bijective transformation blocks.
Revised: In particular, the normalizing flow map is composed of bijective transformation blocks.

Original: For each block, one coupling layer, Softplus activation and output vector permutations are used.
Revised: For each block, one coupling layer, Softplus activation and output vector permutations are used.

Original: The total flow model contains eight flow blocks.
Revised: The total flow model contains eight flow blocks.

Original: By denoting the input as x and the output as y at each flow block, it can be formulated as (Kingma & Dhariwal, 2018; Ardizzone et al., 2018-2022):
Revised: By denoting the input as x and the output as y at each flow block, it can be formulated as (Kingma & Dhariwal, 2018; Ardizzone et al., 2018-2022):

Original: [Equation]
Revised: [Equation]

Original: where R is a 1 × 1 invertible convolution that reverses the ordering of the channels, Ψ ( · ) is the
Revised: where R is a 1 × 1 invertible convolution that reverses the ordering of the channels, Ψ ( · ) is the

Original: Softplus activation function 1 β ∗ log(1 + exp( β ∗ · )) with β = 0 .
Revised: Softplus activation function 1 β ∗ log(1 + exp( β ∗ · )) with β = 0 .

Original: 5 , s global and t global are learning parameters corresponding to the scale and bias in the actnorm step, ⊙ is point-wise product.
Revised: 5 , s global and t global are learning parameters corresponding to the scale and bias in the actnorm step, ⊙ is point-wise product.

Original: For the coupling layer, we split the input x into two compositions ( x 1 , x 2 ) ⊤ by the channels, and denote the corresponding output as u = ( u 1 , u 2 ) ⊤ :
Revised: For the coupling layer, we split the input x into two compositions ( x 1 , x 2 ) ⊤ by the channels, and denote the corresponding output as u = ( u 1 , u 2 ) ⊤ :

Original: = Coupling ( x ) and:
Revised: = Coupling ( x ) and:

Original: [Equation]
Revised: [Equation]

Original: where s ( x 2 ) and t ( x 2 ) represents the scale and bias which are predicted by one NN, i.e. s ( x 2 ) , t ( x 2 )
Revised: where s ( x 2 ) and t ( x 2 ) represents the scale and bias which are predicted by one NN, i.e. s ( x 2 ) , t ( x 2 )

Original: =
Revised: =

Original: NN ( x 2 ) .
Revised: NN ( x 2 ) .

Original: Two subnet architectures are considered for this NN: one follows the network designed by Yu et al.
Revised: Two subnet architectures are considered for this NN: one follows the network designed by Yu et al.

Original: (2021) for which contains two convolutional layers.
Revised: (2021) for which contains two convolutional layers.

Original: The second one is based on ResNets (He et al., 2016) structure for a better image structure representation and easier training.
Revised: The second one is based on ResNets (He et al., 2016) structure for a better image structure representation and easier training.

Original: Details can be found in the later experiments section.
Revised: Details can be found in the later experiments section.

Original: Decoder.
Revised: Decoder.

Original: The decoder g : Z ′ → X ′ is used to reconstruct the image x ′ from normal distributed data by minimizing the distance between the input image x and reconstructed image x ′ .
Revised: The decoder g : Z ′ → X ′ is used to reconstruct the image x ′ from normal distributed data by minimizing the distance between the input image x and reconstructed image x ′ .

Original: In the decoder, we adopt four modules and each module contains two convolution layers.
Revised: In the decoder, we adopt four modules and each module contains two convolution layers.

Original: In each module, we use 3 × 3 kernel transposed convolution layer for upsampling operation and another 3 × 3 kernel convolution layer to adjust the number of channels and enhance the reconstruction ability.
Revised: In each module, we use 3 × 3 kernel transposed convolution layer for upsampling operation and another 3 × 3 kernel convolution layer to adjust the number of channels and enhance the reconstruction ability.

Original: Loss Function.
Revised: Loss Function.

Original: The loss function consists of two components, the flow loss and the reconstruction loss.
Revised: The loss function consists of two components, the flow loss and the reconstruction loss.

Original: The negative logarithm probability likelihood p Z ( z ′ ) is taken as the flow loss.
Revised: The negative logarithm probability likelihood p Z ( z ′ ) is taken as the flow loss.

Original: The reconstruction loss is defined as the mean square error (MSE) between the input image and the reconstructed image.
Revised: The reconstruction loss is defined as the mean square error (MSE) between the input image and the reconstructed image.

Original: We choose MSE in order to force the model to recover main image structures.
Revised: We choose MSE in order to force the model to recover main image structures.

Original: The weighted sum of two losses will represent the total loss.
Revised: The weighted sum of two losses will represent the total loss.

Original: The loss function with parameter α is formulated as:
Revised: The loss function with parameter α is formulated as:

Original: [Equation]
Revised: [Equation]

Original: where
Revised: where

Original: [Equation]
Revised: [Equation]

Original: We note that the reconstruction loss can be also replaced with other measures, such as Structural
Revised: We note that the reconstruction loss can be also replaced with other measures, such as Structural

Original: Similarity Index Measure (SSIM).
Revised: Similarity Index Measure (SSIM) (Wang et al., 2004).

Original: Anomaly Score.
Revised: Anomaly Score.

Original: Similar to the loss function, we use two components for the anomaly score in the inference step.
Revised: Similar to the loss function, we use two components for the anomaly score in the inference step.

Original: The negative probability density of the normalized feature is used as the flow score.
Revised: The negative probability density of the normalized feature is used as the flow score.

Original: For the reconstruction loss, we use the negative SSIM(Wang et al., 2004) between the input image and reconstructed image as its range is of [ − 1 , 1] and it mainly measure the structure difference between images.
Revised: For the reconstruction loss, we use the negative SSIM between the input image and reconstructed image as its range is of [ − 1 , 1] and it mainly measure the structure difference between images.

Original: The window size for calculating SSIM score is set to be 11.
Revised: The window size for calculating SSIM score is set to be 11.

Original: The total anomaly score is formulated as a weighted summation of the two scores by a parameter β
Revised: The total anomaly score is formulated as a weighted summation of the two scores by a parameter β

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: where
Revised: where

Original: [Equation]
Revised: [Equation]

Original:  E XPERIMENTS 3. D ATASETS
Revised:  E XPERIMENTS 3. D ATASETS

Original: The proposed method was performed on five datasets with image-level annotations: the OCT (Optical Coherence Tomography) dataset, the Chest X-ray dataset (Kermany et al., 2018), the skin image
Revised: The proposed method was performed on five datasets with image-level annotations: the OCT (Optical Coherence Tomography) dataset, the Chest X-ray dataset (Kermany et al., 2018), the skin image

Original: ISIC2018 ( International Skin Imaging Collaboration) dataset (Tschandl et al., 2018; Codella et al., 2019), the brain tumor BraTS

Revised: ISIC2018 ( International Skin Imaging Collaboration) dataset (Tschandl et al., 2018; Codella et al., 2019), the brain tumor BraTS

Original: (Brain Tumor Segmentation) dataset (Menze et al., 2014; Bakas et al.
Revised: (Brain Tumor Segmentation) dataset (Menze et al., 2014; Bakas et al., 2017;

Revised: Baid et al., 2021) and the microscopic images MIIC (Microscopic Images of Integrated

Revised: Circuits) dataset (Huang et al., 2021).

Original: OCT: The OCT dataset consists of 26 , 315 normal images for training.
Revised: OCT: The OCT dataset consists of 26 , 315 normal images for training.

Original: In the test phase, there are four categories: normal, drusen, CNV and DME.
Revised: In the test phase, there are four categories: normal, drusen, CNV and DME.

Original: The later three categories are regarded as abnormal data and each category has 242 images.
Revised: The later three categories are regarded as abnormal data and each category has 242 images.

Original: The image number of test data for the four categories is chosen to be the same to avoid the data imbalance problem.
Revised: The image number of test data for the four categories is chosen to be the same to avoid the data imbalance problem.

Original: Chest X-ray: The X-ray chest dataset contains 1 , 349 cases of normal imageswhich are used for training.
Revised: Chest X-ray: The X-ray chest dataset contains 1 , 349 cases of normal images for training.

Original: There are 234 normal images and 390 abnormal images which are diagnosed as pneumonia for testing.
Revised: There are 234 normal images and 390 abnormal images which are diagnosed as pneumonia for testing.

Original: ISIC2018:
Revised: ISIC2018:

Original: The ISIC2018 challenge dataset (task three) contains seven categories and we refer to
Revised: The ISIC2018 challenge dataset (task three) contains seven categories and we refer to

Original: Lu & Xu (2018) for classifying the NV (nevus) category as normal samples and the rest category as abnormal data.
Revised: Lu & Xu (2018) for classifying the NV (nevus) category as normal samples and the rest category as abnormal data.

Original: The training set includes 6705 healthy images, and the original validation set are used as the test data, which includes 123 normal images and 70 abnormal images.
Revised: The training set includes 6705 healthy images, and the original validation set are used as the test data, which includes 123 normal images and 70 abnormal images.

Original: BraTS2021:
Revised: BraTS2021:

Original: The Brain Tumor Segmentation (BraTS) Challenge 2021 dataset consists of 3D multimodal magnetic resonance imaging (MRI) scans.
Revised: The Brain Tumor Segmentation (BraTS) Challenge 2021 dataset consists of 3D multimodal magnetic resonance imaging (MRI) scans.

Original: We slice the central portion of each 3D data set to make up the 2D dataset for our model.
Revised: We slice the central portion of each 3D data set to make up the 2D dataset for our model.

Original: There are 11 , 739 healthy slices in our training set, and 1 , 051 slices with tumors and 459 slices without tumors are used as the test set.
Revised: There are 11 , 739 healthy slices in our training set, and 1 , 051 slices with tumors and 459 slices without tumors are used as the test set.

Original: MIIC:
Revised: MIIC:

Original: The MIIC dataset first appeared in (Huang et al., 2021) contains real microscopic images of integrated circuits (ICs), with 23,888 normal images for training, and 1 , 272 normal andabnormal images for testing.
Revised: The MIIC dataset first appeared in (Huang et al., 2021) contains real microscopic images of integrated circuits (ICs), with 23,888 normal images for training, and 1 , 272 normal andabnormal images for testing.

Original: We directly compare our methods with the proposed method in the original paper(Huang et al., 2021).
Revised: We directly compare our methods with the proposed method in the original paper(Huang et al., 2021).

Original: 3. I MPLEMENTATION D ETAILS
Revised: 3. I MPLEMENTATION D ETAILS

Original: We note that only normal data are used for the training and only the parameters in the flow model and the decoder are updated for the stability of training process.
Revised: We note that only normal data are used for the training and only the parameters in the flow model and the decoder are updated for the stability of training process.

Original: For the encoder, we chose ImageNetpretrained Wide-ResNet-50 (Zagoruyko & Komodakis, 2016) as the feature extractor as it shows to be effective and and provide sufficient reception field.
Revised: For the encoder, we chose ImageNetpretrained Wide ResNet-50-2 (Zagoruyko & Komodakis, 2016) as the feature extractor as it shows to be effective and and provide sufficient reception field.

Original: In the following, we provide the network details of the three blocks and the optimizer.
Revised: In the following, we provide the network details of the three blocks and the optimizer

Original: • Encoder:
Revised: • Encoder:

Original: The encoder contains four modules, and the feature map of the fourth block were chosen as the extracted features.
Revised: The encoder contains four modules, and the feature map of the fourth block were chosen as the extracted features.

Original: In more detail, each image is resized to 256 × 256 and the size of the extracted feature map is 16 × 16 with 1024 channels.
Revised: In more detail, each image is resized to 256 × 256 and the size of the extracted feature map is 16 × 16 with 1024 channels.

Original: • Flow:
Revised: • Flow:

Original: The flow architecture was constructed using FrEIA library (Ardizzone et al., 20182022).
Revised: The flow architecture was constructed using FrEIA library (Ardizzone et al., 20182022).

Original: Two types of subnets are considered for the coupling layer.
Revised: Two types of subnets are considered for the coupling layer.

Original: One is based on (Yu et al., 2021), for which each block contains two convolutional layers with ReLU activation function, and the corresponding kernel size is 3 × 3 and 1 × 1 respectively.
Revised: One is based on (Yu et al., 2021), for which each block contains two convolutional layers with ReLU activation function, and the corresponding kernel size is 3 × 3 and 1 × 1 respectively.

Original: The other is a ResNet-type network with two 3 × 3 convolution layers with batch normalization and ReLU function, and a shortcut connection with 1 × 1 convolution will be added as the output.
Revised: The other is a ResNet-type network with one 3 × 3 convolution layer with batch normalization and ReLU function, and a shortcut connection with 1 × 1 convolution will be added as the output.

Original: We find the ResNet-type network performs better in most situations, especially when it comes to large-scale datasets.
Revised: We find the ResNet-type network performs better in most situations, especially when it comes to large-scale datasets.

Original: The total flow steps is set to be
Revised: The total flow steps is set to be

Original: 8 . • Decoder: The decoder consists of four blocks.
Revised: 8 . • Decoder: The decoder consists of four blocks.

Original: We choose transposed convolution with 2 × 2 kernel and stride 2 for the upsampling operation, and the channel number is symmetric with the encoder (1024 → 512 → 256 → 64) .
Revised: We choose transposed convolution with 2 × 2 kernel and stride 2 for the upsampling operation, and the channel number is symmetric with the encoder (1024 → 512 → 256 → 64) .

Original: For the former three blocks, we employ another 3 × 3 convolutional layer with the same channel number to enhance the ability of reconstructing the image.
Revised: For the former three blocks, we employ another 3 × 3 convolutional layer with the same channel number to enhance the ability of reconstructing the image.

Original: The final transposed convolution layer with 3 channels will directly output the reconstructed image x ′ ∈ R 3 × H × W .
Revised: The final transposed convolution layer with 3 channels will directly output the reconstructed image x ′ ∈ R 3 × H × W .

Original: • Optimizer:
Revised: • Optimizer:

Original: The network parameters were acquired via minimizing the loss function using the Adam method.
Revised: The network parameters were acquired via minimizing the loss function using the Adam method(Kingma & Ba, 2014).

Original: The momentum parameter, the batch size and the learning rate  were set to be 0 .
Revised: The momentum, batch size, learning rate and weight decay hyperparameters were set to be 0 .

Original: 9 , 64 ,  and 10 − 3 respectively.
Revised: 9 , 64 , 2 · 10 − 4 and 10 − 5 respectively.

Revised: The last two hyperparameters were set to be 10 − 3 and 0 for the Chest X-ray dataset.

Original: The model was trained for 100 epochs.
Revised: The model was trained for 100 epochs.

Original: • Weight parameters: For the loss function, we find the model attains the best results whenthe weight parameters α = 0 .
Revised: • Weight parameters: For the loss function, we find the model attains the best results when weight parameters α = 0 .

Original: 5 and β = 0 .
Revised: 5 and β = 0 .

Original: 9 , which we will elaborate more in the later section.
Revised: 9 , which we will elaborate more in the later section.

Original: 3. R ESULTS
Revised: 3. R ESULTS

Original: The proposed methods were compared with five deep learning-based approaches: one AE-based method ( and GANomaly (Akcay et al., 2018): and two recently proposed NF-based methods: DifferNet (Rudolph et al.
Revised: The proposed methods were compared with five deep learning-based approaches: one AE-based method (Kramer, 1991), two representative GAN-based methods: f-AnoGAN (Schlegl et al., 2019) and GANomaly (Akcay et al., 2018): and two recently proposed NF-based methods: DifferNet (Rudolph et al.

Original:  2021)and Fastflow (Yu et al., 2021).
Revised: , 2021)and Fastflow (Yu et al., 2021).

Revised: We refer to Zhao et al.

Revised: (2021) for the quantitative metrics.

Revised: All results are reported based on the best performance on the test set.

Original: See Tab. 1 for the quantitative comparison of the different methods on the four medical datasets.
Revised: See Tab.for the quantitative comparison of the different methods on the four medical datasets.

Original: We adopted Area Under Curves (AUC), f1score (F1), accuracy (ACC), sensitivity (SEN) and specificity (SPE) as the evaluation metrics and the threshold of ACC, SEN and SPE is determined based on the best F1-score.
Revised: We adopted Area Under Curves (AUC), f1-score (F1), accuracy (ACC), sensitivity (SEN) and specificity (SPE) as the evaluation metrics and the threshold of ACC, SEN and SPE is determined based on the best F1-score.

Original: The threshold used for evaluation is determined based on the best value of F1-score.
Revised: The threshold used for evaluation is determined based on the best value of F1-score.

Original: The corresponding AUC curves for all models are also illustrated in Fig.
Revised: The corresponding AUC curves for all models are also illustrated in Fig.

Original: 3 for a better illustration.
Revised: 3 for a better illustration.

Original: For the OCT dataset, AE-FLOW achieves the best results among all methods in all metrics: AUC,
Revised: For the OCT dataset, AE-FLOW achieves the best results among all methods in all metrics: AUC,

Original: F1, ACC, SEN and SPE.
Revised: F1, ACC, SEN and SPE.

Original: GAN-based methods are superior to AE method and GANomaly outperforms f-AnoGAN.
Revised: GAN-based methods are superior to AE method and GANomaly outperforms f-AnoGAN.

Original: Normalizing-flow based methods, i.e. DifferNet and Fastflow, have shown to be superior while the proposed AE-FLOW method outperforms Fastflow by a noticeably margin with an additional reconstruction block.
Revised: Normalizing-flow based methods, i.e. DifferNet and Fastflow, have shown to be superior while the proposed AE-FLOW method outperforms Fastflow by a noticeably margin with an additional reconstruction block.

Original: Thus adding the reconstruction part in AE-FLOW is beneficial to promote the detection accuracy.
Revised: Thus adding the reconstruction part in AE-FLOW is beneficial to promote the detection accuracy.

Original: The Chest X-ray datasets contains significantly less data than the OCT dataset, thus the accuracy are less high than the OCT data for all the methods.
Revised: The Chest X-ray datasets contains significantly less data than the OCT dataset, thus the accuracy are less high than the OCT data for all the methods.

Original: We find the performance of GAN-based method can not exceed that of autoencoder on this dataset, which may be due to the insufficiency of training data.
Revised: We find the performance of GAN-based method can not exceed that of autoencoder on this dataset, which may be due to the insufficiency of training data.

Original: However, we find that even with a small dataset, normalizing flow-based techniques may still provide satisfactory results.
Revised: However, we find that even with a small dataset, normalizing flow-based techniques may still provide satisfactory results.

Original: By taking the advantage of normalizing flows and the reconstruction error measure, our model still achieves the best results in terms of AUC, F1, accuracy and specificity.
Revised: By taking the advantage of normalizing flows and the reconstruction error measure, our model still achieves the best results in terms of AUC, F1, accuracy and specificity.

Original: For the ISIC2018 dataset, the given normal category (nevus) can be easily mistaken with other categories, which makes anomaly detection challenging.
Revised: For the ISIC2018 dataset, the given normal category (nevus) can be easily mistaken with other categories, which makes anomaly detection challenging.

Original: For this case, our method still performs significantly superior to other methods, and the impact of α and β on the AUC (provided in supplementary material) also demonstrates that the flow score is a robust index.
Revised: For this case, our method still performs significantly superior to other methods, and the impact of α and β on the AUC (provided in supplementary material) also demonstrates that the flow score is a robust index.

Original: The experimental results of each method for the BraTS2021 dataset are comparatively similar, while our method is still competitive and produces marginally improved results than the others.
Revised: The experimental results of each method for the BraTS2021 dataset are comparatively similar, while our method is still competitive and produces marginally improved results than the others.

Original: For the non-medical dataset MIIC, we compare with the method in Huang et al.
Revised: For the non-medical dataset MIIC, we compare with the method in Huang et al.

Original: , and the results are shown in Tab 2.
Revised: , and the results are shown in Tab 2.

Original: Although the performance of the approach in the literature is impressive, our method is still quite competitive and demonstrates its effectiveness.
Revised: Although the performance of the approach in the literature is impressive, our method is still quite competitive and demonstrates its effectiveness

Original: [Table] [Table]
Revised: [Table] [Table]

Original: To further validate the effectiveness

Original: [Table]
Revised: [Table]

Original:  the distribution curves of the computed anomaly score for the two categories, i [Table] e. normal and anomaly are shown in Fig.
Revised: To further validate the effectiveness, the distribution curves of the computed anomaly score for the two categories, i.e. normal and anomaly are shown in Fig.

Original: The horizontal axis of the figure represents the anomaly score and the vertical axis represents the probability density.
Revised: The horizontal axis of the figure represents the anomaly score and the vertical axis represents the probability density.

Original: It is shown that the predicted anomaly score of anomaly data is commonly larger than that of normal data.
Revised: It is shown that the predicted anomaly score of anomaly data is commonly larger than that of normal data.

Original: Moreover, the overlapping area takes a small percentage of the total area [Table] and each curve hits its own unique peak, which demonstrates that the proposed anomaly score function is a valid and effective measure for the classification of normal and abnormal samples.
Revised: Moreover, the overlapping area takes a small percentage of the total area, and each curve hits its own unique peak, which demonstrates that the proposed anomaly score function is a valid and effective measure for the classification of normal and abnormal samples.

Original: Compared to flow based method, the proposed AE-flow model allows to provide some visual interpretability of the anomaly detection model.
Revised: Compared to flow based method, the proposed AE-flow model allows to provide some visual interpretability of the anomaly detection model.

Original: In Fig.
Revised: In Fig.

Original: 5 and Fig.
Revised: 5 and Fig.

Original: 6, we show the original, the reconstructed and the residual images for some normal and abnormal samples from the OCT and
Revised: 6, we show the original, the reconstructed and the residual images for some normal and abnormal samples from the OCT and

Original: Chest X-ray datasets respectively.
Revised: Chest X-ray datasets respectively.

Original: For the OCT dataset, the original images x , reconstructed images x ′ and residual images | x − x ′ | are shown in Fig.
Revised: For the OCT dataset, the original images x , reconstructed images x ′ and residual images | x − x ′ | are shown in Fig.

Original: It shows that the reconstruction image smooth out the zig-zag shape structure, which is identified as the lesion part in the original image.
Revised: It shows that the reconstruction image smooth out the zig-zag shape structure, which is identified as the lesion part in the original image.

Original: In contrast, the normal data do not contain these structures and the corresponding reconstructed image are much closer to the input images with negligible reconstruction error, and this difference helps to distinguish the anomaly and provide visual interpretability of the detection model.
Revised: In contrast, the normal data do not contain these structures and the corresponding reconstructed image are much closer to the input images with negligible reconstruction error, and this difference helps to distinguish the anomaly and provide visual interpretability of the detection model.

Original: For the chest
Revised: For the chest

Original: X-ray dataset, the difference between the original and the reconstructed image for anomaly data is more obvious.
Revised: X-ray dataset, the difference between the original and the reconstructed image for anomaly data is more obvious.

Original: Most of X-ray images of pneumonia contain transparent white areas while there are less such structures in normal data.
Revised: Most of X-ray images of pneumonia contain transparent white areas while there are less such structures in normal data.

Original: These structures are not present in the reconstructed images based on the AE-FLOW learned from normal data.
Revised: These structures are not present in the reconstructed images based on the AE-FLOW learned from normal data.

Original: Thus the differences between the original image and the reconstructed image are in a higher level than that of normal data.
Revised: Thus the differences between the original image and the reconstructed image are in a higher level than that of normal data.

Original: 3. A BLATION STUDY
Revised: 3. A BLATION STUDY

Original: As the proposed model is composed of the reconstruction error and the flow likelihood for both loss function and anomaly score, we investigated the performance of difference choices of the two weights parameters α and β in the loss (equation 1) and anomaly score (equation 2) respectively.
Revised: As the proposed model is composed of the reconstruction error and the flow likelihood for both loss function and anomaly score, we investigated the performance of difference choices of the two weights parameters α and β in the loss (equation 1) and anomaly score (equation 2) respectively.

Original: Fig 7 gives the AUC value of the proposed model with different α and β ranged in [0 , 1] .
Revised: Fig 7 gives the AUC value of the proposed model with different α and β ranged in [0 , 1] .

Original: When α = β = 1 . 0 , only the flow loss is used for the training and only the flow score is used for anomaly detection.
Revised: When α = β = 1 . 0 , only the flow loss is used for the training and only the flow score is used for anomaly detection.

Original: By letting 0 . 0 < α < 1 . 0 , the reconstruction error is introduced in the loss function and we can see that the AUC get improved ( 96 → 97 for the OCT dataset, 90 → 92 for the Chest
Revised: By letting 0 . 0 < α < 1 . 0 , the reconstruction error is introduced in the loss function and we can see that the AUC get improved ( 96 → 97 for the OCT dataset, 90 → 92 for the Chest

Original: X-ray dataset) even if only the flow score is used as the anomaly score ( β = 1 . 0 ).
Revised: X-ray dataset) even if only the flow score is used as the anomaly score ( β = 1 . 0 ).

Original: This shows that the reconstruction loss ( L recon ) positively affects the flow model.
Revised: This shows that the reconstruction loss ( L recon ) positively affects the flow model.

Original: Specifically, for both datasets, α = 0 . 5 , β = 0 . 9 , the best results can be achieved.
Revised: Specifically, for both datasets, α = 0 . 5 , β = 0 . 9 , the best results can be achieved.

Original: Furthermore, it can be seen that the proposed model stably attains high AUC for a relatively large range of parameters α and β , which implies the robustness of the proposed model.
Revised: Furthermore, it can be seen that the proposed model stably attains high AUC for a relatively large range of parameters α and β , which implies the robustness of the proposed model.

Original: In Tab. 3, we show the AUC score under different settings for the subnet of the NF model and the reconstruction loss function using MSE or SSIM.
Revised: In Tab. 3, we show the AUC score under different settings for the subnet of the NF model and the reconstruction loss function using MSE or SSIM.

Original: The results show that the model with these four different combinations still lead to stably high perforamnce, compared to other methods.
Revised: The results show that the model with these four different combinations still lead to stably high perforamnce, compared to other methods.

Original: ResNets with
Revised: ResNets with

Original: MSE loss achieves the best AUC for the OCT (98.15), ISIC2018 (87.79) and BraTS2021 (81.58) dataset, while 2-Layer CNN with SSIM achievest the best for Chest X-ray (92.00) dataset, which may due to the facts that the anomaly in OCT datasets are mainly due to local lesions while the anomaly in Chest X-ray dataset are more spread out.
Revised: MSE loss achieves the best AUC for the OCT (98.15), ISIC2018 (87.79) and BraTS2021 (81.58) dataset, while 2-Layer CNN with SSIM achievest the best for Chest X-ray (92.00) dataset, which may due to the facts that the anomaly in OCT datasets are mainly due to local lesions while the anomaly in Chest X-ray dataset are more spread out.

Original: [Table]
Revised: [Table]

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this paper, we proposed AE-FLOW, a self-supervised model that integrates the tractable likelihood of the NF model and the image dissimilarity between the input and the reconstructed images for the representation of normal data and detecting medical anomalies.
Revised: In this paper, we proposed AE-FLOW, a self-supervised model that integrates the tractable likelihood of the NF model and the image dissimilarity between the input and the reconstructed images for the representation of normal data and detecting medical anomalies.

Original: The proposed anomaly score function takes into account both the image reconstruction error and the likelihood in feature space for classifying normal and abnormal data, by learning a representation model only from normal data.
Revised: The proposed anomaly score function takes into account both the image reconstruction error and the likelihood in feature space for classifying normal and abnormal data, by learning a representation model only from normal data.

Original: The method provides not only image-level tractability of normal data but also pixel-level interpretability of anomaly data.
Revised: The method provides not only image-level tractability of normal data but also pixel-level interpretability of anomaly data.

Original: Experiments on different medical image datasets demonstrated the effectiveness and robustness of the proposed approach, with a large margin of improvement over the other related and representative approaches for anomaly detection.
Revised: Experiments on different medical image datasets demonstrated the effectiveness and robustness of the proposed approach, with a large margin of improvement over the other related and representative approaches for anomaly detection.

Revised: A CKNOWLEDGMENTS

Revised: This work was supported by Shanghai Municipal Science and Technology Major Project (2021SHZDZX0102) and NSFC (No. 12090024 and 12201402).

Revised: We thank the Student Innovation Center at Shanghai Jiao Tong University for providing us the computing services

Original: A C ROSS V ALIDATION
Revised: A C ROSS V ALIDATION

Original: We performed five-fold cross-validation experiments for all four medical datasets to further evaluate the robustness and effectiveness of our proposed method.
Revised: We performed five-fold cross-validation experiments for all four medical datasets to further evaluate the robustness and effectiveness of our proposed method.

Original: We randomly split the initial test set into validation and test sets in the ratio of 2:8.
Revised: We randomly split the initial test set into validation and test sets in the ratio of 2:8.

Original: In order to maintain the balance of each category, we keep the original percentage of normal and abnormal data in each set.
Revised: In order to maintain the balance of each category, we keep the original percentage of normal and abnormal data in each set.

Original: The training settings are the same as Section 3.2.
Revised: The training settings are the same as Section 3.2.

Original: We list each metric’s means and standard deviations for the five rotations in the Tab.
Revised: We list each metric’s means and standard deviations for the five rotations in the Tab.

Original: [Table]
Revised: [Table]

Original: As shown in the table
Revised: As shown in the table

Original: [Table]
Revised: [Table]

Original: our approach continues to perform well across all datasets.
Revised: our approach continues to perform well across all datasets.

Original: Due to the size of the dataset
Revised: Due to the size of the dataset

Original: [Table]
Revised: [Table]

Original: the Chest X-ray and ISIC2018 datasets have more fluctuating outcomes than the larger
Revised: the Chest X-ray and ISIC2018 datasets have more fluctuating outcomes than the larger

Original: OCT and BraTS2021 datasets.
Revised: OCT and BraTS2021 datasets.

Original: B M ORE RESULTS FOR THE RATIOS
Revised: B M ORE RESULTS FOR THE RATIOS

Original: The variations of AUC scores with different loss ratios α and score ratios β for the other two medical datasets are displayed in the Fig. 8.
Revised: The variations of AUC scores with different loss ratios α and score ratios β for the other two medical datasets are displayed in the Fig. 8.

Original: Similar to the pattern in Fig.
Revised: Similar to the pattern in Fig.

Original: 7, we see that the flow part’s accuracy is higher than the reconstruction part’s
Revised: 7, we see that the flow part’s accuracy is higher than the reconstruction part’s

Original: [Table]
Revised: [Table]

Original: but after adding the reconstruction part for training ( α < 1 ), the performance of the likelihood-only anomaly score rises ( 87 → 88 for ISIC2018 and 80 → 82 for BraTS2021).
Revised: but after adding the reconstruction part for training ( α < 1 ), the performance of the likelihood-only anomaly score rises ( 87 → 88 for ISIC2018 and 80 → 82 for BraTS2021).

Original: Specifically, we can still achieve best results for α = 0 .
Revised: Specifically, we can still achieve best results for α = 0 .

Original: 5 and β = 0 .
Revised: 5 and β = 0 .

Original: 9 , and we report our experiments under this setting.
Revised: 9 , and we report our experiments under this setting.

Original: C V ISUALIZATION FOR MORE DATASETS
Revised: C V ISUALIZATION FOR MORE DATASETS