You May Need both Good-GAN and Bad-GAN for Anomaly Detection
Authors: Riqiang Gao
Zhoubing Xu
Guillaume Chabin
Awais Mansoor
Florin-Cristian Ghesu
Bogdan Georgescu
Bennett A. Landman
Sasa Grbic
Venue: ICLR 2022 Submitted (2022)
Keywords: Anomaly Detection
GAN
Orthogonal Regularization
Bad-GAN

Abstract: Generative adversarial nets (GAN) have been successfully adapted for anomaly detection, where end-to-end anomaly scoring by so-called Bad-GAN has shown promising results. A Bad-GAN generates pseudo anomalies at the low-density area of inlier distribution, and thus the inlier/outlier distinction can be approximated. However, the generated pseudo anomalies from existing Bad-GAN approaches may (1) converge to certain patterns with limited diversity, and (2) differ from the real anomalies, making the anomaly detection hard to generalize. In this work, we propose a new model called Taichi-GAN to address the aforementioned issues of a conventional Bad-GAN. First, a new orthogonal loss is proposed to regularize the cosine distance of decentralized generated samples in a Bad-GAN. Second, we utilize few anomaly samples (when available) with a conventional GAN, i.e., so-called Good-GAN, to draw the generated pseudo anomalies closer to the real anomalies. Our Taichi-GAN incorporates Good-GAN and Bad-GAN in an adversarial manner; which generates pseudo anomalies that contributing to a more robust discriminator for anomaly scoring, and thus anomaly detection. Substantial improvements can be observed from our proposed model on multiple simulated and real-life anatomy detection tasks.

Under review as a conference paper at ICLR 2022
YOU MAY NEED BOTH GOOD-GAN AND BAD-GAN FOR ANOMALY DETECTION
Anonymous authors Paper under double-blind review
ABSTRACT
Generative adversarial nets (GAN) have been successfully adapted for anomaly detection, where end-to-end anomaly scoring by so-called Bad-GAN has shown promising results. A Bad-GAN generates pseudo anomalies the low-density area of inlier distribution, and thus the inlier/outlier distinction can be approximated. However, the generated pseudo anomalies from existing Bad-GAN approaches may (1) converge to certain patterns with limited diversity, and (2) differ from the real anomalies, making the anomaly detection hard to generalize. In this work, propose new model called Taichi-GAN to address the aforementioned issues of a conventional Bad-GAN. First, a new orthogonal loss is proposed to regularize the cosine distance of decentralized generated samples in Bad-GAN. Second, we utilize few anomaly samples (when available) with a conventional GAN, i.e., so-called Good-GAN, to draw the generated pseudo anomalies closer the real anomalies. Our Taichi GAN incorporates Good-GAt and Bad-GAN in an adversarial manner, which generates pseudo anomalies that contributing to a more robust discriminator for anomaly scoring, and thus anomaly detection. Substantial improvements can be observed from our proposed model on multiple simulated and real-life anatomy detection tasks.
INTRODUCTION
Anomaly detection (AD), a.k.a. outlier detection, refers to detecting uncommon samples (usually heterogencous) out of inlier distribution (Chandola Varun et al., 2009; Pang al., 2021a). AD can be performed either in an unsupervised manner when only inliers are involved during the training process (Schlegl et al., 2019; Ngo et al., 2019; Ruff al., 2018), or in a semi-supervised manner when few labeled anomalies are available (Ruff et al., 2020b; Liznerski et al., 2021).
Generative adversarial nets (GAN) (Goodfellow al., 2014) are considered the most effective generative models nowadays, which have also been applied for AD (Schlegl et al., 2017: 2019; Zenati al., 201 2018b; Siddiquee al., 2019; Perera et al., 2019). Likewise, one-class classification and its variants are often used the backbone of AD approaches (Schölkopf et al., 2001; Ruff al., 2018; Liznerski et al., 2021; Wu et al., 2019). Recently, the adversarial training and one-class classification have been leveraged together to discriminate inlier/outlier in end-to-end manner (Zheng et al., 2019; Ngo et al., 2019). Such efforts share a similar spirit with the work introduced Dai et al. (2017) and here call them Bad-GANG Instead of generating samples match the training data distribution as a conventional GAN (Goodfellow et al., 2014) that we refer as Good-GAN in this paper, a Bad-GAN pushes its generated samples towards the peripheral area of the training data distribution and considers them as pseudo anomalies. Rather than defining anomaly scores based on 1) the residual between reconstructed and original samples such as AnoGAN series (Schlegl et al., 2017; 2019), or 2) existing one-class classification-ban based measures such as Deep SVDD (Ruff et al., 2018) and FCDD (Liznerski et al., 2021), Bad-GAN directly leverage the learned discriminator to identify unseen anomalies from normal cases.
Although Bad-SANs have shown great potentials, they suffer from several disadvantages. Disl: the generated samples can converge to limited patterns rather than being heterogeneous real anomalies;
'Tacchi is concept from traditional Chinese culture, which refers to the co-existence and well- ballncalance of two components with different properties. We borrow this concept to indicate the relationship between "Bad-GAN" and "Good-GAN" in model.

---

Under review as a conference paper at ICLR 2022
Dis2: it is hard to ensure generated samples to resemble real anomalies, while abundance of irrelevant pseudo anomalies may negatively affect the discrimination of real anomalies; Dis3: no application has been found to integrate labeled anomalies in prior-art Bad-GANs
In this work, we propose a new Taichi-GAN to address aforementioned disadvantages. First, based on state-of-the-art Bad-GAN (FenceGAN (Ngo et al., 2019)) framework, orthogonal loss is proposed to increase the angular diversity, which addresses DisI, and partially relieves Dis2 by increasing the overlap between generated pseudo anomalies and real anomalies. Second, we are the first to incorporate Good-GAN within Bad-GAN framework, where the Good-GAN contributes to generating pseudo anomalies guided by few real anomalies, which addresses Dis2 and Dis3. Our model is termed Taichi-GAN, as the Good-GAN and Bad-GAN are integrated in a competing yet balanced manner to generate pseudo anomalies towards better AD. We illustrate our ideas with 2D synthetic data and validate the proposed model with five datasets including clinical applications.
2 RELATED WORKS
Shallow AD models have their limitations in dealing with high-dimensional data and large-scale datasets (Ruff et al., 2020b). Here we focus on the most related deep AD approaches.
iiconstruction-eerrr-aasss (REB) Model. REB approaches have been widely applied to image- based AD, where auto-encoder (Sato et al., 2018; Baur et al., 2018; Chen & Konukoglu, 2018) or GAN (Schlegl et al., 2017; 2019; Zenati et al., 2018b Siddiquee et al., 2019; Akcay et al., 2018; Zenati et al., 2018a) is typically used for reconstruction. REB methods assume that anomalies would not be well reconstructed when the model is only trained with normal samples. Therefore, the reconstruction errors can be used to evaluate the abnormality. AnoGAN (Schlegl et al., 2017) is representative REB approach which learns a manifold of anatomical variability and detects the anomalies with reconstruction errors mainly on pixel level. F-AnoGAN (Schlegl et al., 2019) follow the same framework as AnoGAN with improved processing speed. Han et al. (2021) ensemble multiple GANs for AD. However, REB approaches are mainly designed for data synthesis rather than AD, which may limit their comprehension for heterogeneous anomalies.
One-class Classification based Measure (OCCBM). OCCBM approaches (Ruff et al., 2018; Wu et al., 2019; Ruff et al., 2020b; Chalapathy et al., 2018) learn one-class embedding from normal samples to measure the derivation of an unseen case from the inlier distribution, where the measure- ment is then served for AD purposes. Deep OCCBM models extend the classical one-class models, e.g., one-class SVM (Schölkopf et al., 2001) and SVDD (Tax & Duin, 2004), by extracting features via deep nets (Ruff et al., 2018; Liznerski et al., 2021). OCCBM methods have been deployed to sophisticated applications, e.g., detecting anomalous event in practical scenes (Wu et al., 2019).
Bad-GANs. The Bad-GAN idea (Dai et al., 2017) is introduced for semi-supervised learning in multi-class classification. Similar approaches are found in the scope of the AD such Sabokrou et al. (2018); Lim et al. (2018); Zheng et al. (2019); Ngo et al. (2019). Such techniques are termed as Bad-GANs in this paper. The key idea of Bad-GANe is learn classifier (i.e., the discriminator) based on inliers and generated pseudo outliers during training so that it can distinguish unseen outliers and inliers during inference. Bad-GANs belong to the one-class classification family and can be trained end-to-end. In addition, it does not require manually defined one-class classification measure as OCCBM models (Pang et al., 2021a); the Bad-GAN discriminator handles the measure.
AD Models Utilizing Real Anomalies. Deep SAD (Ruff et al., 2020b) and ESAD (Huang et al., 2020) extend the Deep SVDD (Ruff et al., 2018) to take advantage of labeled anomalies. Siddiquee et al. (2019) propose REB model for AD by considering normal and anomalies as two "styles" via -nage-to-image translation techniques. Few-shot learning and reinforcement learning are adapted to utilize limited label information, respectively (Tian et al., 2020; Pang et al., 2021b).
3 METHOD
3.1 BACKGROUND: GOOD-GAN AND BAD-GAN
GANs (Goodfellow et al., 2014) train generative models using a minimax game, which generate data instances from noise distribution p= match the given data distribution Paata by iteratively
2

---

Under review as a conference paper at ICLR 2022
optimizing generator G and discriminator D with the following objective: min max ( G) = Prpppar log D(x) + Ez-p, [log(1 D(G())))
(1)
G
Dai et al. (2017) introduce the concept of the Bad-GAN that generates samples the periphery of the training data. The generator is optimized by minimizing Ey-pp log Pdata (I)I[paaaa (æ) > cl, where [[x]]] is indicator function. With threshold (0, 1), the training process penalizes the high-density samples and leaves the low-density ones unaffected, where the data distribution Patta of Dai et al. (2017) estimated with additional model PixeICNN++ (Salimans et al., 2017).
The more recent FenceGAN (Ngo et al., 2019) shares the similar spirit of Dai et al. (2017), which generates samples the boundary of normal distribution. FenceGAN gets rid of the additional intensity estimation model as in Dai et al. (2017), and thus formulates an end-to-end training process. The generator loss the batch with size N is defined
B-N LG E (() 1))
+ (CE()(G())).
where BCE(A, is the binary cross-entropy between A and B. The first term is the dispersion loss which regularizes generated samples to be distant from their center μ. The hyperparameter a is discriminative anomaly score that pushes generated pseudo anomalies on inlier distribution boundary and / is the so-called dispersion hyperparameter. The discriminator loss is defined as:
LD = (CE(((()), 0) + >IBCE(D(G(2))
where y E (0, 1] is the so-called anomaly hyperparameter. The discriminator is used to compute an anomaly score ranged (0,1) for each unseen case during the inference stage.
3.2 ORTHOGONAL Loss
As anything not normal is defined as anomaly and anomalies are heterogeneous (Ruff et al., 2020b), we hypothesize that generated pseudo anomalics a Bad-GAN framework should have high diversity to cover many abnormal patterns as possible. Dispersion loss (Ngo et al., 2019) (La in Fig. I) regularizes generated pseudo anomalies lying at boundary of inlier distribution. Here, we propose to push pseudo anomalies further distributed to evenly cover the entire angular space.
Lorth
Ld
Lorth
G(z:)
d
oLd oLd dd, ddg
La
G((z)
d1
02
G(Zy)
G(Zz) 01
OLorth a0,
@Lorth a02
Figure 1: Motivation of our orthogonal loss. Two situations illustrated, where G(2+) and G(z,) are a pair of generated pseudo anomalies. Indicated by right formulas, 1) the euclidean based loss Taa e.g., dispersion loss (Ngo al., 2019), stay the same for G(z,) given dj = d2; and 2) the orthogonal loss Torth penalizes more on G(z:) given a smaller angle between (G(=;,, G(zz))-
As verified in empirical experiments, the generated samples from Bad-GAN can be prone to specific patterns, even with the Euclidean dispersion regularization of FenceGAN (Ngo et al., 2019). We introduce an orthogonal loss Toorth to expand generated samples from the angular perspective. As depicted in Fig. paired samples with smaller angles (left in Fig. 1) to each other can have
Equations adapted from the official Github FenceGAN, which have different form with its paper.
3

---

Under review as a conference paper at ICLR 2022
larger penalty given the orthogonal regularization. The orthogonal regularization has been used for weight initialization and optimized on deep features (Brock et al., 2017; Arjovsky et al., 2015; Zhao et al., 2016). Here, define our Lorth in decentralized way and optimized generated pseudo anomalies an AD framework:
N
(G(zz) ()) 11) G[(z) ul1 [[((z,) - Mlll
Lorth N(N 1)
(4)
ffs
Instead of optimizing the commonly used Euclidean distance and individual regularization (e.g., the dispersion term of Ngo et al. (2019) and the losses OCCBM models (Ruff et al., 2018; 2020a;b)), proposed orthogonal loss leverages Cosine distance and relational regularization" to be complementary to the dispersion term in the Bad-GAN framework.
3.3 THE INTUITION OF ADDING GOOD-GAN TO BAD-GAN
As Good-GANs optimize generated samples to match the training distribution, given few real anomalies available, take this Good-GAN nature guide pseudo anomaly generation in AD tasks. Simultaneously, to avoid generated samples overfitting to limited anomaly patterns, we balance the Good-GAN and the Bad-GAN (including Torrth) with hyperparameter d. Specifically, the Good-GAN guides the generated pseudo anomalies to resemble the reference distribution from given real anomalies, while the Bad-GAN regularizes pseudo anomalies the boundary of inlier distribution. The GAN and the Bad-GAN share the same generator, and have their own discriminators separately. The overall objective is defined as follows:
min max max (oa) + G
Lorth
8V
(5)
Dod
Det
Bad-GAN
angular diversity
Good GAN
where Vaa and Vgd the objective functions of the Bad-GAN and Good-GAN, respectively. G is the shared generator. Dad and Dgd are discriminators of Bad-GAN and Good-GAN, respectively. The generated samples serve as pseudo anomalies train classifier (i.e., Dba) to discriminate unseen real anomalies from inliers during inference. Lorth is the proposed orthogonal loss to further increase the diversities pseudo anomalies. The overall model (Eq. termed Taichi-GAN.
(a)
(b)
(d)
1.0
0.7
0.3
0.0
(((20, 20),3) Rn [((0,0), 8), Va = G(za). (a) Bad-GAN, (b) Bad-GAN + pull-away term ((Zhao et al., 2016). orthogonal regularization without decentralizing samples). (e) Bad Bad-GAN Lorth (d) Taichi-GAi (Bad-GAN + Good-GAN Lorth). The green, red, and blue dots represent real inliers, reall outliers, and generated pseudo outliers, respectively. We
Figure 2: Experiments conducted on 2D synthetic data where
0.9 in Bad-GAN (here real anomalies score 1) following the setting FenceGAN for MNIST. To illustrate the effect of a Good-GAN, we include three anomaly samples {(10, 10), (12, 8), (8. 12)} illustrate the efficacy of angular diversity and Good-GAN in (d). The bar on the right encodes the anomaly score in color for reference.
As shown in Fig. 2(a), the Bad-G.B Bad-GAN generates samples along the boundary of the inlier distribution, however, with limited diversity, even including a dispersion loss on Euclidean distance space. Adding
Concepts of individual regularization and relational regularization have heen introduced in Liu al. (2021).
4

---

Under review as a conference paper at ICLR 2022
pull-away term (Zhao al., 2016) (an orthogonal regularization without decentralization to the sample average) is also not effective in this AD framework as in Fig. 2(b). The generated samples are more evenly distributed with the proposed orthogonal loss (Fig. 2(c)), which are more likely cover samples that resemble real anomalies even though no real anomaly is included for training. Further, when few labeled anomalies available, the Good-GAN inclusion pushes the generated samples have higher density near the given anomalies, meanwhile, Lorsh maintains high diversity to cover other potential anomaly patterns (Fig. 2(d)).
3.4
THE PROPOSED TAICHI-GAN
normal
generated
anomaly
Training:
{xq, xa)
D
(1)
borsht Lpaan
LeaaD
Random Signal
6
(2)
goodd
soodd
Dga
(3)
forward
backward
X normal
i: generated
Xn anomaly
Testing:
(4)
(x)
Dbd
Pa
(a)
(b)
Figure 3: (a) Schematic visualization of the proposed method. The upper section illustrates the scenario without anomalies, which targets generating diverse samples. The bottom section includes few labeled anomalies (red dots), which targets generating samples under the guidance of few labeled anomalies while keeping high diversity. (b) Training (upper) and testing (bottom) framework. Under the hood of the entire Taichi-GAN framework, the Bad-GAN components are marked in black, while the additional Good-GAN components are in red. The discriminator of the Bad-GAN (i.e., Doa) is used during the inference phase score the anomalies.
Fig. 3(a) shows our motivation with inlier and (pseudo) outlier example from HAM10000 (Tschand! al., 2018). The upper depicts the training with only normal samples, where we target generating highly diverse samples (e.g.. (I) vs. (2) in Fig. 3(a)) at the boundary of normal distribution. The bottom shows the motivation on sample generation after adding few anomalies. Besides keeping the diversity, some generated samples are close known anomalics (c.g., (3) is close to (4)).
We borrow Bad-GAN components from FenceGAN, The generator loss is defined as follows:
B.N EE G[)(()) x)ll BCE() (BCC)) gd (G(z)), 0)
LG = Lorth +
(6)
Lgooa
Laado
The generator losses of Bad-GAN (LaadG) and Good-GAN goodG). is consistent with the form of Eq. 2 and shares the same spirit of Goodfellow al. (2014), respectively. G denotes the generator, Dea and Dgd are the discriminators of Bad-GAN and Good-GAN, respectively. Lorth is the proposed orthogonal loss increase the diversity from the angular perspective.
The loss of Good-GAt GANN discriminator defined as: (Goodfellow et al., 2014):
LgoodDD BCC ( (r. 0) (CE((),.((())))
5

---

Under review as a conference paper at ICLR 2022
MNIST
FNSHI
CIFARO
HAM10000
2
8
Inlier
2
Bad-GAN
Bad-GAN + Larch
Bad-GAN' Lorth
3
9
Good-GAN
Figure 4: Generated pscudo anomalies among multiple datasets. The first row shows inlier examples. The second row shows samples generated only with Bad-GAN, The third row illustrates samples from Bad-GAN + Lorth and the fourth row from the model of Bad-GAN + Good-GAN Lorth i.e., Taichi-GAN. Red boxes in the second row highlight examples clustering to specific patterns. Green boxes in the fourth row highlight cases that are close to the same anomaly class while keeping some extent beterogeneity, even though only one sample per anomaly class is provided.
The loss of Bad-GAN discriminator is adapted from Ngo et al. (2019) by adding third term:
Lbadn = (C,( 0) +BBCE(D,.G((e)) 1) + ()CC((Dn(E.a)..
(8)
where Tn and Ta are normal (inlier) and anomaly (outlier) cases, respectively. The third term includes real anomalies to train the discriminator of Bad-GAN, = when no anomalies available. Note Vha and Vya (from Eq. 5) are composed of LLadd, Loean] (from Eq. and Eq. 8) and LggoodG } (from Eq, and Eq. 7), respectively.
The procedures of Taichi-GAs are illustrated in Fig. 3(b). The unsupervised training setting (no real anomalies) is performed with only the black text components (Bad-GAN + Lorth). The red text components are added when few anomalies are available.
4 EXPERIMENTS
In addition to 2D synthetic data in Fig. 2, we involve five datasets to evaluate our method for AD. MNIST, Fashion-MNIST and CIFAR10 popular in previous literature (Ruff al., 2020b; Ngo et al., 2019; Lee et al., 2018) 8). HAM1000 and BraTS20 are medical datasets where the anomalies are well defined clinically. The Area Under the ROC Curve (AUC) (Fawcett, 2006) is used evaluate the AD performance. The reported AUCs are average values over three independent runs of each experiment using different random seeds. Detailed network architectures, dataset splits, dataset sizes, model hyperparameters, and baseline implementations can be found in the Appendix A, B, and E.
4.1
DATA AND SETUP
MNIST (LeCun et al., 1998) contains 10 classes of digits (0-9). As one- me-rettrett setting (i.e., one class inlier and the rest classes outlier) in MNIST is proved be easy (Ruff al., 2018; 2020b),

---

Under review as a conference paper at ICLR 2022
consider the N-vs-rest setup. Class 0-2 are formalized as normal set and class 3-9 are formalized as anomaly set in our main setting (Table 1). Ablation studies considering different "N" and different amount of labeled anomalies are shown in Table 2 and Table 3, respectively.
Fashion-MNIST (Xiao et al., 2017) & CIFARIO (Krizhevsky, 2009) use one-vs-rest setting that we select class "T-shirt" and "Airplane" as the normal class and the rest as anomalies, respectively.
HAM10000 (Tschandl et al., 2018) is a clinical-based dataset consisting of 100000 dermatoscopic images with skin lesions of seven classes. The dominated class Melanocytic nevi (>70%) is common benign skin lesion which is set as inlier and the rest as outliers.
BraTS20 (UPenn, 2020) is recent version of BraTS challenge (Bakas, 2018). Three types of anomalies are included: the GD-enhancing tumor, the peritumoral edema, and the necrotic and non-enhaneing tumor core. Here we simplify the problem to image-level disease detection motivated by (Siddiquee et al., 2019) in the AD setting. We extract the 2D slices from T2-FLAIR volumes, where slices with anomaly annotations are considered as outliers and the rest as inliers. Special handling and generated pseudo anomaly examples on BraTS20 can be found in Appendix D.
Labeled Anomalies for Training. We categorize our experiments on each of the five data collections into two folds, i.e., "No anomalies available" and "Few anomalies available" (see Table 1). For the latter fold, we include the following numbers of anomalies: MNIST, one sample per class in 3-9 classes; Fashion-MNIST & CIFARIO, one sample per class in classes except "T--shrt""; 10000, five samples per class except "Melanocytic nevi"; BraTS20, 300 slices with tumor from scans.
Methods for Comparison. When no anomalies are available, the benchmark methods include OC-SVM (Schölkopf et al., 2001) (shallow AD model), Deep SSVD (Ruff al., 2018), F-AnoGAN (Schlegl et al., 2019) (reconstruction-eres arror-based model), original badGAN (Dai et al., 2017). and FCDD (Liznerski et al., 2021). When few anomalics are available, representative methods using outlier exposure or semi-super learning are included for comparison. FCDD+OE (Liznerski et al., 2021) integrates outlier exposure (OE) (Hendrycks et al., 2019) technique to utilize available real anomalies. Deep SAD (Ruff et al., 2020b) generalizes the Deep SVDD to semi-supervised setting. Two other naive approaches are included for benchmarking: (1) training a classification model using BCE regardless of limited anomalies, and (2) involving the available anomalies in the Bad-GAN discriminator training. i.e., Bad-GAN (semi).
Variants of Taichi-GAN. We perform experiments on the following settings: 1) Bad-GAN only uses FenceGAN, 2) Bad- Lod-GANNLortA includes orthogonal loss regularization, 3) Red-GAA(semi) trains the discriminator with few labeled anomalies besides the normal samples and generated samples, 4) Bad- Badd-GANssemi++Cn Good-GAr involves the Good-GAN training for sample generation on top of setting 3), 5) Taichi-GAN is our ultimate approach, equivalent to Badd-GANisemmii-GGooGGGAAA Lorth- Different settings serve an approach-level ablation study.
4.2 BENCHMARK RESULTS
The AD performances five datasets are reported in Table 1 across all benchmark methods and approach-level ablation settings described in Section 4. The proposed methods achieve the highest AUCs in both folds where No and Few anomalies available. Across all five datasets, we observe con- sistently that 1) the improvement from introducing orthogonal loss to Bad-GAN, 2) the improvement from leveraging few anomaly samples with a GAN, and 3) Taichi-GAN including Good-GAN, Bad-GAN, and orthogonal loss achieves the best performances.
Generated pseudo anomalies are shown in Fig. 4 and appendix Fig. 6 (bottom left). With our orthogonal regularization, the generated samples are more diversified (3rd row vs. 2nd row). When few anomalies are available, by incorporating the Good-GAN, the generated samples look closer to real anomalies and maintain reasonable diversity (indicated by green boxes in the fourth row).
Benchmark Analyses. When no anomalies are available, increasing the diversity of generated samples with the proposed Lorth effectively address Disl (e.g., +5.6%% AUC in MNIST and visual contrast of 3rd row vs. 2nd row in Fig. 4). Our proposed model achieves even better performance by guide the pseudo anomaly generation in the Bad-GAN framework. and
integrating the Good-GAN demonstrates the best effectiveness of consuming the limited labeled anomalies. By doing so, Dis2 and Dis3 are effectively addressed.
7

---

Under review as a conference paper at ICLR 2022
Table 1: AUCs (%) of benchmark comparison. "Bad-GAN" and "original BadGAN" refer to FenceGAN (Ngo et al., 2019) and the approach of Dai et al. (2017), respectively. Items with are our settings and "Taichi-GAN" our ultimate model. The highest AUCs are bold.
Method
MNIST FMNIST
CIFARTO HAM10000
Bra BraTS20
No anomalies available
OC-SVM (Schölkopf al., 2001)
75.98
86.53
61.60
57.11
85.01
Deep SSVD (Ruff et al., 2018)
86.13
89.28
68.31
78.42
83.08
F-AnoGAN (Schlegl et al., 2019)
86.49
87.65
71.10
80.60
83.52
orignal BadGAN (Dai et al., 2017)
83.70
88.13
69.81
80.29
88.84
FCDD (Liznerski et al., 2021)
77.60
73.72
68.10
74.85
79.30
Bad-GAN (Ngo et al., 2019)
86.52
87.85
68.01
79.77
88.64
Bad-GAN + Lorth
92.10
90.65
72.68
80.95
92.87
Few anomalies
available (anomaly amount details in Section 4.1)
FCDD+OE (Liznerski et al., 2
"9.II3"
9I.58
76.23
79.42
90.92
Deep SAD (Ruff et al., 2020b)
96.95
93.11
73.61
81.51
84.13
BCE
92.82
93.50
78.21
81.39
90.46
Bad-GAN (semi)
93.13
94.10
76.35
83.71
91.40
Bad-GAN (semi) Good-GAN
96.60
96.25
79.35
85,40
92.52
'Tachi-GAN (ultimate)
97.38
96.71
80.43
85.70
94.01
(a)
(b)
0.96
0.96
0.94
0.92
0.92
0.90
0.90
0.86
0.86
Bed-CAN Good-GAN
Bad-CAN Good-GAN
Bad GAN
(semi) + Good-GAN
Bad GAN (semi) Good-GAN
Bad-GAN GAN
Goad-GAN Lun
Bad-GAN Good-GAN lun
0.82
(somi)
Lye (ultimate)
0.82
(somi)
Lys (uttimate)
1e:2
1e:3
Weight
Weight
Figure 5: (a) Different weights of á on MNIST. (b) Different weights of 5 on Fashion-MNIST. 5 = 0
represents the Good-GAN is disfunctioned.
4.3 ABLATION STUDIES
In addition to the approach-level ablation study mentioned in Variants of Taichi-GAN, three more ablation studies have been conducted: 1) different weights 8 between the Good-GAN and Bad-GAN, 2) different definitions of inlier/outlier, and 3) different availabilities of anomaly samples.
Fig. shows results using different weights on MNIST and Fashion-MNIS, Note compared to Bad-GAN (semi), the Bad-GAN represents that the real anomalies (even when available) are not included in training Dra. thus the comparison between Bad Boad-GAN+Goodd Good-GNN and Bad-GAN can better highlight the contribution of Good-GAN to AD tasks. Table shows results of different definitions on inlier/outlier. The results of different available anomaly numbers/styles are shown in Table 3. Based on those ablation studies, have further findings and analyses follows:
(1) As in Fig. 5, both too large and too small decrease the performance of the Taichi-GAN, Generated pseudo anomalies over-fit few real anomalies (with limited patterns) with too large Contrastingly, too small 5 decreases the efficacy of the Good-GAN (converge to Bad-GAN only).
(2) (ad-GGA(semmi) includes real anomalies in training Das where the discrimination performance can be considerably affected by real anomalies, which does not directly show the efficacy comparison of generated pseudo anomalies across methods. By contrast, the value of the Good-GAr GANN are more
8

---

Under review as a conference paper at ICLR 2022
Table 2: AUCs (%) of different anomaly class (MNIST). For example, {0-4) represents 0-4 are
outliers and the rest inliers. 1 sample per outlier classes is available for bottom three settings.
Anomaly Class
{5}
{0-2]
{0-4)
{0-6)
Bad-GAN
90.21
86.49
73.41
66.72
Bad-GAN + Lorth
91.23
92.12
75.67
75,42
94.72
93.13
84.02
75.94
Bad-GAA(seemi) + Good-GAN
96.67
96.59
90.01
88.41
Taichi-GAl (ultimate)
96.90
97.41
90.12
88.73
Table AUCs (%) of different anomaly availability where all 0-2 are inliers (MNIST). For example, [3-9] represents 5 anomalies per 3-9 classes available.
Available Anomalies
[3-6]11
[3-9]#1
[3-9]
[3-]]
[3-9]*20
BCE
89.01
92.82
95.48
96.82
98.89
Bad-GAN (semi)
91.10
93.13
94.70
95.80
97.62
Bad-GAN (semi) Good-GAN
93.70
96.60
96.80
97.42
99.11
Taichi-GAN (ultimate)
95.33
97.44
97.73
97.81
98.87
clearly highlighted in settings with Bad-GAN in Fig. 5 (c.g.. +9.0% AUC in MNIST when adding Good-GAN to Bad-GAN, i.e., 8 = 0.1 vs. 0 in Bad--GAN
(3) Results from Table prove that contributions of the orthogonal loss and the integration of Good-GAN make improvements to the baseline Bad-GAN consistently, while the degree of improvement varies in different settings and datasets (also supported by Table 1).
(4) When fewer anomalies are available (e.g., settings of "[3-6]*1" and "[3-9]*1" in Table 3), the performance gain from Torrhh and Good-GAN more substantial (+ >4% AUCs).
(5) As Table 3, with increasing number of real anomalies (e.g.. "(3"9]"20"), we find 1) Bad-GAN (semi) can be less effective than BCE, and 2) the Lorth does not necessarily improve the performance. This indicates when available anomalies can already cover a reasonable number of relevant patterns, the generated samples from pure Bad-GAN could introduce noise and thus decrease the discrimination power of Dod- The Lorth is built upon Bad-GAN by increasing the angular diversity of generated samples, and therefore also less effective in this scenario. This could be limitation of the full-blown Taichi-GA when sufficient diversity is already covered by the real anomalies. Yet, variant of Taichi-GAN, Bad-GAN(semi)) Moooseemmi--Goo-Giiii Good-GAN, remains most effective in this experiment.
5 CONCLUSION
In this paper, we address two key aspects in the Bad-GAN framework for AD that 1) generated pseudo anomalies from the Bad-GAN should have high diversity, 2) available anomalies (even few) can be used to guide pseudo anomaly generation. We propose new Taichi-GAN that balances "Good-GAN" with a "Bad-GAN" during training. Evaluated on five datasets, our model achieves the most promising performance in AD tasks compared with representative state-of-the-art methods. Instead of simply combine two types of GANs, Taichi-GAN value lies in the data efficiency as below.
Further Discussion. The key point of end-to-end anomaly scoring is train effective classifier (i.e., the discriminator of Bad-GAN) to distinguish unseen inliers and outliers, under the contexts that inliers are relatively sufficient while no or few real outliers are available during training. Therefore. the management of pseudo outlier generation is critical. When no outliers are available, the Bad-GAN trained to generate peripheral samples of the inlier distribution as pseudo outliers. We validate in our experiments that higher sampling diversity achieved by the proposed orthogonal loss improves the quality of pseudo outliers towards a better classifier in inference. When few outliers are available, we demonstrate that the most effective way (compared with existing benchmarks with empirical experiments) of leveraging the available outliers is through our proposed Taichi-GAN which enable the guidance from real outliers to the generation of pseudo outliers (e.g., the pseudo outlier boundary gets "thicker" near real outliers in Fig. 2(d)). We believe the core ideas of this work can be extended to other unsupervised highly imbalanced classification tasks.

---

Under review as a conference paper at ICLR 2022
Ethics Statement. The concepts and information presented in this paper based research results that are not commercially available. Future commercial availability cannot be guaranteed.
REFERENCES
Samet Akcay, Amir Atapour-Abarghouii, and Toby P. Breckon. GANomaly: Semi-Supervised Anomaly Detection via Adversarial Training. ACCV, pp. 622-637. 2018.
Martin Arjovsky, Amar Shah, and Yoshua Bengio. Unitary Evolution Recurrent Neural Networks. In ICML, 11 2015.
Spyridon et al. Bakas. Identifying the Best Machine Learning Algorithms for Brain Tumor Segmenta- tion, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge. In arXiv preprint arXiv.181..02622. volume 124, 2018.
Christoph Baur, Benedikt Wiestler, Shadi Albarqouni, and Nassir Navab. Deep autoencoding models for unsupervised anomaly segmentation in brain MR images. In International MICCCAI Brainlesion Workshop, pp. 161-169. Springer Verlag, 2018.
Andrew Brock, Theodore Lim, J. M. Ritchie, and Nick Weston. Neural Photo Editing with Introspec- tive Adversarial Networks. ICLR, 2017.
Raghavendra Chalapathy, Aditya Krishna Menon, and Sanjay Chawla. Anomaly Detection using One-Class Neural Networks. In preprint arXiv: (802.063600 2018.
VhandolaVarun, Bridaac,, and Kumar Vipin. Anomaly Detection: A Survey, ACM Comput- ing Surveys (CSUR), 14(1):1-22, 7 2009.
Xiaoran Chen and Ender Konukoglu. Unsupervised detection of lesions in brain MRI using con- strained adversarial auto-encoders. In arXiv:1806.04992 arXiv, 2018.
Zihang Dai, Zhilin Yang, Fan Yang, William W Cohen, and Ruslan Salakhutdinov. Good semi- supervised learning that requires a bad GAN. In NIPS, pp. 6511-6521, 2017.
Tom Fawcett. An introduction to ROC analysis. Pattern Recognition Letters, 27(8):861-874, 2006. 2006.
ISSN 01678655. doi: 11t.:/6ff.parre22005.0....
lan J Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In NIPS, volume 3, pp. 2672-2680, 2014.
Xu Han, Xiaohui Chen, and Li-Ping Liu. GAN Ensemble for Anomaly Detection. In AAAI, 12 2021. ISBN 2012.07988v1. URL httpt:tt///o //tp///axvvvv//////////////aaava org/ abs /2022.2222222 07 97988v1.
Dan Hendrycks, Mantas Mazeika, and Thomas Dietterich. Deep Anomaly Detection with Outlier Exposure. ICLR, 12 2019.
Chaoqin Huang, Fei Ye, Ya Zhang, Yan-Feng Wang, and Qi Tian. ESAD: End-to-end Deep Semi- supervised Anomaly Detection. In arXiv:2012.049900 04905, 12 2020.
Alex Krizhevsky. Learning Multiple Layers of Features from Tiny Images. Technical report, 2009.
Yann LeCun, Leon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEFF,
1998.
Kimin Lee, Honglak Lee, Kibok Lee, and Jinwoo Shin. Training confidence-calibratee classifiers for detecting out-of-distte of-diistibuttion samples. In ICLR, 11 2018.
Swee Kiat Lim, Yi Loo, Ngoc Trung Tran, Ngai Man Cheung, Gemma Roig, and Yuval Elovici. DOPING: Generative Data Augmentation for Unsupervised Anomaly Detection with GAN. Pro- ceedings IEEE International Conference Data Mining, ICDM, 1011-NNvember:1122-1111 2018. ISSN 15504786. doi: 10. 10910009000200188..0 0014.6. URL https : /ttpiv.iiivv////a///aa//aaaaa org/abs/ 1808. 07632v2.
8
10

---

Under review as a conference paper at ICLR 2022
Weiyang Liu, Rongmei Lin, Zhen Liu, Li Xiong, Bernhard Schölkopf. and Adrian Weller. Learning with Hyperspherical Uniformity. In International Conference On Artificial Intelligence and Statistics, 2021.
Philipp Liznerski, Lukas Ruff, Robert A. Vandermeulen, Billy Joe Franks. Marius Kloft, and Klaus- Robert Müller. Explainable Deep One-Class Classification. ICLR, 7 2021.
Cuong Phuc Ngo, Amadeus Aristo Winarto, Connie Kou Khor Li, Sojeong Park, Farhan Akram, and Hwee Kuan Lee. Fence GAN: Towards Better Anomaly Detection. ICTAI, pp. 141-148, 4 2019.
Guansong Pang, Chunhua Shen, Longbing Cao, and Anton van den Hengel. Deep Learning for Anomaly Detection: A Review. ACM Computing Surveys, 2021a.
Guansong Pang, Anton van den Hengel, Chunhua Shen, and Longbing Cao. Toward Deep Supervised Anomaly Detection. In KDD, pp. 1298-1308, 9 2021b.
Pramuditha Perera, Ramesh Nallapati, and Bing Xiang. OCGAN: One-class novelty detection using gans with constrained latent representations. In CVPR, volume 2019-June, pp. 2893-2901, 2019, ISBN 1221112229999
Lukas Ruff, Robert Vandermeulen, Nico Goernitz, Lucas Deecke, Shoaib Ahmed Siddiqui, Alexander Binder, Emmanuel Müller, and Marius Kloft. Deep One-Class Classification. In ICML, pp. 4393- 4402. PMLR, 7 2018.
Lukas Ruff, Jacob R. Kauffmann, Robert A. Vandermeulen, Grégoire Montavon, Wojciech Samek, Marius Kloft, Thomas G. Dietterich, and Klaus-Robert Müller. Unifying Review of Deep and Shallow Anomaly Detection. Proceedings of the IEEE, 9 2020a.
Lukas Ruff, Robert A. Vandermeulen, Nico Görnitz, Alexander Binder, Emmanuel Müller, Klaus- Robert Müller, and Marius Kloft. Deep Semi-Supervised Anomaly Detection. In ICLR. arXiv, 2020b.
Mohammad Sabokrou, Mohammad Khalooei, Mahmood Fathy, and Ehsan Adeli. Adversarially Learned One-Class Classifier for Novelty Detection. CVPR, pp. 1379-3388 2 2018.
Tim Salimans, Andrej Karpathy, Xi Chen, and Diederik P. Kingma. PixeICNN+: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modifications. ICLR, 2017.
Daisuke Sato, Shouhei Hanaoka, Yukihiro Nomura, Tomomi Takenaga, Soichiro Miki, Takeharu Yoshikawa, Naoto Hayashi, and Osamu Abe. A primitive study on unsupervised anomaly detection with an autoencoder in emergency head CT volumes. SPIE medical imaging, 2018
Thomas Schlegl, Philipp Seeböck, Sebastian M. Waldstein, Ursula Schmidt- Erfuurth, and Georg Langs. Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. In IPMI, pp. 146-147, 3 2017.
Thomas Schlegl, Philipp Seeböck, Sebastian M. Waldstein, Georg Langs, and Ursula -hhmidt-Erfurth. f-AnoGAN: Fast unsupervised anomaly detection with generative adversarial networks. Medical Image Analysis, 54:30-44, 2019.
Bernhard Schölkopf, John C. Platt, John Shawe-Taylor, Alex J. Smola, and Robert C. Williamson. Estimating the support of a high-dimensional distribution. Neural Computation, 13(7):14431471,, 2001.
Md Mahfuzur Rahman Siddiquee, Zongwei Zhou, Nima Tajbakhsh, Ruibin Feng, Michael Gotway, Yoshua Bengio, and Jianming Liang. Learning fixed points in generative adversarial networks: From image-to-image translation disease detection and localization. In ICCV, volume 2019- Octob, pp. 191-200, 2019. ISBN 9781728 07728181033
David M.J. Tax and Robert P.W. Duin. Support Vector Data Description. Machine Learning, 54(1): 45-66, 1 2004.
Yu Tian, Gabriel Maicas, Leonardo Zorron Cheng Tao Pu, Rajvinder Singh, Johan W. Verjans, and Gustavo Carneiro. Few-Shot Anomaly Detection for Polyp Frames from Colonoscopy. In MICCAI, pp. 274-284, 6 2020.
11

---

Under review as a conference paper at ICLR 2022
Philipp Tschandl, Cliff Rosendahl, and Harald Kittler. The HAM10000 dataset, large collection of multi-source dermatoscopic images of common pigmented skin lesions. Scientific Data, 5(1):1-9, 2018.
UPenn. Multimodal Brain Tumor Segmentation Challenge 2020: Data, 2020. URL https:
//www. www...ed.dd upenn edu/ cttcc//baatttt0000/00 http:////odaaaaammm......
Peng Wu, Jing Liu, and Fang Shen. A Deep One-Class Neural Network for Anomalous Event Detection in Complex Scenes. JEEE TNNLS, 2019. ISSN 21622388.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mniST: mniST: novel image dataset for benchmark- ing machine learning algorithms. In arr.v::108..0777 07747,, 8 2017.
Houssam Zenati, Chuan Sheng Foo, Bruno Lecouat, Gaurav Manek, and Vijay Ramaseshan Chan- drasekhar. Efficient GAN-Baser Anomaly Detection. In ICLR workshop. arXiv, 2 2018a.
Houssam Zenati, Manon Romain, Chuan Sheng Foo, Bruno Lecouat, and Vijay Chandrasekhar. Ad- versarially Learned Anomaly Detection. In ICDM, pp. 727-736, 12 2018b. ISBN 9781538691588.
Junbo Zhao, Michael Mathieu, and Yann LeCun. Energy-based Generative Adversarial Network. In NIPS, 9 2016.
Panpan Zheng, Shuhan Yuan, Xintao Wu, Jun Li, and Aidong Lu. One-Class Adversarial Nets for Fraud Detection. In AAAI. arXiv, 3 2019.
A BASELINES AND HYPEE-PARAMETERS
For baselines of benchmark Table I, we adapt their official public code into our datasets/settings:
OC-SVM: CCW.At/tt////ghhhhhh....o//////a
SSVD: SST.9.tttee/stttuo.ooom///////sp
PyTorch
Original BadGAN: bactdhhwhttrr//hhhhhboooo//////s
FCDD, FCDD + OE: h0t009/00000...//rr///////aaaa/0
SAD: httpg//gtpgggggaaaoooo///aa/////
FencGGAN: httpechh....to///.tttcccccccoooc
For the Variants of Taichi-GAN mentioned in Section 4.1, we share the same network structure and hyper-parameter for the fair comparison. We did not include additional data augmentation across all compared methods. Our experiments are conducted with PyTorrch 1.1.
Our hyper-parameters are highly motivated from FenceGAN (Ngo et al., 2019), we use FenceGAN as the "Bad-GAN" in model. We follow the hapereparameters of FenceGAN paper in MNIST, ie., a 0.9, 3 = 30, 7 = 0.5 for (q.(6. Note that FenceGAN set a 0.1 for MNIST since the defined as 0 and anomalies
normal cases = 0.9 in this paper. For the balance hyper- prpee-prraeeter between "Bad-GAN" and "Good-GAN". set 5 = 1 for MNIST, 5 0.1 for Fashion-MNIST, HAM10000 and BraTS20. An ablation study of different has conducted Section 4.3. If not specifically noted, the hyper-parameter v) is set to 0.1.
defined as 1 in their paper, which is equivalent to
B DATA SPLITS
We use the same data splits and the same evaluation criteria for all compared methods for fair comparison. Note that the splits and criteria in some of original official Github are different, we have made necessarily adaptions about the data loader and evaluation checkpoint. The AUC results are reported test sets. Note that all normal samples in the training set will be used for training, while no or few anomalies are included depending on settings. The general introduction of datasets can be found in Section 4.1. The following about data splits:
12

---

Under review as a conference paper at ICLR 2022
MNIST & Fashion-MNIST: Original training set of MNIST contains 60000 samples from 10 classes. We randomly split the original training set as 90% 10% for training and validation splits. We use the original test set set and the model selected based on the performance of validation set.
test set, which contains 10000 samples. The AUCs reported test
Original training set of MNIST contains 50000 samples from 10 classes. We randomly split the original training set as 90% / 10% for training and validation splits. We use the original test as test set, which contains 10000 samples.
CIFARIO:
HAM10000: The HAM10000 consists of 10015 dermatoscopic images can be used for academic machine learning purposes. We split the HAM10000 dataset into trainingvvlldaaiionnestt split is 7001/1001/2013 samples for the AD task.
Bra All the data in our experiments are from 369 patients of original BraTS20 training set. The valiiing/aliaatio//tess splits are 12534/1351/264 slices extracted from 280/30/59 patients.
C USED METRIC DISCUSSION IN THIS PAPER
We use the Area Under receiver operating characteristic Curve (AUC) to evaluate the model's performance. We want have an overall discrimination performance measurement benchmark. AUC is known for the discrimination comparison across models and widely used in related references such as Deep SVDD and FCDD. When working on a specific application (which is not the target of this paper), for example, when there is preference on less false positive rate or false negative rate, then precision / recall based metries might be more preferred.
D SPECIAL HANDLING OF BRATS20
Different from other three datasets, the intra-class variations can be larger than inter-class variations at pixel-level in BraTS20 (as the case in upper left Fig, 6), which increase the difficulty to learn compact normal representation. Instead of feeding original image to AD detection models, we create processed different maps with F-AnoGAN (Schlegl et al., 2019). The advantages of creating the difference map include that all normal samples have the same ideal difference map (i.e., all-zero matrix), without the need to capture patterns of original images.
E NETWORK STRUCTURES
We adapt the demonstration style of Liznerski al. (2021) show network structures in different datasets. Table 4, 5 show the generator and discriminator of MNIST and Fashion-MNST. Table 6, show the generator and discriminator of HAM10000. Table 8, 9 show the generator and discriminator of Bra0
13

---

Under review as a conference paper at ICLR 2022
Yort Original Image
Xrec Reconstructed Image
Xrri
Xort Xrec
Post- process
F-anoGAN
Diff
Anomolles ore used training F-anoGAN
Taichi-GAN
Dpdd
Anamalies are used when the Good-GAN Included
Inference
Figure 6: Anomaly detection pipeline for BraTS20. The upper left is examples of origin brain slices, ie., two normal cases (upper) and one anomaly case (lower). Note that only normal cases are used for training F-anoGAN. Orange components (upper) are pre-processing steps for creating inputs (processed difference map) for the Taichi-GAN The post-process includes three steps motivated from Baur et (2018): 1) threshoding with 0.3, 2) Gaussian Blur the image with kernel size 5 and standard deviation values [1, 1]. 3) median filter with kernel size 5 5. The right bottom is inputs of Taichi-GAN and the left bottom are generated pseudo anomalies by Taichi-GAN, In the inference stage, the anomaly score of each slice is obtained using D
Table 4: Network structures of generator for MNIST, Fashion-MNIST
Layer (type)
Output Shape
Param #
Linear-1
[-1, 256]
51,456
ReLU-2
[56]
0
BatchNormld-3
[-1, 256]
512
Linear-4
[-1]
131,584
ReLU-5
[1,5122]
BatchNorm1d-6
[-1,512]
1024
Linear-7
[2,44
3,217,538
ReLU-8
[2744
0
BatchNormld-9
[-1.6274]
12.544
Reshape-11 10
[-1, 7,
Conv ConvTranspose2d- -1
11, 14. 14]
131,136
ReI.U-12
[4, 14, 14]
BatchNorm22d13 13
[-1, 64, 14, 14]
128
ConvTranspose2d-I 14
[-1, 1, 28, 28]
1025
Ta Input shape: 1-1, 200]
[-]. 1,,
0
Total Params: 3.5M
Trainable Params: 3.5M
14

---

Under review as a conference paper at ICLR 2022
Table 5: Network structures of discriminator for MNIST, Fashion-MNIST. The discriminator of Bad-GGAN" and "Good-GAN" share the same structure.
Layer (type)
Output Shape
Param
Conv2d-1
[-, 64, 14, 14]
1088
LeakyReI.U-2
[-1, 64, 14, 14]
0
Conv2d-3
[-1, 64,
65600
LeakyReLU-4
[-1, 64, 7.
0
Reshape-5
[-1,
Linear-6
[-1,512]
1605632
LeakyReLU-7
[-1,512]
0
Linear-8
[-1,
655664
LeakyReLUU77
[-,, 128]
Linear-8
[]
256
Sigmoid-9
[-1,
Input shape: 28,
28]
Total Params: 1.7M
Trainable Params: 1.7M
Table 6: Network structures of generator for HAM10000
Layer (type)
Output Shape
Param #
Linear-1 I
51,456
ReLU-2
[-1,
0
BatchNorm1d-3
512
Linear-4
[-1,
131,584
ReLU-5
[12,
BatchNorm
[-,,
1024
Linear-7
[11,608]]
2363904
ReLU-8
[61,4
0
BatchNorm
[61,4608]
9216
Reshape-10 10
[-1, 128, 6, 6]
NonyTranspose22-11
[1, 12]
131,136
ReLU-12
[1,, 12, 12]
BatchNNrm2d-13
[1, 12, 12]
128
ConsTranspose2d-
[-1, 64,
65600
ReLU-15
64, 24]
0
BatchNorm22-11 16
[-1, 64, 24,
128
ConvTranspose22d-17 17
[1,3, 48]
3075
Tanh-18
[1,,3,48 48,
Input shape: [-1, 200]
Total Params: 2.8M
Trainable Params: 2.8M
15

---

Under review as a conference paper at ICLR 2022
Table 7: Network structures of discriminator for HAM10000. The discriminator of "Bad-GAN" and
"Good-GAN" share the same structure.
Layer (type)
Output Shape
Param
Conv2d-1
[-l, 64, 24, 24]
3072
Lea
[-1, 64, 24, 24]
0 65600
Conv2d-3
[-1, 64, 12, 12]
LeakyReLU-4
[-1, 64, 12, 12]
0
Conv2d-5
[-1, 64, 6]
65600
LeakyReLU-6
64,
0
Reshape-7
[-1,2304]
0
Linear-8
[-1,512]
1180160
LeakyReLUU-9
[-,,
0
Linear- 10
[-1. 128]
655664
LeakyReLU-11
[-1. 128]
0
Linear-12 12
[-1,
256
Sigmoid-13
[-1, 1]
0
Input shape: [-1, 1, 48, 48]
Total Params: 1.4M
Trainable Params: 1.4M
Table 8: Network structures of generator for BraTS20
Layer (type)
Output Shape
Param
ConnTranspose2d-
256, 6, 6]
1638400
BatchNorm2dd-2
[-1, 256, 6, 6]
512
ReLU-3
[51, 6, 6]
0
2onwTraanposee2-4
[28, 12, 12]
819200
BatchNorm2d-5
[28, 12, 12]
256
ReLU-6
12,,
12]
0
ConvTranspose2d-7
204800
BatchNorm22-8
64, 24]
128
ReI.U-9
[-1,64,
Conv Cony/raanspss22d-10
[-1,32, 48)
51200
BatchNorm26-11
+1,32,48,4 48]
64
ReLU-12
[21,32, 48]
0
Conv TonwTranspose24-13
[1,,22, 96, 96]
25600
BatchNorm2d-11 14
[-1, 96, 96]
64
ReLU-15
[-1,32, 96]
0
Upsample-16
[-1, 192, 192]
0
Conv2d-17
1, 192, 192]
800
Tanh-18
[-1, 1, 192]
Input shape: [-1, 256, 3,3]
Total Params: 2.7M
Trainable Params: 2.7M
16

---

Under review as a conference paper at ICLR 2022
Table 9: Network structures of discriminator for BraTS20. The discriminator of "Bad-GAN" and
"Good- "rood-GAN'AA share the same structure.
Layer (type)
Output Shape
Param #
Conv2d-1
32, 96, 96]
832
LeakyReLU-2
[2, 96, 96]
Conv2d-3
[-1, 64, 48, 48]
51264
InstanceNorm22.44
[-1, 68, 48]
128
LeakyRel.U-5
[-1, 64, 48)
Conv2d-6
[-1, 64, 24]
102464
InstanceNorm2d-7
[-1, 64, 24]
128
LeakyReLU-8
64, 24]
0
Conv2d-9
[1, 12, 12]
204928
InstanceNorm2dd110
[28, 12, 12]
256
LeakyReL.-11
[-1, 128, 12, 12]
Conv2d-12
[-1, 6. 6]
819456
InstanceNorm22-113
[-, 12, 12]
512
LeakyReLU-14
[-1, 12, 12]
Conv2d-15
[-1, 3,
1638656
LeakyReLU-166
[1,
Conv2d-17
1,
2305
Sigmoid-18
1,
Input shape: [-1, 1, 192, 192]
Total Params: 2.8M
Trainable Params: 2.8M
17


Reviews:
[{"review_id": "wSqxRxPLPRy", "reviewer": "Reviewer_HJ5T", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": " * The problem addressed by the authors is important and GAN based AD does seem appropriate for certain datasets. Experimentally, a lot of the experimental improvements are quite small. \n * It would be interesting to see how the method performs on CIFAR10 one v rest, since that seems to be a bit of a standard dataset and fairly challenging for generative model based AD. \n * The paper lacks any theory.\n * The lack of theory is compounded by the rather small experimental setup. Lacking any theory a paper should have very strong experimental support.\n * Clarity is pretty good.\n * Good lit review.\n\nOverall the paper is okay, I don't think the results are impressive enough, or that the proposed method is novel enough for a top tier conference.", "summary_of_the_paper": "In this paper the authors propose a method for image anomaly detection (AD) based on GANs. The use the GAN discriminator based on a method called \"Bad GAN\" where one trains a GAN to produce low likelihood samples (trained using the nominal data) and then use the discriminator, which has been generalized to a large class of samples, to determine if a test sample is anomalous. In the proposed method the authors include $L_{orth}$ to enforce angular diversity of generated samples, a \"good\" gan discriminator that is trained to correctly determine if a sample is generated, and a bad gan discriminator inducing the generator to make points near the boundary of the nominal set distribution. At test time the \"bad\" discriminator used to determine an anomaly score. The authors test their method experimentally on F/MNIST and some medical datasets.", "summary_of_the_review": "Overall the paper is okay, I don't think the results are impressive enough, or that the proposed method is novel enough for a top tier conference.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "pcNjkEhabqY", "reviewer": "Reviewer_h6RB", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall the paper introduces a new method that is of interest for the machine learning community.\n\nStrengths:\n- The paper is relevant to this conference and tackles an important research problem\n- The related work is well presented in this paper and introduces well different anomaly detection methods\n- The contributions of this work are motivated\n\nHowever, my two main concerns are on the experimental methodology and on a less important manner on the writing. \n\nWeaknesses/ Experiments:\n- Dataset generations: it is unclear how the inlier/anomaly sets were selected; on MNIST Table 1, how would the methods perform for other combinations of \"normal\" sets and anomalies (than the 0-2 normal and 3-9 outliers)? on Fashion MNIST what does we \"randomly\" select the T-shirt as normal class mean? Would you attain similar results for other configurations? Why did you change these configurations for the ablation study ? On the labeled anomalies for training how did you decide to include 1 sample per class for MNIST/ Fashion MNIST and 5 per class for HAM10000, etc... ? \n- Evaluation metrics: the authors provide results using AUCs (from an ROC curve, probably) of the anomaly detection methods, yet conventional methods in the literature also consider PRC curves using precision, recall; and eventually F1 scores. Depending on whether the focus is given on positive or negative classes (which may depend on datasets) in the anomaly detection task, both ROC and PRC curves could be discussed. This point is missing in the paper.\n- Model selection: the method uses a set of hyper parameters to combine different losses (delta, Eq. 5; alpha, beta, gamma, etc. Eq. 2, 3; eta Eq. 8) and deep representations with network structures. Even though the paper relies on some previously picked models (information from Appendix A) the model selection procedure is *crucially* lacking in the paper. In Appendix B validation sets are mentioned but how are validated the models? This point is not discussed in the main paper but different hyperparameters are used for different datasets (Appendix A). \n- Statistical significance: Were the experiments randomized and averaged (different random seeds) ? \n\n\nWeaknesses/ Writing:\n- The writing of the different disadvantages presented in the introduction (with the Dis notations) could be improved; especially Dis 3 seems an artefact of the Dis 2.\n- The figures are not always clear; for e.g the orthogonal loss figure does not convey sufficiently the intuition of the orthogonal loss\n- Evaluation metrics are not discussed enough; this is crucial in an empirical study paper. \n\n\n", "summary_of_the_paper": "The authors of this paper propose a GAN-based method for the anomaly detection task. Their method relies on so-called Bad GANs (that uses the trained GAN discriminator to distinguish between inliers and pseudo-anomalies), a new orthogonal loss (that favorises generated samples to cover a larger angular space) and eventually a so-called Good GAN (which generator is shared with the Bad GAN; and discriminator is trained on \"real\" anomalies). The method is evaluated along a set of baselines in experiments that also include ablation studies. ", "summary_of_the_review": "The paper introduces a novel method that could be potentially interesting should the authors clarify all the concerns raised in the previous paragraph and provide extensive experiment results to address them. In the current state and given all the concerns on experimental methodology, I would recommend the rejection of the paper with regards to the standards of the ICLR conference.  \n\nUPDATE:\n\nPlease see the response I have provided to the authors. I decided to lower my score marginally and I encourage the authors to undertake the modifications suggested by the reviewers for future venues, especially on the empirical methodology and the writing. The paper in its current form may not be ready to my mind. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "otMLCigDC9p", "reviewer": "Reviewer_LRLn", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "# Strengths\n\nThe paper is organized well and easy to follow. The ideas to combine GANs of different quality is interesting and novel, however seems to be slightly limited as far as applying them to more complex benchmarks is concerned (see Weaknesses).\n\n# Weaknesses\n\nThis manuscript has a couple of issues that need addressing. For one, the experimental evaluation is somewhat limited, in that it focuses on predominantly simple distributions and benchmarks (e.g. MNIST, FashionMNIST). I would recommend that the authors streamline their experiments for multi-modal problems, i.e. those that use multiple classes from MNIST (shown in Table 2 and 3).\n\nIn particular, the community has moved toward more demanding benchmarks, for example recent work proposes the use of semantic benchmarks for AD, see Ahmed & Courville (AAAI 2020), where 9 out of 10 classes are used as normal, and only 1 out of 10 is anomalous. Aligning experiments on MNIST with this challenging setting could potentially improve the paper's contribution, and make it a bit clearer how it compares to existing work. In addition, I would however strongly recommend that the proposed method is also evaluated on more complex benchmarks, e.g. CIFAR-10.\n\nNext, the proposed method seems to require the availability of labeled anomalies. This is a very strong assumption for AD, and as \"Rethinking Assumptions in Deep Anomaly Detection\" of Ruff et al. (2021) clearly shows, often very simple classification-based methods that require only very few labeled anomalies achieve very robust performance. I am afraid this will make readers wonder why they should go through the trouble of training not one but two GANs to determine anomaly.\n\n# Minor\n\nThere appears to be a formatting issue with the citation of Varun Chandola (2009).\n\nTwo GAN-based anomaly detection works which seem to be missing from this manuscript are:\n- A. Berg, M. Felsberg, and J. Ahlberg. Unsupervised adversarial learning of anomaly detection in the wild. In European Conference on Artificial Intelligence, pages 1002–1008, 2020.\n- L. Deecke, R. Vandermeulen, L. Ruff, S. Mandt, and M. Kloft. Image anomaly detection with generative adversarial networks. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 3–17. Springer, 2018.\n\n# Update\n\nMy rating has been increased slightly in response to a thorough revision, which now includes CIFAR-10 in the experiments of the manuscript. Concerns remain however with regards to performance versus competitors (such as F-AnoGAN), and because recent benchmarks like semantic AD have not been incorporated fully.", "summary_of_the_paper": "The authors propose the coupling of coupling two GANs for anomaly detection: one which focuses on generating examples at the periphery of the distribution (hence the name BadGAN) and one that generates reference examples (the GoodGAN portion). These two generative models result from a joint optimization, and are evaluated on several anomaly detection benchmarks in this manuscript.\n", "summary_of_the_review": "The paper presents an interesting idea that involves multiple GANs, but limits the evaluation to relatively simple benchmarks. Moreover, the proposed method relies on the presence of labeled anomalies, which are very difficult to obtain in practice.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "Qpi80gU94U6", "reviewer": "Reviewer_83KL", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Pros**\n \n1. The paper addresses how to generate pseudo anomalies to improve the anomaly detection. For me, the application of orthogonal loss into the problem itself looks simple but effective. \n \n2. The proposed TaiChi-GAN, which combines Good-GAN, Bad-GAN and Orthogonal loss,  is novel for capturing inliers and generating better pseudo anomalies. Good-GAN is sort of the regularization to Bad-GAN regarding the learning of the anomaly distribution. The design for the framework is reasonable and interesting. \n \n3. This paper provides comprehensive studies and experiments, including both qualitative analysis and quantitative results, to show the effectiveness of the proposed framework. \n\n**Cons**\n\n1. However, I have concerns regarding how Good-GAN learns to generate “pseudo anomalies to resemble the reference distribution from given real anomalies”. Not sure how the authors verify this claim, e.g., when looking at Fig. 2d? \n \n2. It is good to have studies over new hyper-parameters introduced in the framework, e.g., \\gamma, \\eta. \n \n3. MNIST and Fashion MNIST are not too challenging datasets, so it would be good to have experiments in e.g., CIFAR, SVHN like Bad-GAN and GAB-based anomaly papers?\n \n4. May consider citing missing GAN works on anomaly detection (ideally comparing them) in the paper: \n[1] DOPING: Generative Data Augmentation for Unsupervised Anomaly Detection with GAN\n[2] Efficient GAN-Based Anomaly Detection\n[3] GAN Ensemble for Anomaly Detection", "summary_of_the_paper": "The paper proposes a new interesting method for anomaly detection. In particular, to overcome the limitations of existing Bad-GAN, the authors introduce the orthogonal loss to regularize the generation of anomaly samples to be distributed evenly at the periphery of the training data. Furthermore, in the scenario of available anomalies, the authors combine Bad-GAN and Good-GAN together, in which Good-GAN learns to generate the anomalies while Bad-GAN reguralizes the anomaly pseudo anomalies at the boundary of inlier distribution. ", "summary_of_the_review": "Overall, I vote for accepting. I like the idea of combining Good-GAN and Bad-GAN together, which is novel and interesting to me. The proposed orthogonal loss is simple and effective as being used as the complementary of the existing dispersion loss.  The results on benchmark datasets shown in the paper are encouraging and outperforms many existing works in both scenarios without or few anomalies observed. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a method that combines Bad-GAN and Good-GAN, in which Good-GAN learns to generate the anomalies while Bad-GAN reguralizes the anomaly pseudo anomalies at the boundary of inlier distribution. In addition, a new orthogonal loss is proposed to  regularize the generation of anomaly samples to be distributed evenly at the periphery of the training data. The proposed method is new and shows some improvement over existing methods.

However, there are some detailed technical concerns raised by reviewers. Some of the concerns still remain unresolved after the discussion. 1) The proposed method lacks a principled way to select hyperparameters. 2) The experimental setting is a bit simple to verify the effectiveness of the proposed method in challenging real world applications. Especially, there is no theoretical guarantee of the proposed method, empirical evaluation is the only way to show the effectiveness of the proposed method. 3) The overall performance improvement is not very significant compared to existing methods. For example, the performance is very close to a method F-AnoGAN published in 2019. Addressing the concerns needs a significant amount of work. Thus, I do not recommend acceptance of this paper.
Decision: Reject