Title: DIFFERENCE-SEEKING GENERATIVE ADVERSARIAL NETWORK

Abstract: We propose a novel algorithm, Difference-Seeking Generative Adversarial Network (DSGAN), developed from traditional GAN. DSGAN considers the scenario that the training samples of target distribution, p t , are difficult to collect. Suppose there are two distributions p d and p d such that the density of the target distribution can be the differences between the densities of p d and p d . We show how to learn the target distribution p t only via samples from p d and p d (relatively easy to obtain). DSGAN has the flexibility to produce samples from various target distributions (e.g. the out-of-distribution). Two key applications, semi-supervised learning and adversarial training, are taken as examples to validate the effectiveness of DSGAN. We also provide theoretical analyses about the convergence of DSGAN.

1. INTRODUCTION:

In machine learning, how to learn a probability distribution is usually conducted in a unsupervised learning manner. Generative approaches are developed for learning data distribution from its samples and thereafter produce novel and high-dimensional samples from learned distributions, such as image and speech synthesis (Saito et al. (2018) ). The state-of-the-art approaches is so-called Generative Adversarial Networks (GAN) (Goodfellow et al. (2014) ). GAN produces sharp images based on a game-theoretic framework, but can be tricky and unstable to train due to multiple interacting losses. Specifically, GAN consists of two functions: generator and discriminator. Both functions are represented as parameterized neural networks. The discriminator network is trained to classify whether or not inputs belong to real data or fake data created by the generator. The generator learns to map a sample from a latent space to some distribution to increase the classification errors of discriminator. GAN corresponds to a minimax two-player game, which ends if the generator actually learns the real data distribution. The generator is of main interest because the discriminator will be unable to differentiate between both distributions once the generator has been trained well.

1.1. MOTIVATIONS:

In reality, it is difficult to collect training samples from unseen classes, which none of their samples involves the training phase, but their samples could be encountered in the testing phase. How to reject or recognize unseen data as "abnormal" (not belonging to the training data) is an important issue known as one-class classification (Ruff et al. (2018) ). Due to the absence of unseen data, most of algorithms are unsupervised (Scholkopf & Smola (2001) ). Based on the assumption that there do exist very few unseen examples, some approaches (Wu & Ye (2009) ) focus on supervised learning using unbalanced data. In addition to one-class classification, Dai et al. (2017) theoretically show that complementary data, which is also considered as unseen data, can improve semi-supervised learning. Another related issues is adversarial attack, where classifiers may be vulnerable to adversarial examples, which are unseen during the training phase. 8) in Appendix A illustrates the applications regarding unseen data. Apparently, if we can generate unseen data via GAN, it is helpful for those applications. But, traditional GAN requires preparing plenty of training samples of unseen classes for training, leading to the contradiction with the prerequisite. This fact motivates us to design the proposed DSGAN, which can generate unseen data by taking seen data as training samples shown in Fig. (1) . The nuclear idea is to consider the distribution of unseen data as the difference of two distributions, which both are relatively easy to obtain. For example, out-of-distribution examples in the MNIST dataset, from another point of view, are found to belong to the difference of the set of examples in MNIST and the universal set.

Examples in both sets are relatively easy to obtain. It should be noted that the target distribution is equal to the training data distribution in traditional GANs; nevertheless, both distributions are considered different in DSGAN.

Figure 1 : The illustration of difference between traditional GAN and DSGAN.

1.2. CONTRIBUTIONS:

In this paper, we make the following contributions:

(1) We propose a novel algorithm, Difference-Seeking Generative Adversarial Network (DS-GAN), where the density of target distribution p t is the difference between those of any two distributions, p d and p d . Nevertheless, the differences of two densities being negative are not well-defined. Thus, instead of learning the target distribution p t directly, the generator distribution approximates p t by minimizing the statistical distance between the mixture distribution of p d and the generator distribution, and p d.

(2) Theoretical results based on traditional GAN are extended to the case of mixture distribution considered in this paper. With enough capacity of the generator and the discriminator, we show DSGAN can learn the generator distribution p g under mild condition where the support set of p g is the difference of support sets of p d and p d .

(3) We show that DSGAN possesses the flexibility to learn different target distributions in two key applications: semi-supervised learning and adversarial training. Samples from target distribution in semi-supervised learning must satisfy two conditions: i) are linear combination of any label data and unlabel data; ii) do not belong to neither label data nor unlabel data. For adversarial training, samples from the target distribution are assigned as out-of-distribution examples with bounded distortion. Experiments validate that DSGAN can learn these two kinds of distributions well in various datasets.

The paper is structured as follows. In Sec. 2, we introduce DSGAN, including the algorithm in the training procedure. Theoretical results are described in Sec. 3. In Sec. 4, two applications are taken as examples to show the effectiveness of DSGAN. Finally, we present the experimental results in Sec. 5 and conclude by pointing out some promising directions for future work in Sec. 7.

2. PROPOSED METHOD-DSGAN:



2.1. THE SCHEME OF DSGAN:

We denote the generator distribution as p g and training data distribution as p d , both in a Ndimensional space. Let p d be the distribution decided by user. For example, p d can be the convolution of p d and normal distribution. Let p t be the target distribution which the user is interested in, and it can be expressed as

EQUATION

where α ∈ [0, 1]. Our method, DSGAN, aims to learn p g such that p g = p t . Intuitively, our method tries to learn p g such that p g (x) ∼ p d(x) -αp d (x) 1 -α . In other words, the generator is going to output samples located in high-density areas of p d and low-density areas of p d .

At first, we formulate the generator and discriminator in GANs. The inputs z of the generator are drawn from p z (z) in an M -dimensional space. The generator function G(z; θ g ) : R M → R N represents a mapping to data space, where G is a differentiable function with parameters θ g . The discriminator is defined as D (x; θ d ) : R N → [0, 1] that outputs a single scalar. D (x) can be considered as the probability that x belongs to a class of real data.

Similar to traditional GAN, we train D to distinguish the real data and the fake data sampled from G. Meanwhile, G is trained to produce realistic data as possible to mislead D. But, in DSGAN, the definitions of "real data" and "fake data" are different from those in traditional GAN. The samples from p d are considered as real but those from the mixture distribution between p d and p g are considered as fake. The objective function is defined as follows:

min G max D V (G, D) = E x∼p d (x) [log D(x)] + (1 -α)E z∼pz(z) [log (1 -D (G (z)))] + αE x∼p d (x) [log (1 -D(x))] .

( 

06.:

Update the discriminator by ascending its stochastic gradient:

∇ θ d 1 m m i=1 log D x (i) d + log 1 -D G z (i) + log 1 -D x (i) d

07.:

end for 08.

Sample minibatch of m noise samples z (1) , ..., z (m) from p g (z).

09.:

Update the generator by descending its stochastic gradient:

∇ θg 1 m m i=1 log 1 -D G z (i)

10. end for

2.2. CASE STUDY ON SYNTHETIC DATA AND MNIST:

To get more intuitive understanding about DSGAN, we conduct several case studies on 2D synthetic datasets and MNIST. Those results validate that DSGAN can learn the distribution we desire. Low-density samples generation Fig. 2 illustrates that DSGAN is able to generate boundary samples on the 2D swissroll. Given the density function of the swissroll as p d , we assign p d as the convolution of p d and the normal distribution. Then, by applying DSGAN, we achieve our goal to generate boundary samples. The intuition of our idea is also illustrated by a 1D example in Fig. 4 . In general, our idea will lead DSGAN to generate low-density samples like another example in Fig. 3 . In this case, the density is low in the center of the circle, and our generator can not only create the boundary samples but also the samples located in low-density area.

Difference-set generation We also validate DSGAN on high dimensional dataset such as MNIST.

In this example, we define p d be the distribution of digit "1" and p d be the distribution contains both digits "1" and "7". Since the density p d (x) is high when x is digit "1", the generator is prone to output digit "7" with high probability.

From the above results, we can observe two properties of generator distribution p g : i) the higher density of p d (x), the lower density of p g (x); ii) p g prefers to output samples from high-density areas of p d(x)-p d (x). In those case studies, α = 0.8 in Eq. ( 1) is used.

In the next section, we will show that the objective function is equivalent to minimizing the Jensen-Shannon divergence between the mixture distribution (p d and p g ) and p d as G and D are given enough capacity. Furthermore, we provide a trick (see Appendix C in details) by reformulating the objective function ( 2) such that it is more stable to train DSGAN.

3. THEORETICAL RESULTS:

There are two assumptions for subsequent proofs. First, in a nonparametric setting, we assume both generator and discriminator have infinite capacity. Second, p g is defined as the distribution of the samples drawn from G(z) under z ∼ p z . We will first show the optimal discriminator given G and then show that minimizing V (G, D) via G given the optimal discriminator is equivalent to minimizing the Jensen-Shannon divergence between (1 -α)p g + αp d and p d. Proposition 1. For G being fixed, the optimal discriminator D is

D * G (x) = p d (x) p d (x) + (1 -α)p g (x) + αp d (x)

.

Proof. See Appendix B.1 in details.

Moreover, D can be considered to discriminate between samples from p d and those from

((1 -α)p g (x) + αp d (x))

. By replacing the optimal discriminator into V (G, D), we obtain

EQUATION

where the third equality holds because of the linearity of expectation.

Actually, the previous results show the optimal solution of D given G being fixed in (3). Now, the next step is to find the optimal G with D * G being fixed.

Theorem 1. Suppose αp d (x) ≤ p d(x) for all x's. The global minimum of the virtual training criterion C(G) is achieved if and only if (1 -α)p g (x) + αp d (x) = p d(x) for all x's. At that point, C(G) achieves the value -log 4. Proof. See Appendix B.2 in details.

The assumption, αp d (x) ≤ p d(x) for all x's, in Theorem 1 may be impractical in real applications. We discuss that DSGAN still works well even though the assumption does not hold. There are two facts: i) given D, V (G, D) is a convex function in p g and ii) Due to x p g (x)dx = 1, the set collecting all feasible solutions of p g is a convex set. In other words, there always exists a global minimum of V (G, D) given D, but it may not be -log(4). In this following, we show that the support set of p g is contained within the difference of support sets of p d and p d while achieving the global minimum such that we can generate the desired p g by designing appropriate p d. Proof. See Appendix B.3 in details.

In sum, the generator is prone to output samples located in high-density areas of p d and low-density areas of p d .

Another concern is the convergence of Algorithm 1. Proposition 3. The discriminator reaches its optimal value given G in Algorithm 1, and p g is updated by minimizing

E x∼p d (x) [log D * G (x)] + E x∼(1-α)pg(x)+α∼p d (x) [log (1 -D * G (x))] . If G and D have enough capacity, then p g converges to argmin pg JSD (p d (1 -α)p g + αp d ).

Proof. See Appendix B.4 in details.

4. APPLICATIONS:

DSGAN can be applied to two applications: semi-supervised learning and adversarial training. In semi-supervised learning, DSGAN acts as a "bad generator", which creates complement samples in the feature space of real data. As for adversarial training, DSGAN generates adversarial examples located in the low-density areas of training data.

4.1. SEMI-SUPERVISED LEARNING:

Semi-supervised learning (SSL) is a kind of learning model with the use of a small number of labeled data and a large amount of unlabeled data. The existing SSL works based on generative model (e.g., VAE (Kingma et al. ( 2014)) and GAN (Salimans et al. (2016) )) obtain good empirical results. Dai et al. (2017) theoretically show that good semi-supervised learning requires a bad GAN with the objective function:

EQUATION

where (x, y) denotes a pair of data and its corresponding label, {1, 2, . . . , K} denotes the label space for classification, and L = {(x, y)} is the label dataset. Moreover, in the semi-supervised settings, the p d in (4) is the distribution of the unlabeled data. Note that the discriminator D in GAN also plays the role of classifier. If the generator distribution exactly matches the real data distribution (i.e., p g = p d ), then the classifier trained by the objective function (4) with the unlabeled data cannot have better performance than that trained by supervised learning with the objective function:

max D E x,y∼L log P D (y | x, y ≤ K) .

(5)

On the contrary, the generator is preferred to generate complement samples, which lie on lowdensity area of p d . Under some mild assumptions, those complement samples help D to learn correct decision boundaries in low-density area because the probabilities of the true classes are forced to be low on out-of-distribution areas.

The complement samples in Dai et al. (2017) are complicate to produce. We will demonstrate that DSGAN is easy to generate complement samples in Sec. 5.

4.2. ADVERSARIAL TRAINING:

Deep neural networks have impacted on our daily life. Neural networks, however, are vulnerable to adversarial examples, as evidenced in recent studies (Papernot et al. (2016) ) (Carlini & Wagner (2017) ). Thus, there has been significant interest in how to enhance the robustness of neural networks. Unfortunately, if the adversary has full access to the network, namely white-box attack, a complete defense strategy has not yet been found. We first introduce how DSGAN generates complement samples in the feature space. Specifically, Dai et al. (2017) proved that if complement samples generated by G can satisfy the following two assumptions in ( 7) and ( 8):

EQUATION

)

where f is the feature extractor and w i is the linear classifier for the i th class and

EQUATION

then all unlabeled data will be classified correctly via the objective function (4).

The assumption in (8) implies the complement samples have to be at the space created by linear combination of labeled and unlabeled data. Besides, they cannot fall into the real data distribution p d due to the assumption (7). In order to have DSGAN generate such samples, we let the samples of p d be the linear combination of those from L and p d . Since

p g (x) ≈ p d(x) -αp d (x) 1 -α

, the p g will tend to match p d while the term -αp d ensures that samples from p g do not belong to p d . Thus, p g satisfies both assumptions in ( 7) and ( 8).

In practice, we parameterized f and all the w together as a neural network. The details of the experiments, including the network models, can be found in Appendix D.

5.1.1. MNIST, SVHN, AND CIFAR-10:

For evaluating the semi-supervised learning task, we used 60000/ 73257/ 50000 samples and 10000/ 26032/ 10000 samples from the MNIST/ SVHN/ CIFAR-10 dataset for training and testing, respectively. Due to the semi-supervised setting, we randomly chose 100/ 1000/ 4000 samples from the training samples as the MNIST/ SVHN/ CIFAR-10 labeled dataset, and the amount of labeled data for all classes are equal.

Our criterion to determine the hyperparameters are in Appendix D.1. We perform testing with 10/ 5/ 5 runs on MNIST/ SVHN/ CIFAR-10 based on the selected hyperparameters and randomly selected labeled dataset. Following Dai et al. (2017) , the results are recorded as the mean and standard deviation of number of errors from each run.

5.1.2. MAIN RESULTS:

First, the hyperparameters we chose is depicted in Table 3 in Appendix D.1. Second, the results obtained from our DSGAN and the state-of-the-art methods on three benchmark datasets are depicted in Table 1 .

It can be observed that our results can compete with state-of-the-art methods on the three datasets. Moreover, in comparison with Dai et al. (2017) , our methods don't need to rely on an additional density estimation network PixelCNN++ (Salimans et al. (2017) ). Although PixelCNN++ is one of the best density estimation network, it cannot estimate the density in the feature space, which is dynamic during training. This drawback make the models in Dai et al. (2017) cannot completely fulfill the assumptions in their paper. Our results in Table 1 is slightly inferior to the best record of badGAN (Dai et al. (2017) ) but outperforms other approaches. In comparison with them, there is a probable reason to explain the slightly inferior performance in the following. Since the patterns of images are complicated, the generator without enough capacity is not able to learn our desired distribution, which is the distribution meets the conditions in ( 7) and ( 8). However, this problem will be attenuated with the improvements of GAN, and our models benefit from them. In badGAN, they rely on the feature matching in their objective function. No matter how they change the divergence criterion on two distributions, feature matching still let them learn the distribution matching the first-order statistics, so they cannot totally get the advantage of the progress of GANs.

5.2. DSGAN IN ADVERSARIAL TRAINING:

Our proposed DSGAN is capable to be used to improve the robustness of the classifier against adversarial examples. In the experiments, we mainly validate DSGAN on CIFAR-10, which is widely used in adversarial training.

Recall that the objective function ( 6 9)). Third, we fine-tune the classifier C θ with all training data and the data produced by the generator via minimizing the following objective:

EQUATION

where the first term is a typical supervised loss such as cross-entropy loss and the second term is the entropy loss H of generated unlabeled samples corresponding to the classifier, meaning that we would like the classifier to confidently classify the generated samples. In other words, if an adversarial example x g is the closest to one of labeled data x, it should be classified into the class of x. Thus, the additional entropy loss will prevent our model from the attack by adversarial examples.

Furthermore, in (9), one can view the weight w is the trade-off between the importance of labeled data in high-density area and unlabeled data in low-density area. If w is 0, the model might be prone to classify correctly only on the labeled data. When increasing w, the model will place more emphasis on unlabeled data. Since the unlabeled data acts as adversarial examples, therefore, the classifier is more robustness.

5.2.1. EXPERIMENTS SETTINGS:

We evaluate the trained models against a range of adversaries, where the distortion is evaluated by 2 -norm or inf -norm. The adversaries include:

• White-box attacks with Fast Gradient Sign Method (FGSM) (Goodfellow et al. (2015) ) using inf -norm.

• White-box attacks with PGD (Kurakin et al. ( 2017)) using inf -norm.

• White-box attacks with Deepfool (Moosavi-Dezfooli et al. ( 2016)) using 2 -norm and infnorm.

According to different adversaries, we generate 10000 adversarial examples from testing data and calculate the accuracy of the model after attacking. The accuracy is record as the probability that adversarial examples fail to attack when the distortion created by attacking algorithm cannot exceed a maximum value. We also train our models with different ranges of uniform distribution. The experimental detail can be found in Appendix E.

To validate our method, we propose two kinds of baseline networks. One is a baseline classifier we train in the first stage, which is a typical classifier trained by all data. The other one is the model with noisy inputs. Adding noise to the input is a prevalent strategy to train a classifier and it is also able to protect the neighborhood of the training data. For fare comparison to our method, uniform noise is used in the second baseline model.

Fig. 7 demonstrates that our models exhibit stronger robustness among all the adversaries. w is set to 10 in this figure, other results with different w are displayed in the Appendix E and we claim that our method can outperform other baselines in a wide range of values of w. We notice that the model benefits from controlling the weight w. When we increase the w from 1 to 3, and then from 3 to 10, the robustness keeps becoming stronger.

Our second baseline models have the similar intuition with our method, they propagate the label information to the neighborhood of each data point by introducing the noise to inputs. This strategy can improve the accuracy and the robustness of the model. Nevertheless, the training data distribution after applying noise can be viewed as a smoother version of original distribution. Most of samples still are locate on high-density area of original distribution. Due to this reason, the second baseline models cannot emphasize low-density samples via w like the proposed model, leading to the inferior robustness.

Our method relies on a generator to produce low-density data. The generated samples help our model to put decision boundary outside low-density area. Thus, the model can resist adversarial attacks with larger distortion theoretically. It's worth mentioning that our method is able to combine with the idea of second baseline to the supervised term in (9) and the performance might be improved.

6. RELATED WORKS:

We introduce related works about generating unseen data. Yu et al. (2017) proposed a method to generate samples of unseen classes in the unsupervised manner via an adversarial learning strategy. But, it requires solving an optimization problem for each sample, which may lead to high computation cost. On the contrary, DSGAN has the capability to create infinite diverse unseen samples. Hou et al. (2018) presented a new GAN architecture that can learn both distributions of unseen data from part of seen data and unlabeled data. But, unlabeled data must be a mixture of seen and unseen samples; DSGAN does not require any unseen data instead. Both Dai et al. (2017) aim to generate complementary samples (or out-of-distribution sample) but assume that in-distribution can be estimated by a pretrained model such as PixelCNN++, which might be difficult and expensive to train. Lee et al. (2018) use a simple classifier to replace the role of PixelCNN++ in Dai et al. (2017) such that the is much easier and more suitable. Nevertheless, their method only focuses on generates unseen data surrounding low-density area of seen data, but DSGAN is more flexible to generate different kinds of unseen data (e.g., the linear combination of seen data shown in Sec.5.1). Besides, their method needs the label information of data while ours is fully unsupervised.

7. CONCLUSIONS:

In this paper, we propose DSGAN that can produce samples from the target distribution based on the assumption that the density of target distribution can be the difference between the densities of any two distributions. DSGAN is useful in the environment when the samples from the target distribution are more difficult to collect than those from the two known distributions. We demonstrate that DSGAN is really applicable to, for example, semi-supervised learning and adversarial training. Empirical and theoretical results are provided to validate the effectiveness of DSGAN. Finally, because DSGAN is developed based on traditional GAN, it is easy to extend any improvements of traditional GAN to DSGAN. 

B.2 PROOF OF THEOREM 1:

Proof. We start from

(3) = -log(4) + E x∼p d (x) log 2p d(x) p d(x) + (1 -α)p g (x) + αp d (x) + E x∼(1-α)pg(x)+α∼p d (x) log 2 ((1 -α)p g (x) + αp d (x)) p d(x) + (1 -α)p g (x) + αp d (x) = -log(4) + KL p d p d + (1 -α)p g + αp d 2 + KL (1 -α)p g (x) + αp d p d + (1 -α)p g + αp d 2 = -log(4) + 2 JSD (p d (1 -α)p g + αp d ) ,

where KL is the Kullback-Leibler divergence and JSD is the Jensen-Shannon divergence. The JSD returns the minimal value, which is 0, iff both distributions are the same, namely p d = (1 -α)p g + αp d . Because p g (x)'s are always non-negative, it should be noted both distributions are the same only if αp d (x) ≤ p d(x) for all x's. We complete this proof.

B.3 PROOF OF PROPOSITION 2:

Proof. Recall Furthermore,

C(G) = x p d(x) log p d(x) p d(x) + (1 -α)p g (x) + αp d (x) + ((1 -α)p g (x) + αp d (x)) log (1 -α)p g (x) + αp d (x) p d(x) + (1 -α)p g (x) + αp d (x) dx = x S(

T (p g ; x) = ∂S(p g ; x) ∂p g (x) = log (1 -α)p g (x) + αp d (x) p d(x) + (1 -α)p g (x) + αp d (x)

is increasing on p g (x) and converges to 0. When x ∈ Supp(p d) Supp(p d ), T (p g ; x) ≥ log 1 2 always holds due to the assumption αp d (x) ≥ p d(x). Based on the following expression,

x∈Supp(p d )-Supp(p d ) p d(x)dx + x∈Supp(p d ) p d(x)dx = 1 ⇒ x∈Supp(p d )-Supp(p d ) p d(x)dx ≥ 1 - x∈Supp(p d ) αp d (x)dx ⇒ x∈Supp(p d )-Supp(p d ) p d(x)dx ≥ 1 -α ⇒ x∈Supp(p d )-Supp(p d ) p d(x)dx ≥ x∈Supp(p d )-Supp(p d )

(1 -α)p g (x)dx, the last inequality implies that there exists a solution such that (1 -α)p g (x) ≤ p d(x) for x ∈

Supp(p d) -Supp(p d ) with x∈Supp(p d )-Supp(p d )

p g (d)dx = 1 In this case, T (p g ; x) < log 1 2 for x ∈ Supp(p d) -Supp(p d ). We complete this proof.

B.4 PROOF OF PROPOSITION 3:

Proof. Consider V (G, D) = U (p g , D) as a function of p g . By the proof idea of Proposition 2 in Goodfellow et al. (2014) , if f (x) = sup α∈A f α (x) and f α (x) is convex in x for every α, then ∂f β (x) ∈ ∂f if β = argsup α∈A f α (x). In other words, if sup D V (G, D) is convex in p g , the subderivatives of sup D V (G, D) includes the derivative of the function at the point, where the maximum is attained, implying the convergence with sufficiently small updates of p g . We complete this proof.

C TRICKS FOR STABLE TRAINING:

We provide a trick to stabilize the training procedure by reformulating the objective function. Specifically, V (G, D) in ( 2) is reformulated as:

EQUATION

Instead of sampling a mini-batch of m samples from p z and p d in Algorithm 1, (1 -α)m and αm samples from both distributions are required, respectively. The computation cost in training can be reduced due to fewer samples. Furthermore, although (10) is equivalent to (2) in theory, we find that the training using (10) achieves better performance than using (2) via empirical validation in Table 2 . We conjecture that the equivalence between (10) and ( 2) is based on the linearity of expectation, but mini-batch stochastic gradient descent in practical training may lead to the different outcomes. The hyperparameters were chosen to make our generated samples consistent with the assumptions in ( 7) and ( 8). However, in practice, if we make all the samples produced by the generator following the assumption in (8), then the generated distribution is not close to the true distribution, even a large margin between them existing in most of the time, which is not what we desire. So, in our experiments, we make a concession that the percentage of generated samples, which accords with the assumption, is around 90%. To meet this objective, we tune the hyperparameters. Table 3 shows our setting of hyperparameters, where β is defined in (8). Dai et al. (2017) . However, different from previous works that have only a generator and a discriminator, we design an additional discriminator in the feature space, and it's architecture is similar across all datasets with only the difference in the input dimensions. Following Dai et al. (2017) , we also define the feature space as the input space of the output layer of discriminators.

Compared to SVHN and CIFAR-10, MNIST is a simple dataset as it is only composed of fully connected layers. Batch normalization (BN) or weight normalization (WN) is used to every layer to stable training. Moreover, Gaussian noise is added before each layer in the classifier, as proposed in Rasmus et al. (2015) . We find that the added Gaussian noise exhibits positive effect for semisupervised learning and keep to use it. The architecture is shown in Table 4 .

Table 5 and Table 6 are models for SVHN and CIFAR-10, respectively, and these models are almost the same except for some implicit differences, e.g., the number of convolutional filters and types of dropout. In these tables, given a dropping rate, "Dropout" is a normal dropout in that the elements of input tensor are randomly set to zero while Dropout2d is a dropout only applied on the channels to randomly zero all the elements.

Furthermore, the training procedure alternates between k steps of optimizing D and one step of optimizing G. We find that k in Algorithm 1 is a key role to the problem of mode collapse for different applications. For semi-supervised learning, we set k = 1 for all datasets. 

:



Figures and Tables:

Fig. (8) in Appendix A illustrates the applications regarding unseen data. Apparently, if we can generate unseen data via GAN, it is helpful for those applications. But, traditional GAN requires preparing plenty of training samples of unseen classes for training, leading to the contradiction with the prerequisite. This fact motivates us to design the proposed DSGAN, which can generate unseen data by taking seen data as training samples shown in Fig. (1). The nuclear idea is to consider the distribution of unseen data as the difference of two distributions, which both are relatively easy to obtain. For example, out-of-distribution examples in the MNIST dataset, from another point of view, are found to belong to the difference of the set of examples in MNIST and the universal set.

Figure 2: DSGAN generated boundary points of a swissroll.

Figure 3: Left: A circle which the density is low in the center. Right: Green points are generated by DSGAN.

Figure 4: The illustration about generating boundary data around training data. First, the convolution of p d and normal distribution makes the density on boundary data be no longer zero. Second, we seek p g such that Eq. (1) holds, where the support set of p g is approximated by the difference of those between p d and of p d .

Figure 5: Illustration of difference-set seeking in MNIST.

Figure 6: DSGAN learn the difference between two sets.

Proposition 2. Suppose αp d (x) ≥ p d for x ∈ Supp(p d ) and all density functions p d (x), p d(x) and p g (x) are continuous. If the global minimum of the virtual training criterion C(G) is achieved, then Supp (p g ) ⊆ Supp (p d) -Supp(p d ).

Athalye et al. (2018) surveyed the state-of-the-art defense strategies and showed that adversarial training(Madry et al. (2018)) is more robust than other strategies. Given a trained classifier C parameterized by θ and a loss function (x; y; C θ ), adversarial training solves a min-max game, where the first step is to find adversarial examples within -ball for maximizing the loss, and the second step is to train the model for minimizing the loss, given adversarial examples. Specifically, the objective(Madry et al. (2018)) isargmin θ E (x,y)∼L max δ∈[-, ] N (x; y; C θ ) .(6)The authors used projected gradient descent (PGD) to find adversarial examples by maximizing the inner optimization. Instead of relying on PGD, our DSGAN generates adversarial examples directly, which are combined into real training data to fine-tune C θ . -ball in terms of 2 or inf can be intuitively incorporated into the generation of adversarial examples.5 EXPERIMENTSIn this section, we demonstrate the empirical results about semi-supervised learning and adversarial training in Sec. 5.1 and Sec. 5.2, respectively.Note that, the training procedure of DSGAN can be improved by other extensions of GANs such as WGAN(Arjovsky et al. (2017)), WGAN-GP(Gulrajani et al. (2017)), EBGAN(Zhao et al. (2017)), LSGAN(Mao et al. (2017)) and etc. We use the idea of WGAN-GP in our method such that DSGAN is stable in training and suffers less mode collapse.5.1 DSGAN IN SEMI-SUPERVISED LEARNINGFollowing the previous works, we apply the proposed DSGAN in semi-supervised learning on three benchmark datasets, including MNIST (LeCun et al. (1998)), SVHN(Netzer et al. (2011)), and CIFAR-10 (Krizhevsky (2009)).

) requires finding adversarial examples to maximize the classification error(•). Adversarial examples usually locate on the low-density area of p d and are generated from labeled data via gradient descent. Instead of using gradient descent, we aim to generate adversarial examples via GAN. By assigning p d as the convolution of p d and uniform distribution, samples from p g will locate on the low-density area of p d . Furthermore, the distortion is directly related to the range of uniform distribution. It, however, may be impractical for training the generator for each class. Thus, we propose a novel semi-supervised adversarial learning approach here.Three stages are required to train our model: First, we train a baseline classifier on all the training data. All the training data are labeled and represent samples from L in (9). Second, we train a generator to generate adversarial examples and treat these adversarial examples as additional unlabeled training data (x ∼ p g in (

Figure 7: The accuracy of baseline and our models after attacks. Blue line is first baseline model. Orange, green and red lines are second baseline models with different range of uniform noise. Purple, brown and pink lines are our methods. The float number also indicates the range of noise where "w10" means that w in (9) is set to 10. The epsilon means that the 2 (or inf ) norm between original image (pixel values are normalized to range [-0.5, 0.5]) and corresponding adversarial example.

Figure 8: The illustration of applications related to unseen data.

FC layer ReLU 400 × 200 FC layer ReLU 200 × 100 FC layer ReLU 100 × 1 FC layer GN, std = 0.3 784 × 1000 FC layer with WN ,ReLU GN, std = 0.5 1000 × 500 FC layer with WN, ReLU GN, std = 0.5 500 × 250 FC layer with WN, ReLU GN, std = 0.5 250 × 250 FC layer with WN, ReLU GN, std = 0.5 250 × 250 FC layer with WN, ReLU 250 × 10 FC layer with WNTable 5: Architectures of generator and discriminator for semi-supervised learning on SVHN and CIFAR-10. N was set to 128 and 192 for SVHN and CIFAR-10, respectively.Generator G Discriminator D Input z ∈ R 100 from unif(0, 1) Input 32 × 32 RGB image 100 × 8192 FC layer with BN, ReLU Reshape to 4 × 4 × 512 5 × 5 conv. transpose 256 stride = 2 with BN, ReLU 5 × 5 conv. transpose 128 stride = 2 with BN, ReLU 5 × 5 conv. transpose 3 stride = 2 with WN, Tanh N × 400 FC layer, ReLU 400 × 200 FC layer, ReLU 200 × 100 FC layer, ReLU 100 × 1 FC layer Table 6: The architecture of classifiers for semi-supervised learning on SVHN and CIFAR-10. (GN: Gaussian noise, lReLU(leak rate): LeakyReLU(leak rate)) Classifier C for SVHN Classifier C for CIFAR-10 Input 32 × 32 RGB image Input 32 × 32 RGB image GN, std = 0.05 Dropout2d, dropping rate = 0.15 3 × 3 conv. 64 stride = 1 with WN, lReLU(0.2) 3 × 3 conv. 64 stride = 1 with WN, lReLU(0.2) 3 × 3 conv. 64 stride = 2 with WN, lReLU(0.2) Dropout2d, dropping rate = 0.5 3 × 3 conv. 128 stride = 1 with WN, lReLU(0.2) 3 × 3 conv. 128 stride = 1 with WN, lReLU(0.2) 3 × 3 conv. 128 stride = 2 with WN, lReLU(0.2) Dropout2d, dropping rate = 0.5 3 × 3 conv. 128 stride = 1 with WN, lReLU(0.2) 1 × 1 conv. 128 stride = 1 with WN, lReLU(0.2) 1 × 1 conv. 128 stride = 1 with WN, lReLU(0.2) Global average Pooling 128 × 10 FC layer with WN GN, std = 0.05 Dropout2d, dropping rate = 0.2 3 × 3 conv. 96 stride = 1 with WN, lReLU(0.2) 3 × 3 conv. 96 stride = 1 with WN, lReLU(0.2) 3 × 3 conv. 96 stride = 2 with WN, lReLU(0.2) Dropout, dropping rate = 0.5 3 × 3 conv. 192 stride = 1 with WN, lReLU(0.2) 3 × 3 conv. 192 stride = 1 with WN, lReLU(0.2) 3 × 3 conv. 192 stride = 2 with WN, lReLU(0.2) Dropout, dropping rate = 0.5 3 × 3 conv. 192 stride = 1 with WN, lReLU(0.2) 1 × 1 conv. 192 stride = 1 with WN, lReLU(0.2) 1 × 1 conv. 192 stride = 1 with WN, lReLU(0.2) Global average Pooling 192 × 10 FC layer with WN E EXPERIMENTAL DETAILS FOR ADVERSARIAL TRAININGThe size of labeled data for CIFAR-10 is 50000 and we balance the number of data for each class.

Figure 9: The setting is the same with Fig. 7 unless w = 1.

Figure 10: The setting is the same with Fig. 7 unless w = 3.

2)During the training procedure, an iterative approach like traditional GAN is to alternate between k steps of training D and one step of training G. In practice, minibatch stochastic gradient descent via back propagation is used to update θ d and θ g . In other words, for each of p g , p d and p d, m sample are required for computing gradients, where m is the number of samples in a minibatch.

Table content: <table><tr><td/><td/><td/><td/><td>Algorithm 1</td></tr><tr><td colspan="5">illustrates the training procedure in detail. DSGAN suffers from the same drawbacks with traditional</td></tr><tr><td colspan="5">GAN (e.g., mode collapse, overfitting, and strong discriminator such that the generator gradient</td></tr><tr><td colspan="5">vanishes). There are literatures (Salimans et al. (2016), Arjovsky &amp; Bottou (2017), and Miyato et al.</td></tr><tr><td colspan="5">(2018)) focusing on improving the above problems, and their ideas can be combined into DSGAN.</td></tr><tr><td colspan="5">Li et al. (2017) and Reed et al. (2016) proposed the similar objective function like (2). Their goal</td></tr><tr><td colspan="5">is to learn the conditional distribution of training data. Nevertheless, we aim to learn the target</td></tr><tr><td colspan="4">distribution p t in Eq. (1), not the training data distribution.</td><td/></tr><tr><td colspan="5">Algorithm 1 The training procedure of DSGAN using minibatch stochastic gradient descent. k is</td></tr><tr><td colspan="5">the number of steps to apply to the discriminator. α is the ratio between p g and p d in the mixture</td></tr><tr><td colspan="4">distribution. We used k = 1 and α = 0.8 in experiments.</td><td/></tr><tr><td colspan="2">01. for number of training iterations do</td><td/><td/><td/></tr><tr><td>02.</td><td>for k steps do</td><td/><td/><td/></tr><tr><td>03.</td><td colspan="4">Sample minibatch of m noise samples z (1) , ..., z (m) from p g (z).</td></tr><tr><td>04. 05.</td><td>Sample minibatch of m samples x Sample minibatch of m samples x</td><td>(1) d , ..., x (1) d , ..., x</td><td>(m) d (m) d</td><td>from p d (x). from p d(x).</td></tr></table>

Comparison of semi-supervised learning between our DSGAN and other state-of-the-art results. For fair comparison, we only consider the GAN-based methods. * indicates the use of the same architecture of classifier. † indicates a larger architecture of classifier. ‡ indicates the use of data augmentation.

Table content: <table><tr><td>Methods</td><td>MNIST (# errors)</td><td colspan="2">SVHN (% of errors) (% of errors) CIFAR-10</td></tr><tr><td>CatGAN (Springenberg (2016))</td><td>191(±10)</td><td>-</td><td>19.58(±0.46)</td></tr><tr><td>TripleGAN  † (Li et al. (2017))</td><td>91(±58)</td><td colspan="2">5.77(±0.17) 16.99(±0.36)</td></tr><tr><td>FM  *  (Salimans et al. (2016))</td><td>93(±6.5)</td><td>8.11(±1.3)</td><td>18.63(±1.32)</td></tr><tr><td>badGAN  *  (Dai et al. (2017))</td><td colspan="3">79.5(±9.8) 4.25(±0.03) 14.41(±0.30)</td></tr><tr><td>CT-GAN  ‡ (Wei et al. (2018))</td><td>-</td><td>-</td><td>9.98(±0.21)</td></tr><tr><td>Our method  *</td><td colspan="3">82.7(±4.6) 5.01(±0.14) 15.08(±0.24)</td></tr></table>

; x) is to simplify the notations inside the integral. For any x, S(p g ; x) in p g (x) is nonincreasing and S(p g ; x) ≤ 0 always holds. Specifically, S(p g ; x) is decreasing along the increase of p g (x) if p d(x) > 0; S(p g ; x) attains the maximum value, zero, for any p g (x) if p d(x) = 0. Since DSGAN aims to minimize C(G) with the constraint x p g (d)dx = 1, the solution attaining the global minima must satisfy p g (x) = 0 if p d(x) = 0; otherwise, there exists another solution with smaller value of C(G). Thus, Supp (p g ) ⊆ Supp (p d).

Table content: <table><tr><td>p g ; x)dx</td><td/><td/></tr><tr><td>=</td><td>S(p g ; x)dx +</td><td>S(p g ; x)dx</td></tr><tr><td>x∈Supp(p d )-Supp(p d )</td><td>x∈Supp(p d )</td><td/></tr><tr><td>S(p g</td><td/><td/></tr></table>

Comparing the semi-supervised learning results on MNIST whether to use the sampling tricks.

Table content: <table><tr><td>Methods</td><td>MNIST (# errors)</td></tr><tr><td>Our method w/o tricks</td><td>91.0(±7.0)</td></tr><tr><td>Our method w/ tricks</td><td>82.7(±4.6)</td></tr><tr><td colspan="2">D EXPERIMENTAL DETAILS FOR SEMI-SUPERVISED LEARNING</td></tr><tr><td>D.1 HYPERPARAMETERS</td><td/></tr></table>

Hyperparameters in semi-supervised learning.

Table content: <table><tr><td colspan="4">Hyperparameters MNIST SVHN CIFAR-10</td></tr><tr><td>α</td><td>0.8</td><td>0.8</td><td>0.5</td></tr><tr><td>β</td><td>0.3</td><td>0.3</td><td>0.1</td></tr><tr><td>D.2 ARCHITECTURE</td><td/><td/><td/></tr><tr><td colspan="4">In order to fairly compare with other methods, our generators and classifiers for MNIST, SVHN,</td></tr><tr><td colspan="2">and CIFAR-10 are same as in Salimans et al. (2016) and</td><td/><td/></tr></table>

Network architectures for semi-supervised learning on MNIST. (GN: Gaussian noise) Input z ∈ R 100 from unif(0, 1) Input 28 × 28 gray image Input 28 × 28 gray image 100 × 500 FC layer with BN Softplus 500 × 500 FC layer with BN Softplus 500 × 784 FC layer with WN Sigmoid

Table content: <table><tr><td>Generator G</td><td>Discriminator D</td><td>Classifier C</td></tr></table>

Peer review (iclr2019):
 The paper presents a new Generative Adversarial Network (GAN) for learning a target distribution that is defined as the difference between two other distributions. Applications in semi-supervised learning and adversarial training are considered in the experimental evaluation and results are presented in computer vision tasks. The paper is not very well written and can be hard to follow. One very important issue for me was motivation for defining the target distribution as a difference between two other distributions. I am not familiar with this area, but reading through the introduction it was never clear to me why this is a useful scenario, in practice. Furthermore, some statements in the introduction felt quite arbitrary. For example, the authors state that PixelCNN "does not have a latent representation" in a manner that makes it sound as if that is a bad thing. If indeed it is, then why so? It would be very helpful to motivate the setting more and to provide a couple of examples of where this method would be useful, in the introduction. Also, regarding the MNIST example in the end of page 1, what is the "universal set"? This paragraph also felt a bit arbitrary and unclear. Some comments about the rest of the paper: - The theoretical results of section 3 are just stated/listed, but are not connected to algorithm 1. Please connect them to the different parts of the algorithm and state in a couple sentences what they imply for the algorithm. - Right after theorem 1, which assumption are you referring to when you say "the assumption in Theorem 1"? - The reformulation of section 3.1 is never justified. What led you to use this reformulation and why do you think it is more stable in practice? - You should mention in the caption of table 4, what quantity you are computing. Note that my evaluation for this paper is based mainly on the way it is written as, in its current state, it is hard for me to judge what is novel and what is useful, and what readers are supposed to take in by reading this paper. The main question that the paper definitely needs to answer, but does not do so currently (in my opinion) is: When is this method useful to readers? For solving which problems and under what conditions? And also, when is this method bad and should not be used? == Experiments == Section 5.1 is hard to follow and I don*t quite get how it connects to the rest. Also, in section 5.1.2 you mention that in comparison to Dai et al. (2017) your method does not need to rely on an additional density estimation network. Even if that is true, I cannot see how it is a useful remark given that the method of Dai et al. seems to always beat your method. == Style == In figure 1, no labels or legends are provided making it hard to figure out what*s going on at a glance. It would be very helpful to include labels and a legend. Equation 2 is not written correctly. The equals sign only refers to "V(G, D)" and not the min-max of that, right? Please make that explicit by first defining "V(G, D)" alone.