D-GAN: D IVERGENT GENERATIVE ADVERSARIAL APPROACH FOR POSITIVE - UNLABELED LEARNING AND COUNTER - EXAMPLES GENERATION
Anonymous authors
Paper under double-blind review
A BSTRACT
Positive Unlabeled learning consists in learning to distinguish samples of our class of interest, the positive class, from the counter-examples, the negative class, by using positive labeled and unlabeled samples during the training.
Recent approaches suggest to exploit the GANs (Goodfellow et al., 2014) abilities to address the PU learning problem by using relevant generated counter-examples.
In this paper, we propose a new GAN-based PU learning approach named Divergent-GAN (DGAN).
It consists in asking to the generator to converge towards the unlabeled samples distribution while diverging from the positive samples distribution, by incorporating a standard Positive Unlabeled learning risk inside the GAN discriminator loss function.
This results in the generator convergence towards the unlabeled counter-examples distribution without using prior knowledge, while keeping the standard adversarial GAN architecture. Experimental results show that this proposed framework achieves state of the art PU learning performances in terms of prediction, stability, and its ability to work on both simple and complex images.
I NTRODUCTION
Nowadays, the number of available labeled datasets dedicated to perception applications has considerably augmented (Russakovsky et al., 2015), (Yu et al., 2015), (Cordts et al., 2016).
However, when the learning methods are trained on these datasets in order to be applied on real application data, generalization problem is intensiﬁed.
Indeed, the prediction performances of these methods deteriorate when the real data on which they are applied do not follow the same statistical distribution as the training dataset.
Consequently, it is necessary to use a dataset specialized for the given target application.
It turns out that it can be easy to get unlabeled data from the real world such as the autonomous driving domain.
If we consider these unlabeled data as counter-examples of our classes of interest, then this enables a better adaptability of these learning methods to real world applications, without the need to label additional data.
This is known as Positive Unlabeled (PU) learning.
(Sansone et al., 2018) enumerates several learning poblems which can be adressed in this way.
The PU learning task consists in learning to distinguish positive samples, considered as the samples of our class of interest, from negative samples considered as the counter-examples of our class of interest, by using only positive and unlabeled samples during the training phase.
The interest is that unlabeled data can contain relevant counter-examples. The difﬁculty is that unlabeled data can also contain a fraction π p of unlabeled positive samples.
Several PU learning methods exist and some of them are adapted for image classiﬁcation.
They are generally classiﬁed into two categories: censoring PU learning formalized by Elkan & Noto (2008), recently improved by Northcutt et al. (2017) and case-control PU learning methods introduced by Ward et al. (2009), formalized by du Plessis et al. and then consecutively improved by Du Plessis et al.
(2015) and Kiryo et al.
However, in order to highlight GAN-based approaches such that their main goal is to prepare a PN dataset, in this article we classify PU learning approaches into two groups suggested by Kiryo et al.
(2017): One-stage and two-stage PU methods. One-stage ones such as the unbiased PU method (uPU) (Du Plessis et al., 2015) and the non-negative PU method (nnPU) (Kiryo et al., 2017) consist in training a classiﬁer directly with the PU dataset.
These methods have the advantage to need only one training of the classiﬁer. But they require dataset prior knowledge.
In practice uPU and nnPU have to be combined with an approach able to estimate prior knowledge (Jain et al., 2016).
Two-stage methods prepare during the ﬁrst stage a PN dataset by keeping only the samples considered during the training as the most negative among the unlabeled ones. Rank Pruning method (RP) (Northcutt et al., 2017) acchieves two-stage state of the art performances.
Several consecutive trainings of a binary classiﬁer are performed.
After each training, only the samples considered as the most negative are kept in order to substitute the unlabeled samples for the next training of the same classiﬁer.
RP achieves good prediction performances without prior knowledge, but with a high training computational cost and is limited in its prediction performances on complex RGB image datasets like CIFAR-10. This article addresses the PU learning problem with a new GAN-based PU learning approach: the
Divergent-GAN. Two-stage state of the art PU learning performances are achieved on simple MNIST and complex CIFAR-10 image datasets. Our contributions are the following: • a direct incorporation of a PU learning risk (equation 1) inside the standard GAN framework (Goodfellow et al., 2014).
The intuition behind is to ask the generative model to solve the PU learning problem formulated in the discriminator loss function. In this way the generator learns the distribution of the samples which are both unlabeled and not positive: the negative ones included in the unlabeled dataset; • highlight of a Batch Normalization (BN) (Ioffe & Szegedy, 2015) critical issue: A learning model which manipulates different minibatches distributions should not use BN. Alternative normalization techniques are discussed (section 3.2) and experimented (section 4.1) in the context of the proposed framework which manages positive, unlabeled and generated minibatches. From a practical point of view, several previous PU methods beneﬁts are combined in the proposed framework as listed in the table 1 of the next section.
The D-GAN code is available at 1 .
The article is structured as follow.
Section 2 presents GAN-based PU learning approaches. Sectiondescribes the proposed method. Section 4 presents the D-GAN ability to generate relevant counterexamples on the celebA dataset, its prediction performances, and learning stability for the PU learning task compared to state of the art PU learning methods on MNIST and CIFAR-10 datasets. Then, we ﬁnish by a conclusion where we also discuss perspectives.
R ELATED WORK
GAN-based PU learning approaches are a recent subcategory of two-stage PU methods. Both existing are PGAN (Chiaroni et al., 2018) and GenPU (Hou et al., 2018) . The interest of using GANs is double: GANs enable relevant data augmentation and they use high-level feature metrics to evaluate generated samples quality, thanks to the use of a discriminator. The generate samples considered as replace the unlabeled ones by learning on these latter (Chiaroni et al., 2018), or on both unlabeled and positive labeled ones as (Hou et al., 2018) during the ﬁrst step.
Both methods (Hou et al., 2018) and (Chiaroni et al., 2018) exploit GANs (Goodfellow et al., 2014) beneﬁts, but they are not suitable under the same datasets conditions and the reasonings are different:
• GenPU method is based on the original GAN convergence (Goodfellow et al., 2014), such that: π · p G P +(1 − π ) · p G N −→ p U , with p G P the distribution of positive samples generated by the generator G P , p G N the distribution of the negative samples generated by G N , and p U the distribution of real unlabeled samples.
The same idea can be expressed by using one single generator G N such that we replace the generated positive samples by the positive labeled samples initially present in a PU dataset.
Training adversarially ﬁve learning models to address standard PU learning challenge is not necessary.
In practice, GenPU seems to be the most interesting PU method on simple datasets when we own few positive labeled samples and it generates relevant counter-examples.
However, GenPU uses prior knowledge while its training optimization functions need three additional hyper-parameters.
Its architecture composed of ﬁve learning models, instead of two for the standard GANs, is more computational demanding.
• PGAN is trained to converge towards the unlabeled dataset distribution during the ﬁrst step. The PGAN exploits GANs imperfections such that the generated distribution at the adversarial equilibrium is still separable from the unlabeled samples distribution by a classiﬁer.
The PGAN does not use a PU learning risk to train its GAN part. Positive-GAN (PGAN) method presents a relatively steadier behaviour than RP and better results on the complex RGB image dataset CIFAR-10, in terms of prediction performances, and without prior knowledge for varying π P fractions.
However, it appears to be less suited for relatively simpler datasets such as MNIST.
The generated samples are all considered as negative samples by the classiﬁer. But this is true only if the generated samples distribution converges close enough towards the unlabeled samples distribution, while not matching with it.
In other words, if the GAN works as expected theoretically by Goodfellow et al.
(2014), then the PGAN will fall back into the initial PU learning problem.
However, it provides a practical framework based on standard GAN architecture and it is suited for complex datasets, contrary to previous methods.
As far as the classiﬁcation step is concerned, it only needs a standard GAN model, such as (Goodfellow et al., 2014) or (Arjovsky et al., 2017), composed of one generator and one discriminator.
The proposed approach merges the advandages of previously cited methods to address the standard
PU learning task with a different reasoning presented in the next section.
[Table]
M ETHOD
This section starts by highlighting some particularities of a standard PU learning risk (equation 1).
Then we incorporate this risk into a generic GAN framework in order to guide the generator convergence towards the distribution of the negative samples included in the unlabeled dataset.
In addition, we discuss normalization techniques in the context of this framework which manipulates three different types of minibatches: positive, unlabeled and generated ones.
3. A DIVERGENT APPROACH FROM THE DISCRIMINATOR POINT OF VIEW
The D-GAN intuition can be expressed as follow: Show me what IS unlabeled AND NOT positive.
The discriminator D addresses this riddle to the generator G .
It turns out that negative samples included in the unlabeled dataset are both unlabeled and not positive.
Consequently G adresses this riddle by learning to show the negative samples distribution to D .
The Positive Unlabeled (PU) learning problem consists in trying to distinguish positive samples from negative samples by using a PU dataset.
We start by detailing the expected PU functionality that we want to incorporate into the GAN discriminator.
Let X ∈ R
m be the input random variable and y ∈ [0 , 1] its associated label.
X can be a positive x P , negative x N or unlabeled x U sample which respectively follow the distributions p P = p ( X | y = 0) , p N = p ( X | y = 1) and p U
=
(1 − π P ) · p N + π P · p P , with π P ∈ (0 , 1) the unknown prior which represents the fraction of unlabeled positive samples included in the unlabeled dataset.
Let D : R m → (0 , 1) be the decision function which is then considered as the discriminator network of the proposed framework.
We have l (ˆ y, y ) such that l : (0 , 1) × [0 , 1] → R is the arbitrary cost function with the predicted output ˆ y of D for a given sample X and the corresponding label y as inputs.
D is trained with a PU risk R PU to predict the label value y = 1 for the unlabeled samples, and the label value y = 0 for the positive labeled ones.
Given the composition of the unlabeled dataset, we develop R PU ( D ) such that:
[Equation]
Negative samples are associated exclusively to the label value y = 1 for any π P value. Labeled and unlabeled positive {+samples follow the same distribution p P . So p P is associated to both contradictory labels y = 
 and y+}
= 1 .
Thus if D does not overﬁt labeled and unlabeled positive samples, let δ ∈ (0 , 1) be the label associated to all positive samples such that we propose the Positive Negative risk R PN
( D, δ ) =
R PU ( D ) . δ represents the intermediate label value equivalent of associating, in the risk R PU , both contradictory labels y = {+
+}
and y
= 1 respectively to labeled and unlabeled positive samples. We develop the risk R PN as follow: [Equation]
As illustrated in ﬁgure 1, in practice we can train D with the risk R PU to output a value
ˆ y close to δ when the input is a positive sample x P , and close to 1 when the input is a negative sample x N .
We can observe that the global minimum of the approximated risk ˆ
R PN
( D, δ ) corresponds to the global maximum of the distribution predicted by D in output for positive unlabeled samples.
Moreover, the distribution of the negative samples is centered on the label associated to the unlabeled samples during the training, which coincides also with the equation 2 formulation.
This illustrates experimentally the equality R PN ( D, δ ) = R PU ( D ) .
Set of positive samples (a)
Set of unlabeled samples with =0.
(b) 0.
1.0 Discriminator prediction output for the p
y
( y U ) curve, and value for R PU 0.
pd f a n d l o ss e rr o r
Link between and the discriminator PU training p
y
( y U )
R PU (c) 0.
1.0 Discriminator prediction output for the p
y
( y U ) curve, and value for R PU 0.
pd f a n d l o ss e rr o r
Link between and the discriminator PU training p
y
( y U )
R PU (d) and negative labeled samples, once D is trained with R PU risk. (d) shows the same curves as in (c) but by giving in input a concatenation of an unlabeled minibatch with a positive labeled minibatch. Unlabeled and labeled positive samples provide a uniﬁed prediction output distribution.
Next, let us consider the decision function D as the discriminator of a generative adversarial network.
We propose to incorporate the risk R PU into the value function of the original GAN model (Goodfellow et al., 2014) by adding the loss function term log [1 − D ( x P )] . Consequently, the loss function L D of D is:
[Equation]
with z the input random vector of the generative model G such that G ( z ) is a generated sample.
z can follow a uniform or normal distribution. The equation 3 is justiﬁed intuitively as follow. Instead of asking to G to converge towards the distribution p U , we want that G learns to converge towards the distribution p N of negative samples. Thus we propose to replace the standard GAN task show me what is unlabeled by the task show me what is both unlabeled and not positive .
This results in addressing directly the risk R PU to D such that L D is as follow:
[Equation]
{+with y = 
+}
the label associated to the samples G ( z ) generated by G .
Let the cost function l used in the proposed D-GAN framework be the binary cross-entropy H such that l = [-
-]− H . So: [Equation] We recall that the binary cross-entropy H is deﬁned as below:
[Equation] with y the label associated to the D input samples X .
Thus H ( D ( X ) , y = 1) = − log
( D ( X )) and H ( D ( X ) , y = 0) = − log (1 − D ( X )) . Consequently, L D can be developed as in the equation 3.
This shows the incorporation of R PU risk (equation 1) inside the D-GAN discriminator loss function (equation 3). On the other hand, the role of G during the adversarial training is to generate samples considered by D as 1 . So as suggested by Goodfellow et al. (2014) in practice, the training loss function L G of G is such that L G =
E
z ∼ p z ( z ) [ log ( D ( G ( z )))] . Only negative samples are considered as 1 by D thanks to the R PU risk, as shown previously. This justiﬁes intuitively the G convergence towards the negative samples distribution p N .
The corresponding implementation algorithm 1 of this ﬁrst stage is presented in annexe 6.
Once the D-GAN training is ﬁnished, we can start the second step which consists in using a classiﬁer C to distinguish fake generated samples x FN =
G ( z ) , which are ideally equivalent to the real negative samples, from real positive labeled samples as illustrated in ﬁgure 2.
The divergent GAN behaviour is only possible if the discriminator D is able to distinguish positive and negative samples.
Nonetheless, in the worst-case scenario where the discriminator is not able to sufﬁciently encode the complexity of the boundary between positive and negative samples, the D-GAN will then converge towards the behaviour of the PGAN, which seems to be the best solution in this situation.
This is observed experimentally in the table 3 on the CIFAR-10 dataset.
3. W ITHOUT DISCRIMINATOR BATCH NORMALIZATION
Nowadays, Batch Normalization (BN) (Ioffe & Szegedy, 2015), is considered as a relevant regularization technique commonly used in deep neural networks architectures.
Its utility for GANs training has been highlighted by (Radford et al., 2015) for the DCGAN architecture in order to stabilize the adversarial training.
Other variants like the Wasserstein-GAN (Arjovsky et al., 2017) or the LossSensitive GAN (Qi, 2017) conﬁrmed its interest.
Indeed, as developped in (Ioffe & Szegedy, 2015),
BN addresses issues like vanishing, exploding gradient problems, and the risk of getting stuck in a poor local minima.
BN suggests to reduce the internal covariate shift problem of the learning model.
In this way, a higher learning rate can be used and it can improve signiﬁcantly the training speed.
But BN regularizes the model, such that a training sample from a given minibatch is considered in conjunction with other examples of this minibatch.
This is the consequence of estimating the mean and variance normalization parameters one time per minibatch, and then applying them on each minibatch sample.
It turns out that it does not allow to link labeled positive samples to the unlabeled positive samples and to produce a gap between positive and negative samples, when positive samples x P and unlabeled samples x U are not in the same training minibatch.
To counter this problem, we could imagine to apply BN on a uniﬁed minibatch which contains a fraction of each distribution x P , x U and x F .
But the BN effect is inﬂuenced a lot by the content of the minibatch on which it is applied.
So the fraction π P of positive samples included in x U will affect the batchnorm effect.
As illustrated experimentally in ﬁgure 5, with BN, we observe that the classiﬁer only learns to distinguish minibatches of positive samples from minibatches of unlabeled samples.
On the contrary, without the use of BN, both distributions appear in the unlabeled minibatch from the discriminator output point of view.
In other words, the negative samples can be distinguished from the positive samples without BN.
However, BN beneﬁts in a more traditional training are not negligible, so we suggest to use another type of normalization which could replace BN role in a PU learning task.
Layer Normalization (LN) (Ba et al., 2016), is a frequently used technique with sequential networks as it can be applied for each sequential sample separately.
Indeed, with LN the normalization for a given sample is computed on its resulting output feature map layers and the mean and variance are computed separately for each sample of a minibatch.
For this reason, we propose to apply LNs instead of BNs inside our discriminative model D .
As we can observe in ﬁgure 5, it allows the discriminator to keep the behaviour that interests us which is the ability to distinguish unlabeled negative samples to unlabeled positive samples.
Additional qualitative experiments in annexe section 6 show that the LN can substitute BN inside the discriminative model structure to generate acceptable images.
The next section presents experimental results which demonstrate the good performance of the proposed approach.
E XPERIMENTAL R ESULTS
This section demonstrates experimentally the D-GAN abilities on simple and complex datasets:
We present qualitative results for the counter-example generation, the discriminator normalization effect, and quantitative results for the PU learning image classiﬁcation task.
Experimental settings are as follow. Minibatches of labeled positive samples, unlabeled samples, and generated samples have the same size. The convolutional classiﬁer C used 2 in the subsection 4. is described in the PGAN article (Chiaroni et al., 2018) experiments.
C is always trained duringepochs.
We use the DCGAN (Radford et al., 2015) architecture for all the comparative experiments except for celebA and CIFAR-10 datasets where we respectively adapt the LS-GAN (Qi, 2017) and WGAN-GP (Gulrajani et al., 2017) variants to the D-GAN framework.
The D-GAN uses LN instead of BN concerning D .
Concerning the PU dataset initialization from a standard PN dataset, ρ is the fraction of the positive labeled samples of the initial PN dataset that we unlabel such that they are included into the unlabeled dataset. π P is the fraction that represents these unlabeled positive samples among the unlabeled dataset.
From a qualitative point of view, and on the contrary to the PGAN model, the D-GAN generates samples which are only similar to the counter-examples for diverse data types, as illustrated in ﬁgure 4 for a 2D point cloud dataset and in ﬁgure 3 for a real images dataset.
In order to enable the proposed approach reproducibility, a D-GAN implemented version corresponding to the ﬁgure 3 results is available 3 and is applied on the LSGAN model (Qi, 2017).
Our code includes also the method implementation, proposed by Chiaroni et al
(2018), to establish a PU training from a fully labeled
[Figure]
4. N ORMALIZATION EFFECTS ON THE D ISCRIMINATOR
We compare in the ﬁgure 5 the ability of D to distinguish the positive from the negative samples distribution when it is trained on a PU dataset with BN, LN layers or without any normalization technique.
Smaller is surface intersection between the discriminator output histogram distribution predictions on positive and negative samples included in the unlabeled dataset, better it is.
[Figure]
(ﬁgure 5(c)) enables it for the reasons speciﬁed in the subsection 3.2. Thus with LN in D , G can converge exclusively towards the unlabeled negative samples distribution p N , as expected.
4. D IVERGENT -GAN FOR P OSITIVE U NLABELED LEARNING
We compare the D-GAN accuracy prediction to GenPU on the table 2 for the simple One vs. One task with few positive labeled samples. The GenPU method presents better results for fewer labeled positive samples.
The D-GAN presents Accuracy performances better than the GenPU method for one hundred of positive labeled samples after 40 epochs training iterations for the D-GAN counterexamples generation step.
Moreover
[Table] it is interesting to observe that the proposed approach globally slightly outperforms nnPU and uPU methods without using prior knowledge. [Table] Concerning the relatively more challenging One vs. Rest task
[Table]
the D-GAN globally outperforms the PGAN and RP methods in terms of F1-Score predictions performances on the table 3 both on MNIST and CIFAR-10 datasets. Moreover, the D-GAN demonstrates a steadier behaviour than PGAN against the overﬁtting problem as illustrated in ﬁgure 6(b). performances in a One vs. Rest mode on MNIST and CIFAR-10 with respectively 35 Divergent-GAN epochs on MNIST and
Divergent-WGAN-GP epochs on CIFAR-10. PGAN and RP results come from (Chiaroni et al., 2018). PNGAN represents a PN (PU when π P = 0 ) training such that the negative samples are replaced by WGAN generated ones. This highlights the GANs data augmentation effect on complex datasets like CIFAR-10.
[Figure]
function of the GAN epochs ( stage).
8 vs. Rest
MNIST task, with ρ = 5 and π p = GAN and PGAN are trained without normalization layers b) D-GAN and PGAN are respectively trained with LN and BN inside the discriminator. In ﬁgure 6(a), without BN or LN, the D-GAN method gets a faster and better Accuracy prediction performance than the PGAN when both are trained under the same conditions.
In ﬁgure 6(b), the D-GAN with LN follows the learning training rythm of the PGAN with BN. The D-GAN also has a steadier behaviour once the Accuracy progression is ﬁnished: it reduces the PGAN ﬁrst step overﬁtting problem.
C ONCLUSION
To conclude, we have incorporated into the discriminator D loss function a standard PU risk to address the PU learning challenge. In this way, the proposed model generates relevant counterexamples from a PU dataset without prior knowledge. It achieves two-stage state of the art PU learning performances by addressing previous GAN-based PU methods issues: -It reduces the overﬁtting PGAN ﬁrst stage problem.
-It addresses the GenPU mode collapse issue by keeping the practical standard GANs architecture such that it is easily adaptable to conventional GANs variants.
A side contribution of this article is to identify a discriminator batch-normalization critical issue appearing when the discriminator manipulates multiple minibatches distributions.
Nonetheless, we strongly believe that its stability and prediction performances still have the potential to be improved by taking the best of the representation learning and PU learning domains.
For example, recent promising GAN training approaches (Karras et al., 2018) not mandatorily using BN should be suitable to extend the proposed approache for large-scaled image datasets.
In addition, premices of
Noisy labeled learning results presented in annexe section 6.2 could extend the proposed approach to a uniﬁed generative framework which could output relevant generated labeled samples even from a weak, noisy labeled or incremental dataset with varying fractions of unlabeled positive samples.
A NNEXE
The algorithm 1 is the implementation of the proposed approach. It includes the option for the noisy labeled learning task.
The table 4 details for each class the average F1-Score results presented in Table 3. The ﬁgure 9 presents
6. N EEDING PRIOR KNOWLEDGE MAY NOT BE SUITABLE FOR INCREMENTAL APPLICATIONS
In the context of real learning applications like incremental learning (Craye et al., 2018), (Hadsell et al., 2009), the fraction π P of unlabeled positive samples can vary continuously at each new incremental unlabeled minibatch of samples to process.
However, existing PU learning methods using prior knowledge, like uPU and nnPU implicitly do the assumption that the prior knowledge holds the same during all the training.
A PU learning method which does not need prior knowledge as the
D-GAN proposed approach could be extended to such real applications. 6. A N EXTENSION TO NOISY LABELED LEARNING AND FEW LABELED DATA RESSOURCES If a generative model in an adversarial training only converges towards the subdivision of a distribution which is considered as the closest to a given label from the discriminator point of view, then we can extend the proposed approach to the noisy labeled learning task. More precisely, the D-GAN architecture can be modiﬁed to include a generative model G P trained to generate fake positive samples.
This is achieved by adding the loss function term logD ( G P ( z )) in the minimax value function.
Furthermore, this extension also allows to manage problems where we only have few labeled positive samples. Indeed, the positive generator G P will converge towards the distribution considered as the most positive, the closest to zero in our case, which includes also the unlabeled positive samples perceived by the discriminator at the same location that the labeled ones as illustrated in ﬁgure 1 (d). We argue that it is indirectly the same phenomenon that allows the GenPU (Hou et al., 2018) to get relatively good accuracy predictions with few labeled positive samples. Because the discriminator task becomes more difﬁcult, the weight α is applied on the unlabeled loss function log ( D ( x U )) and on the positive one log (1 − D ( x P )) .
It ensures the guidelines that both generators G and G P have to follow.
The boolean variable noisyLabelsMode of the algorithm 1 is set to True to activate this functionality.
The ﬁgure 7 shows the results obtained on clusters of 2D point clouds with the D-GAN, when its noisy labeled mode is activated.
[Figure] [Figure]
Initialize noisyLabelsMode = false .
Initialize α = 2 . for number of training iterations do for k steps do Sample minibatch of m noise samples { z (1) , ..., z ( m ) } from noise prior p G ( z ) . Sample minibatch of m unlabeled examples { x (1) U , ..., x ( m ) U } from data distribution p U .
Sample minibatch of m positive labeled examples { x (1) P , ..., x ( m ) P } from data distribution p P . Update the discriminator D by descending its stochastic gradient: if noisyLabelsMode is false then ∇ θ D 1 m m (cid:88) i = logD ( x ( i ) U ) + log [1 − D ( G ( z ( i ) ))]
+ log [1 − D ( x ( i ) p )] else ∇ θ D 1 m m (cid:88) i =
α · (cid:0) logD ( x ( i ) U ) + log [1 − D ( x ( i ) p )] (cid:1)
+ log [1 − D ( G ( z ( i ) ))]
+ logD ( G P
( z ( i ) )) end if end for
Sample minibatch of m noise samples { z (1) , ..., z ( m ) } from noise prior p G ( z ) .
Update the generator G by descending its stochastic gradient: ∇ θ G 1 m m (cid:88) i = log [ D ( G ( z ( i ) ))] if noisyLabelsMode is true then Update the generator G P by descending its stochastic gradient: ∇ θ GP 1 m m (cid:88) i = log [1 − D ( G P ( z ( i ) )) ] end if end for
The gradient-based updates can use any standard gradient-based learning rule.
We use Adam in our experiments.
on MNIST and CIFAR-10 in a
One vs. Rest mode with 20 classiﬁer epochs
[Table] [Table] [Table] [Table]
[Table]
[Table]
[Table]
[Table]
[Table] [Table] [Table] [Table]
[Table]
[Table]
[Table]
[Table]
[Table] [Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table] [Table] [Table]
[Table] [Table]
[Table] [Table]
AVG
MNIST [Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table]
[Table]
[Table]
[Table]
Plane 0. 0.731 0.718 Auto 0. 0.734 0.783 Bird 0.
[Table]
Dog
[Table]
The LSGAN-GP code is available at: https : //github.com/maple − research − lab/lsgan − gp − alt .