Learning from Positive and Unlabeled Data  with Adversarial Training
Authors: Wenpeng Hu
Ran Le
Bing Liu
Feng Ji
Haiqing Chen
Dongyan Zhao
Jinwen Ma
Rui Yan
Venue:  (2020)
Keywords: Positive and Unlabeled learning

Abstract: Positive-unlabeled (PU) learning learns a binary classifier using only positive and unlabeled examples without labeled negative examples. This paper shows that the GAN (Generative Adversarial Networks) style of adversarial training is quite suitable for PU learning. GAN learns a generator to generate data (e.g., images) to fool a discriminator which tries to determine whether the generated data belong to a (positive) training class. PU learning is similar and can be naturally casted as trying to identify (not generate) likely positive data from the unlabeled set also to fool a discriminator that determines whether the identified likely positive data from the unlabeled set (U) are indeed positive (P). A direct adaptation of GAN for PU learning does not produce a strong classifier. This paper proposes a more effective method called Predictive Adversarial Networks (PAN) using a new objective function based on KL-divergence, which performs much better.~Empirical evaluation using both image and text data shows the effectiveness of PAN.  

Under review as a conference paper at ICLR 2020
LEARNING FROM POSITIVE AND UNLABELED DATA WITH ADVERSARIAL TRAINING
Anonymous authors Paper under double-blind review
ABSTRACT
Positive-umlabeled (PU) learning learns a binary classifier using only positive and unlabeled examples without labeled negative examples. This paper shows that the GAN (Generative Adversarial Networks) style of adversarial training is quite suitable for PU learning. GAN learns a generator to generate data (e.g.. images) to fool a discriminator which tries to determine whether the generated data belong to a (positive) training class. PU learning is similar and can be naturally casted trying to identify (not generate) likely positive data from the unlabeled set also to fool a discriminator that determines whether the identified likely positive data from the unlabeled set (l) are indeed positive (P). A direct adaptation of GAN for PU learning does not produce a strong classifier. This paper proposes a more effective method called Predictive Adversarial Networks (PAN) using a new objective function based on KL-divergence, which performs much better. Empirical evaluation using both image and text data shows the effectiveness of PAN.
INTRODUCTION
Rositive-unlabeled (PU) learning leams a binary classifier from only Positive (P) and Unlabeled (4)
examples with no labeled negative examples (Denis) 1998; [iu et al., 2002: Elkan & Noto (2088]
du Plessis et al.] 2014| Bekker & Davis 2018b). It has many applications in text analysis, bio-
medicine, spam detection, recommendation, remote sensing, matrix and knowledge base completion,
graph learning, etc (Li & Liu 2003; Yu et al. 2004 Fusilier et al. 2015 Scott& & Blanchard [2099] Li
iet al. 2010 Cerulo et al. 2010 Calvo et al. [007]] Ren et al. 2015 Hsich et 2015) Wu et al.
2017 Zupane & Davis 2018). This paper proposes an adversarial PU learning method inspired
by GAN (generative adversary networks) (Goodfellow et al. 2014). as found that the GAN framework naturally suits PU learning. GAN aims to generate data of a particular training class, which is like the positive class P in PU learning. GAN works by generating likely positive data using a generator G(-) to fool a discriminator DC-), which determines whether the generated data indeed belong to the training (positive) class. For PU learning, this is like choosing likely positive instances from the unlabeled set l/ (achieved by classifier) also to fool a discriminator D(-). Thus, we can simply replace GAN's generator with a classifier C(-) to produce a PU learner Note that the classifier is also a discriminator, but we use the term classifier here to distinguish it from the original discriminator of GAN. Note also both C(-) and ((.) are neural networks?
This paper first proposes a direct adaptation of GAN, called a- a-GAN (adapted GAN), for PU learning following the above idea (see Section 3. However, this simple method unable to produce state- of-the-art results. It led us to propose new objective function based on Kullback-Leiller (KL) divergence. Our final proposed method is called PAN (Predictive Adversary Nenworks) due to the use of the classifier to replace the generator in GAN along with new adversarial training method. On the new objective function, it is clearly desirable to consider the overall performance of C(-) on the entire unlabeled set. We use distance metric to measure whether C(+) can produce similar
PU learning is analogous to GAN and vice versa. That is, if we could put all the data (e.g., images) that be generated by GAN's generator in set, the should be regarded as unlabeled as it contains both good (positive) and bad (negative) images. Then what the generator does is like selecting good images fool the discriminator, which is exactly what our classifier does with a set of given unlabeled data.
"Their exact architectures are given in the experiment section (see Training Details in Section 5.1 since C(-) and (-)) have different architectures for text and image.

---

Under review as a conference paper at ICLR 2020
predictions to those of D-)) for all the examples in 4. If C(-) gives similar predictions, it means that the examples obtaining high probabilities from C(.) also get high probabilities from D(-), achieving the goal of fooling D()), which will give good final PU classifier. We employ KL-divergence as the distance metric to measure the difference between C(.) and ()))
One important advantage of PAN is that it does not need the input of class prior probability, which many state-of-the of-the-art systems need (see Section 2). In practice, the class prior is unknown (there are some existing methods to estimate it, see below). [Hou et al. (2018) and [Chirroni et al. (2018) have employed GAN to generate positive and/or negative data and then use separate learner (e.g.. CNN) to learn the final PU classifier using the generated data, they are not adaptations of GAN like PAN and their generators generate only images. PAN can be applied to any data as it has no generator. We evaluate PAN using both text and image classification datasets and show outperforms start-ofthhe-art PU learning baselines even when we give them the perfect class prior probabilities. We also show that when the class prior probability estimation is off, the results can be quite poor.
2 RELATED WORK
Research in PU learning started in early 2000s (Denis) 1998 Liu et al. 2002 Lee & Liu 2003
Yu et al. 2004 Liu et al., 2003 Elkan & Noto 2008). Denis (1988) proposed it under statistical
query framework. Liu et al. (2002) studied sample complexity. Elkan & Noto (2008) showed that if ranking is the goal of P0 learning rather than classification, then PU learning is equivalent to the
conventional binary learning. Due to numerous applications, there has been a recent surge of interest
in PU learning (du Plessis et al.] [2014] Mordelet & Vert [014]] du Plessis et al., 20155 [Sehhdis &
Brown 2015 Claesena et 2015 | et al. [016]] Niu et al. [0 Yi al. [001]] [iu et al.
2017 Jia et al.
2017 Northhutt et al. 2011 Kiryo et al. Sakai et al. 2017
Hou el al. 2018
Xu et al. 2017
Gong et al.
2018 Bekker Davis 201 Saj Chiaroni et al.) 2018 Hsieh et al. 2018
Bao et al. 2018 Shi et al.
2018 Sansone et al. 2018 8 Kato et al. 2019).
Early approaches for PU learning use 2-steps. Step finds some highly probable negative examples
from the unlabeled set. Step 2 uses the positive set, highly probable negative set, and the remaining
unlabeled set to build a classifier (Liu et al. 2002, Yu al. 2004 Li& Liu 2003). Liu al. (2003);
Shi et al. (2018) also regarded the unlabeled data to have noisy labels. Several techniques applied
training data re- weighting via regularization as well (Lee & Liu 2003 Elkan & Noto 2008 Work
in (du Plessis et al.] 2014, 2015) Kiryo et al. (017)) avoided parameter tuning by using unbiased risk estimators. Kato et al. (2019) deals with sample selection bias.
None of these works explores the idea of adversarial learning we do in PAN. Hou et al. (2018) used GAN to generate positive and negative examples and then used them to build the final classifier employing separate classification model. Chiaroni et al. (018)) proposed method to GAN to generate negative training examples. Both papers are for image classification. Generating text and other forms of data using GAN is relatively more challenging. PAN does not generate data. It replaces the generator with a classifier to directly train the required PU classifier in the adversarial learning framework using positive and unlabeled data. Our formulation and objective function are also quite different from those of GAN. Moreover, Xu et al. (017) and Gong et al. 2018) used traditional margin- based methods.
Our a- I-GAN method is also related to the studies of weighted adversarial nets (WAN) (Chen et al. 2018 Zhang et al.) (0188) as a-GAN also weights the discriminator, but the proposed PAN differs significanlly from WAN. That is because although WAN weights the discriminator but the adversarial training procedure is the same as the original GAN (similar to our a-GAN). Our a-GAN is also similar to the method DAN recently posted on arXiv (Liu et al. 2019) as we can see from their Eq. 5 and our Eq. 2. The training methods are slightly different. PAN takes a very different approach as we will see in the next two sections.
Other related works include leveraging biased negative examples (Hsich et al. 2018 Sakai et al. 2017), studying the random assumption of PU learning (Bekker & Davis 2018a), multi-in iinttiinennneeeeee PU learning (Bao et al. 2218)), scalable PU learning (Sansone et al. 2018 etc. More details can be found in the survey (Bekker & Davis 20186).
A weakness with these recent state-of -fthe- systems is that they need the class prior probabil-
ity ddu Plessis et al. 2015 Kiryo al. 2017 Hou al. 2018 Xu et al. 2017 Chiaroni et al. 2018
2

---

Under review as a conference paper at ICLR 2020
A:GAN
PAN
Yes
No
Metri (tem Eq. (3))
From (score for ingul is posilive)
En (3)
D
Give different score with C to recognize "aae instancee"
Give consistant with
Sample and try gel
highest from
D(
CG1
x
+ Positive Set: x² ~ (2(xx") Unlabeled Set x" ~ PM
Postive Set: Set: ~ (x(x))
Uniabeled Set x" ~ (x("))
Figure I: An Illustration of the objective functions of a-GAN (left) and PAN (right) as a comparison of the two models.
Kato et al. 2019). which is hard for the user to provide. Although there are methods that try to estimate the class prior (Menon et al. 2015 Ramaswamy et al. 2016, Jain et al. 2016: du Plessis
ct al. 2017 7 Gong al. 2019), we will show that if the estimate is off, the results can be quite poor.
3 BACKGROUND
GAN is adversarial learner that learns generator for generating data such images. It composes of two networks, a generator and a discriminator. The generator generates new data instances, and the discriminator evaluates them for authenticity, i.e. deciding whether each generated data instance it reviews belongs to the actual training dataset or not. Through an iterative and adversarial process, the generator can generate new data instances that the discriminator has hard time distinguish from the real training data. GAN is formulated as a minmax game as follows:
min max V(D,G) Panta (x)) [log D(x)) E.z P.)) (og((1 D(G(z)))
where G(-) is the generator that aims to generate data that can approximate real data to fool the discriminator D(-), while D(-) tries its best to discriminate the generated data from real/training data. Paota the data generating distribution of the real data, and P3 the data generating distribution of the generator. Through an adversarial training, we expect to learn good generator.
3.1 DIRECT ADAPTATION OF GAN FOR PU LEARNING
We now present the direct adaptation of GAN (a-GAN) for PU learning. The real training data in GAN is our labeled positive data P. As illustrated in Figure 1 the discriminator D(-) in a-GAN still plays the same role as that in GAN, but the generator in a- a-GAN is replaced with another discriminator, which we call the classifier (or predictor) C(-). The goal of C(-) is to identify likely positives from the unlabeled set 2/ to give to the discriminator for it to decide whether these are real positive data. The following equation shows this approach:
min max V(D,C) Ber Dog D(x")] C D
+ Exx tu Cbe [log(1 - D(x*))
where denotes the example judged as a likely positive example from 2/ by C(.). PP and PM are the data generating distributions of the known positive data and the unlabeled data in PU learning, respectively. The known positive examples are randomly selected from the positive population. The hidden positives in the unlabeled set is also a random sample of the same positive population.
Duc to the discreteness of the last term in the equation, use the Policy Gradient method (Sutton ctt al. 2000) from reinforcement learning to train it, where the last term is regarded as the reward for optimizing C(-). We call this adapted version of the system a a-GAN (adapted GAN).
However, since focuses on the positive data only, its learning is not balanced, which cause some confusion with the separation of positive and negative data. Next, we present the final proposed method PAN, which emphasizes both positive and negative and is able produce better separation for them.
a-GAN does reasonably well (see Section 5.2
3

---

Under review as a conference paper at ICLR 2020
4 PROPOSED PAN
PAN adopts the same adversarial learning idea to build a PU classifier C(-). However, as the right part of Figure 1 shows, instead of using D(-) directly discriminate the known positive data and the selected positive data by C()), we propose to use the adversarial learning idea on the probability distributions of (()) and C(-) on each example (or instance). Specifically, in the part surrounded by the red dash-lined box in Figure I D(-) and C(-) produce a score for each input example x from the unlabeled set in parallel with different optimization objectives. D(-) tries to give x" the opposite prediction score to that of C-) in order to identify it as a "fake" example; C(-) tries to give x a similar score to that of (--) to fool (.)) The adversarial learning is performed through a distance metric, (-) tries to enlarge the distance with (.)) but C(-) tries to shrink the distance, which is applied to each example in the unlabeled set (no sampling is used). We choose KL--iiergence as the metric, which minimizes the information loss between the two probability distributions it has been shown to be able to learn and suit complex distributions (Kingma & Welling, 2013] Goodfellow et al. 2014). The green links in Figure show the procedure of optimizing the known positive data. Note that the unlabeled data are regarded as the negative data in PAN to endow ((-) the ability to recognizing negative samples some extent.
4.1 PREDICTIVE ADVERSARY NETWORKS (PAN)
Unlike GAN, which only generates positive examples that are hard to distinguish by the discriminator D(-), we also want the remaining unlabeled examples to be easily distinguishable (as possible negatives) by the discriminator. To this end, C(-) tries to separate positive and negative examples in the unlabeled data with large margin. That is, C(.) not only gives high probabilities to examples that (D-) has difficulty to distinguish (meaning D(-) also gives high probabilities to those examples) but also low probabilities to examples that are easy to distinguish by D(-) (meaning (-)) also gives low probabilities to those examples because of the easy separation). Note, when say C(-) or D(-) gives high/low probability, we mean the probability of being positive. We propose to achieve our goal by controlling the distance (similarity) between the predictions of C(-) and D(-) the unlabeled set. We use the sum of KL-divergences on the predictions of all examples (or instances) in u as the distance. In detail, PAN assumes the output of D(-) (respectively, C(-) on ith instance as discrete distribution over binary outcomes (or classes) of positive and negative, For example, if (-)) (likewise, C(-)) gives an instance the probability of 0.3. It means that for the positive outcome or class, the probability is 0.3, and for the negative outcome, the probability is 0.7. We use D; and C22 to denote the two distributions and KL-divergence employed to measure their distance. Superscripts pa and denote the corresponding datasets. PAN's objective is defined as follows:
min max V(D,C)= = Σ KL(P D
KL(DE Σ KL(D:ICO))
(3)
III
where ppn is the probability distribution of positive and unlabeled of the ith instance (we basically treat unlabeled as negative, which is an issue to be addressed shortly) in the given PU data X (including both positive X" and unlabeled X" data), and and no are the total numbers of training examples in X and X" respectively. C7 denotes the opposite distribution of CH, i... 1-C (with a slight abuse of notation). Å is a hyper-parameter for balancing the distances.
We marked three terms in Eq. 3 The adversarial learning of Eq. 3 works as follows: The first term marked by is to minimize the sum of the divergences between D. and p." (notice the minus sign in front). It aims to achieve the goal of helping D(-) recognize positive instances (it is necessary especially at the beginning of training). When optimizing C(.). the term marked by II minimizes the sum of the Kl-diieegeeness from C. to D., indicating that C(-) tries to give the same probability to
the input x; as D(-). In this case, the instances/examples getting high probabilities (chosen by C(-) as positive) can also get high probabilities from D(.). This achieves the goal of fooling D(-) by C(-) When optimizing D(-). the term marked II maximizes the sum of the KL-divergences between D.
and CH,, meaning that D(-) tries to give low probabilities to the instances that get high probabilities
from C(-) in order to detect 'fake' positive examples, and vice versa.
Using the first two terms can already perform the function of PAN. An advantage of PAN is that can consider and optimize both positive and negative examples in the unlabeled set. However,
4

---

Under review as a conference paper at ICLR 2020
show that the term marked II produces asymmetric gradient for positive and negative examples for both D(-) and C() in Appendix[A AA] That means the term marked II can cause unbalanced training between positive and negative examples and lead to high precision and low recall. To this end, propose the term marked III which can eliminate the concern (see Appendix A Ablation study also shows the effectiveness of term III in Appendix [D.3] With all three terms, we build adversarial learning approach for PU learning through the minimizing and maximizing operations mentioned above, i.c., a minmax game between D-)) and C(-).
4.2 SIMPLIFICATION OF EQUATION 3
To facilitate the optimization of the objective function in Eq we use D to denote D(x²") and C' to denote (x²²)) and simplity Eq. 3 to (see Appendix for derivations::
min max V(D,C)
PF Dog D(x*)) ]] Een. Pe [(x"))]
IV: IV- ((e"))
(4)
+ A Exxm [log[[ - C(x") }) log(C(x"))) 1)]
where PP denotes the distribution of the positive data. As we marked in Eq. 4 term IV is the cross entropy loss between D(xP*) and the ground-truth label distribution ppu of the PU data, denoted by (PP D)x"")). About the term marked VI, elaborate with the following two points:
(1). Term VI can be viewed approximately as a policy gradient reinforcement learning algorithm for training C()) but with sampling operation, if regard D(x*) the reward and term V as the policy. Clearly, (.) outputs a high 'reward' that exceeds 0.5, meaning that D--) judges the current input positive instance with high probability, Eq. 4 will maximize the probability of C(-) over the current input to fit the distribution of D-)). However, if D(-) outputs a low 'reward' below 0.5, minimizing Eq. 4 is equivalent to minimizing the probability of C(-) over the current input. As consequence, the distribution of C(-) is made closer to (--).
(2). The term marked with V in Eq. is comparison game between the likelihood log(C(x")) of choosing an example or the likelihood log(1 C(x") of not choosing an example x". If the choosing probability is greater than the not choosing probability, the value of term V is less than O. Then to optimize (-.,, maximizing Eq. 4 equivalent to minimizing the probability of D(x*). On the contrary, maximizing Eq. iis equivalent to maximizing the probability of D(x"). Clearly, this is an adversarial learning method: for the case that term V is less than 0, D(-) tries to distinguish examples selected by C(-) (to give low probabilities to these examples). For the case that term VII greater than 0, D(-) tries to give high probabilities to examples not selected by C(.), which helps the system move away from local training optimal.
Analysis of the Learned Classifier (C(-)): Although the proposed PAN is quite different from original GAN, its running follows the adversarial procedure. The behaviors of optimal C(-) and D(-) are that C(-) gives the same prediction as D(-) while D(-) cannot move away from C(-), which means D(.) also give positive (or negative) scores to examples that get positive (or negative) scores from C(-) The reason that D(-) cannot move away from C(-) is because moving away from C(-) will let D make errors on known positive data. We give theoretical analysis of the optimal decision surface learned by PAN in Appendix C Due the complexity of PAN, we cannot give the precise optimal decision surface expression, but we show its properties in the Appendix
4.3 TRAINING ALGORITHM OF PAN
Algorithm gives the training algorithm of PAN using stochastic gradient descent for conciseness. Note, our method is not limited to using stochastic gradient descent. In this work, we use Adam for optimization. The algorithm alternately trains the discriminator D(-) and the classifier C(-). In each step or iteration. the lines between 5 to 10 (not including 10) are for training ((-) and the lines after 10 are for training C(.). The details of the algorithm are self-explanatory.
Possitive (or negative) score means the score is greater (or lower) than 0.5.
5

---

end for Under review as a conference paper at ICLR 2020
Algorithm 1 PAN training by minibatch stochastic gradient descent method.
Input: given positive training data X"; given unlabeled training data X"; Initial: Randomly initialize D(-) and C()):
for number of training steps do
// Training D()) &c steps, we set & = 1.
5:
for k steps do
Sample a mini-batch of m positive examples f.
.,............... x from XP; Sample a mini-batch of m unlabeled examples { xm from X";
Update (() by ascending its stochastic gradient:
V 0
log D(x)) + log(1 D)xx)) + A(log(1 - C(xx)) log (2D(xx)) -)]
end for
10:
Sample mini-hatch of m unlaheled examples {x''...., xx, from X": Update C-) by descending its stochastic gradient:
V
A(log(1 - C(xx)) - log -og - 1)]
EXPERIMENTS
We now evaluate the proposed technique PAN and compare it with lfte-of-theeart baselines. Three text and two image classification datasets are used in our experiments.
(1). YELP: collection of online reviews from Yelp. Each review is labeled with star rating ranging from to 5. The dataset is extracted from the Yelp Dataset Challenge 2015. (2). RT: a
collection of online reviews from rotten tomatoes with sentiment labels good and bad. (3). IMDB: another collection of online review for binary sentiment classification. (4). 20News: a collection of about 20,000 newsgroup documents, partitioned (nearly) evenly across 20 different news topics. (5). MNIST: collection of 70,000 images of handwritten digits from 0 to 9. (6). CIFAR10: collection of 60000 32x32 colour images of 10 classes, with 6000 images per class. See [ for all datasets.
5.1 EXPERIMENT SETTINGS
Data Preparation: Since the five datasets are for traditional supervised learning with class labels, we need to prepare positive P and unlabeled U data for PU learning. We use two steps, after which we obtain the training and testing data for each dataset (on the left of the dataset name in Table [
Constructing positive and negative data, As not all datasets have 2 classes, we need to make each of them a two-class (positive and negative) dataset. RT and IMDB are already two-class datasets. For YELP, which has 5 classes, we remove the reviews with the class label of 3-stars, and split the remaining classes into two: one as the positive data (4 or 5 stars) and the other as the negative data (1 or 2 stars) (this is commonly done for sentiment classification (Pang & Lee 2008)). Following the baseline (Kiryo et al. 2017). for 20News, topics 'alt.', 'comp.', "misc.", and 'rec." form the positive data, and topics 'sci." 'soc.' and 'talk.' form the negative data. For MNIST, all images labeled with even numbers form the positive data and all images labeled with odd numbers form the negative data.
Step 1
For CIFAR 10, classes airplane, automobile, ship and truck are used the positive data and the rest as the negative data.
Step 2 - Creating PU learning datasets. After step 1, we get positive and negative training data for each dataset. We then build the PU learning training dataset, which includes positive and unlabeled data as follows. For each dataset (except CIFAR1O), we randomly select 10% (5% for CIFAR 10 for diversity). We also show more results by varying the ratio in Appendix D.I the percent of positive
See
YELP, http://ww.cs.coom. wttp://www....oooooooooooooooeee
RT.
hitps://www..i www.w/www.co..oocccmo www.dd..coo/ineefieee.. for IMDB, http://wwee.com wwt.///oo//aeeeeeeeeeeeeo htwp://jason////eeeee 20sewn/ONewssuuuppp for 20NEWS, http:/yynn. www.a./wwww../e//ommiiii.aa For MNIST, http://ww..c toronto kriz/cifar. 10 py-htt fur gz CIFAR1O.

---

Under review as a conference paper at ICLR 2020
examples from the whole positive set as the known positive data P for PU leamning The unlabeled data U' consists of the negative data and the remaining positive data in the dataset
Baselines: We use a-GAN and five state-of-the art representative approaches as the baselines.
a-GAN. This is the direct adaptation of GAN given in Section 3
UPU (du Plessis et al. 2015). This method proposed a general unbiased estimator that is also convex
for loss functions meeting certain linear-odd conditions.
NNPU ( et al. 2017). This is non-negative risk estimator for PU learning. When minimized, it is more robust against overfitting, and is able to use flexible models even given limited P data. Note that NNPU has two versions, the linear and the MLP versions. We give the results of the MLP version as it does better.
NNPUSB (Kato et al. 2019 This is recent algorithm that extended NNPU with an additional mechanism for handling sample selection bias.
2018) This system uses the GAN framework and array of generators and discriminators to generate both positive and negative data for PU learning.
GenPU (Hou et al.
PMPU (Gong et al.
2018): this is a traditional SVM based PU learning method.
is important to note that both UPU and NNPU need the input of the class prior probability, which is often not available in practice. In experiments, we provide them the correct class prior probabilities. Even with this favorable condition, they are still weaker than PAN, which does not need the class prior probability input. For UPU and NNPU, and a third party" respectively. For NNPUSB, we use the original code provided by the authors. Note also we use the same network these baselines, including architecture, number of parameters, and the optimization method. We also give them exactly the same positive and unlabeled data and the test data, For GenPU, we again use the code provided by the authors. For PGAN's results mentioned above, since there is no source code available, we used the best results reported in the paper.
the opensource code from the authors
For fair comparison, we use the same classifier C for all systems following the baselines. For text, 2-layer convolutional network (CNN), with 5 100 and 3 100 convolutions for layers and 2 respectively, and 100 filters for each layer, used as the classifier C(-) and discriminator (--). The word embeddings are also trained by the system. For MNIST, the classifier a 3-layer MLP (with 2 hidden layers, more specifically, d-512-256-1 1) as it is fairly simple. The classifier for the CIFAR10 dataset
Training Details:
all convolutional net: (32 x 32 3)-[C(3 x 3, 96)] C(3 x 3, 96, 2) - IC(1 x 1, 192)] - C(1 x 1, 10) - 1000 1000 1, where each input is a 32 x 32 RGB image, C33x3, 96) means 96 channels of 3x3 convolutions followed by ReLU, C(3 x 3, 96. 2) means a similar layer but with stride 2, etc. We set X in Eq. and to 0.0001, please see more details in Appendix D.2 We also balance the impact of positive and unlabeled data for term I in Eq. 3 in training; otherwise the positive examples will be dominated by the unlabeled data. We use 1:1 ratio of positive data and unlabeled data in each mini-batch in training. The network parameters are updated using the Adam algorithm with a learning rate of 0.0001. For baseline a-GAN, it needs pre-training of D(-). We use the original positive and unlabeled (regarded as negative) data to pre-train ((-) in order to enable its ability to classify positive and unlabeled data. We pre-train D(-) 3 epochs for the datasets.
acc
precision
recall
1.0
0.8
0.7
0.7
0.4
0.2
0.5
0.5
0.0
25
50
75
100
0
255
50
75
100
0
25
50
75
1000
spach
apoch
spoch
Figure 2: YELP- due space limit, we only show 100 epochs.
'Note that PU learning has two data sampling settings, we use the one-pass (Niu et al. training-set (Elkan & Noto| 2008) setting, not the case-controlled or two-pass (aad et al. 2009 setting.
2016b) or single-
htaps///gthuh..om/Garettt wwtptnpueeeeuppeueeccooo tf
7

---

Under review as a conference paper at ICLR 2020
Table I: Dataset details and experiment results: On the left of Dataset training and testing data for each dataset. On the right- F-score (F) and accuracy (Acc) of PAN and baselines for the dataset
Texting
G-GAN
UPU
NNPU
NNPUSS
PAN
P-Lahel
Unlabel
Dataset
Pos
Neg
Pos
Aec
26,000
260.000
20,000
83.33
83.56
426
4,264
1086
KF
66.10
58.03
66.38
64.10
1,250
11,250
12,500
12,500
12,500
IMDB
73.01
70.64
70.35
69.87
76.21
74.62
74.24
71.86
77.10
78.84
800
7,144
6,056
1,800
1,500
20News
63.48
68.66
59.13
53.07
78.52
78.07
75.87
75.56
$1.06
81.00
3,000
29.492
30,508
4,026
5,074
MNIST
94.67
95.03
94.21
94.29
95.40
95.35
95.60
95.
96.51
96.42
1,000
200000
30,000
4,000
6,000
CIARRI
76.15
83.04
86.20
53.96
86.09
88.34
86.56
88.59
57.22
89.70
Average
76.24
76.45
73.50
73.67
80.34
79.43
80.13
78.82
$1.99
82.27
precision
recall
0.70
0.9
1.0
0.65
0.7
0.6
0.55
0.2
0.50
25
50
75
25
75
100
25
75
100
epoch
epoch
epoch
Figure 3: RT due to space limit, we only show 100 epochs.
acc
precision
recall
0.8
1.0
1.0
0.8
0.6
0.6
0.6
0.4
0.4
0.2
0.2
0.4
25
50
75
100
0
25
50
75
100
25
50
75
100
epoch
epoch
epoch
Figure 4: IMDB due space limit, we only show 100 epochs in this and the figures below.
acc
precision
recall
0.8
0.7
0.6
0.6
0.3
0.4
0.5
25
50
75
100
0
25
50 opoch
75
100
25
50 opoch
75
100
opach
Figure 5: 20News
acc
precision
recall
1.0
0.8
0.8
0.8
0.6
0.6
0.4
X06
0.2
0.0
25
50
75
100
25
50
100
25
50
100
epoch
epoch
epoch
Figure MNIST
5.2 RESULTS AND ANALYSIS
Figures show the test accuracy, precision and recall in each epoch of each method. The usual first-order exponential weighted moving average smoothing with weight 0.7 is applied to the figures. Note that due to space limitations, we only show the curves of 100 epochs, but the final results in Table [ are produced with more epochs NNPU takes slightly longer to reach the peak (see below).
8

---

Under review as a conference paper at ICLR 2020
BCC
precision
recall
1.0
1.0
0.8
0.8
0.6
0.6
0.4
0.2
0.4
0.4
50
50
100
25
50
100
opoch
epoch
epoch
Figure 7: CIFARIO
The final accuracy and F-score results of A-GAN, UPU, NNPU, NNPUSB and PAN are given in Table (on the right side of dataset names) and the results for GenPU and PMPU are given Table 2 Since different epochs give different results, for a fair comparison, we give the average of both the best F-score (F) and best accuracy (Acc) for each system each dataset over 200 epochs (all systems converged before 200 epochs) over 5 runs. The F-score measured on the positive data/class, as in PU learning the user is normally interested in identifying the positive data. The last row in the table gives the average result for each column.
From the Figures and Table 1 we can make the following observations.
(1). From the results in Table [ (on the right side of dataset names in the table), we see that PAN outperforms all baselines all datasets (the last row gives the average of each algorithm for all datasets). Among the baselines, NNPU and NNPUSB are the strongest and their results are very similar. PAN outperforms them markedly. Given that PAN does not need class prior probability input, this is even more significant. The direct adaptation of GAN a-GAN is weaker than both NNPU and PAN, but is better than UPU. Although NNPUSB extends NNPU, did not do better than NNPU. The reason could be that our data do not have sample selection bias, which NNPUSB tries to address.
(2). Figures 2 to 7 show that PAN and NNPUSB are robust across all datasets with both high precision and high recall. And clearly PAN is better than NNPUSB. NNPU is rather unbalanced for precision and recall for YELP and RT, either very high precision but very low recall, or vice versa. a-GAN and UPU have the same problem, which is highly undesirable. PAN also outperforms baselines consistently in accuracy for all six datasets.
(3). From the 6 figures, we also see that a-GAN is unstable. Precision, recall, and accuracy fluctuate greatly from one epoch to another. Stability problem of GAN is well documented (Metz et al. 2017] Berthelot et al. 2017). Our adaptation requiring reinforcement learning to train it is likely to have made the problem worse.
(4). NNPU converges slowly (Figure 3p. It didn't 't converge even 100 epoch. We report its best accuracy and F-score in Table in 200 epachs (it converged earlier than 200).
Table 2 shows the results of GenPU and PMPU. Since GenPU's data generator cannot generate text data and thus that GenPU's results are dramatically worse. Hou et al. (2018) showed that GenPU does well with few classes as the positive and negative, e.g. class as positive and I class as neg- ative. However, in our case, both positive and negative consists of many classes, which probably make GenPU work poorly. PMPU is also significantly poorer than PAN.
Table 2:
Comparison between our method and GenPU and PMPU in terms of Accuracy (%)
results for the text datasets. We
Dataset
GenPU
PMPU
PAN
MNIST CIFARIO
70.43
95.74
96.46
66.25
81.34
89.65
In summary, we can conclude that PAN markedly outperforms the baselines in accuracy, F-score, robustness, and stability. Given that PAN does not need the class priors, this is more significant.
Varying Positive Data and NNPU's Sensitivity to Class Prior:
We use MNIST and CIFAR10 as representatives to study these issues. For each dataset, we randomly select or 2 classes in the original data to form the positive set, and the rest to form the negative set to generate 2 PU learning datasets as discussed above. Since both MNIST and CIFAR have 10 classes, for the 1-class positive PU data, the class prior probability is 10% for positive and 90% for negative (or 1:9 for short). For the 2-class positive PU data, it is 20% for positive and 80% for negative (or 2:8). The results are given in Table 2. We see similar improvements from PAN with the exact class prior given NNPU
9

---

Under review as a conference paper at ICLR 2020
(1:9 or 2:8). For the 1:9 (respectively, 2:8) case for both MNIST and CIFARIO, if we change the class prior from the correct 1:9 (2:8) to the wrong 2:8 (3:7). NNPU's result drops are small (not in Table 2). So NNPU has some robustness. However, if we change the wrong 3:7 4:6 (for the correct 1:9), and 4:6 or 5:5 (for the correct 2:8), the drops are dramatic for MNIST. For CIFAR 10, they are smaller, even small increase in F for the wrong 4:6 (correct 2:8), likely an anomaly this data is hard, but still poorer than PAN. We conclude although it is possible to estimate the class prior (see Section 2), if the estimate is off, the results can be quite poor.
Table 3: Varying the positive data and the class prior probability for NNPU.
class as positive (1:9)- results given as F/ Ace
2 classes as positive (2:8)- resalts given as Ace
Dataset
NNPU
NNPU
PAN
1:9 (correct)
3:7 (wrong)
4:6 (wrong)
2:8 (currect)
4:6 (wrong)
5:5 (wrong)
MNIST
97.88/99.18 199119
97.82/99.18
91.16/96.22
91.13
95.59 / 98.42
95.55 / 98.32
77.50 88.90
66.94/ 81.41
CIFARIO
84.73
51.46/ 84.65
48.55 / 88279
41.99 / 76.64
59.42 78.35
56.45 78.27
57.67 / 77.61
54.62 73.29
CONCLUSIONS
This paper proposed new GAN style PU learning method PAN based on adversarial training. PAN is also significantly different from GAN as PAN does not use generator but a classifier in its place. The objective function of PAN is also entirely different as is based on Kl.-divergence. PAN represents a new way to do PU learning. Empirical evaluation using both text and image datasets showed that PAN outperformed the state-of-the-art baselines. Also importantly, PAN obtained the better results without using any class prior probability information.
REFERENCES
Han Bao, Tomoya Sakai, Issei Sato, and Masashi Sugiyama. Convex formulation of multiple instance learning from positive and unlabeled bags. Neural Networks, 105:132-141, 2018.
Jessa Bekker and Jesse Davis. Beyond the selected completely at random assumption for learning from positive and unlabeled data. arXiv preprint arXiv: 2018a.
Jessa Bekker and Jesse Davis. Learning from positive and unlabeled data: A survey, arXiv preprint 2018b.
David Berthelot, Tom Schumm, and Luke Metz. Began: Boundary equilibrium generative adversarial networks. CoRR, abs/ 2017.
Borja Calvo, Núria López-Bigas, Simon Furney, Pedro Larrañaga, and Jose A Lozano. A partially supervised classification approach to dominant and recessive human disease gene prediction. Computer methods and programs in biomedicine, 85(3):229 237, 2007.
Luigi Cerulo, Charles Elkan, and Michele Ceccarelli. Learning gene regulatory networks from only positive and unlabeled data. BMC bioinformatics, 11(1):228, 2010.
Shiyu Chang, Yang Zhang. Jiliang Tang, Dawei Yin, Yi Chang, and Thomas Hasegawa-Johnson, mark Huang. Positive-unlabelae learning in streaming networks. In Proceedings of the 22nd ACM SIGKDD International Conference Knowledge Discovery and Data Mining, pp. 755764, 2016.
Qingchao Chen, Yang Liu, Zhaowen Wang, Ian Wassell, and Kevin Chetty. Re-weithted adversarial adaptation network for unsupervised domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7976-7985, 2018
F. Chiaroni, Mohamed-Cheel Rahal, Nicolas Hueber, and Frédéric Dufaux. Learning with genera- tive adversarial network from a positive unlabeled dataset for image classification. In Proceedings of 25th IEFF International Conference on Image Processing (ICIP), pp. 1368-1372. 2018.
Mare Claesena, Frank De Smet, Johan A. K. Suykensa, and Bart De Moor. Positive and unlabeled multi-graph learning. Nearocomputing, 160:7384, 2015.
10

---

Under review as a conference paper at ICLR 2020
François Denis. Pac learning from positive statistical queries. In Proceedings of International Conference on Algorithmic Learning Theory, pp. 112-126. Springer, 1998.
Marthinus C. du Plessis, Gang Niu, and Masashi Sugiyama. Analysis of learning from positive and unlabeled data. In Proceedings of Advances in neural information processing systems, pp. 703-711, 2014.
Marthinus C. du Plessis, Gang Niu, and Masashi Sugiyama. Convex formulation for learning from positive and unlabeled data. In Proceedings of International Conference on Machine Learning, pp. 1386-1394, 2015.
Marthinus Christoffel du Plessis, Gang Niu, and Masashi Sugiyama. Class-prior estimation for learning from positive and unlabeled data. Machine Learning, 2017.
Charles Elkan and Keith Noto. Learning classifiers from only positive and unlabeled data. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 210-220..400 ACM, 2008.
Donato Hernández Fusilier, Manuel Montes-y Gómez, Paolo Rosso, and Rafael Guzmán Cabrera. Detecting positive and negative deceptive opinions using pu-learning. Information processing & management, (1)::433 443, 2015.
Chen Gong, Jie Yang, and Jian Yang. Multi-manifold positive and unlabeled learning for visual analysis. IEEE Transactions on Circuits and Systems for Video Technology, 2019.
Tieliang Gong, Jieping Wang, Guangtao Ye, Zongben Xu, and Ming Lin. Margin based pu learning. In Proceedings of The Thirty-Second AAAI Conference on Artificial Intelligence (AAAI-18), 2018.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Proceedings of Advances in neural information processing systems, pp. 2672-2680, 2014.
Ming Hou, Brahim Chaib-Draa, Chao Li, and Qibin Zhao. Generative adversarial positive-umlabelled learning. IJCAI, 2018.
Cho-Jui Hsich, Nagarajan Natarajan, and Inderjit S Dhillon, Pu learning for matrix completion. In ICML, pp. 2445-2453, 2015.
Yu-Guan Hsieh, Gang Niu, and Masashi Sugiyama. Classification from positive, unlabeled and biased negative data. arXiv preprint arXiv:1810.00844 2018.
Shantanu Jain, Martha White, and Predrag Radivojac. Estimating the class prior and posterior from noisy positives and unlabeled data. In NIPS-2016, 2016.
Shantanu Jian, Martha White, and Predrag Radivojac. Recovering true classifier performance in positive unlabeled learning. In Proceedings of the 31st AAAI Conference Artificial Intelligence, pp. 20662073, 2017.
Masahiro Kato, Takeshi Teshima, and Junya Honda. Learning from positive and unlabeled data with selection bias. ICLR, 2019.
Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv:1312.6114, 2013.
arXiv preprint
Ryuichi Kiryo, Gang Niu, Marthinus C. du Plessis, and Masashi Sugiyama. Positive-unlabeled learning with non-negative risk estimator. In Proceedings of Advances in neural information processing systems, pp. 1675-1685, 2017.
Wee Sun Lee and Bing Liu. Learning with positive and unlabeled examples using weighted logistic regression. In ICML,, volume 3, pp. 448-454 2003.
Wenkai Li, Qinghua Guo, and Charles Elkan. A positive and unlabeled learning algorithm for one-class classification remote-sensing data. IEEE transactions on geoscience and remote sensing, 49(2):7177725, 2010.
11

---

Under review as a conference paper at ICLR 2020
Xiaoli Li and Bing Liu. Learning to classify texts using positive and unlabeled data. In IJCAI, volume 3, pp. 587-592, 2003.
Bing Liu, Wee Sun Lee, Philip S. Yu, and Xiaoli Li. Partially supervised classification of text documents. In ICML, volume 2, pp. 387-394, 2002.
Bing Liu, Yang Dai, Xiaoli Li, Wee Sun Lee, and Philip S. Yu. Building classifiers using positive and unlabeled examples. In ICDM, pp. 179. IEEE, 2003.
Fangqing Liu, Hui Chen, and Hao Wu. Discriminative adversarial networks for positive-unlabeled learning, arXiv preprint 1e..ii:100.00000 2019.
Yashu Liu, Shuang Qiu, Ping Zhang, Pinghua Gong, and Fei Wang. Computational drug discovery with dyadic positive-unlabeled learning. In Proceedings of the 2017 SIAM International Conference on Data Mining, pp. 4553, 2017.
Aditya Krishna Menon, Brendan van Rooyen, Cheng Soon Ong, and Robert C. Williamson. Learning from corrupted binary labels via class-probability estimation. In JCML-2015,, 2015.
Luke Metz, Ben Poole, David Pfau, and Jascha Sohl-Dickstein. Unrolled generative adversarial networks. ICLR, 2017.
Fantine Mordelet and Jean phin-philippe Vert. A bagging svm to learn from positive and unlabeled examples. Pattern Recognition Letters, 37, 2014.
Gang Niu, Marthinus C. du Plessis, Tomoya Sakai, Yao Ma, and Masashi Sugiyama. Theoretical com- parisons of positive-unlabeled learning against nositive-negative learning. In NIPS, pp. 11991207, 2016a.
Gang Niu, Marthinus C. du Plessis, Tomoya Sakai, Yao Ma, and Masashi Sugiyama. Theoretical comparisons of positive-unlabeled learning against positive-negative learning. In NIPS-2016, 2016b.
Curtis G. Northcutt, Tailin Wu, and Isaac L. Chuang. Learning with confident examples: Rank pruning for robust classification with noisy labels. In Proceedings of the Thirty-Third Conference on Uncertainty in Artificial Intelligence, 2017.
Bo Pang and Lillian Lee. Opinion mining and sentiment analysis. Foundations and Trends in Information Retrieval, 2(1-2):1-135, 2008.
Harish G. Ramaswamy, Clayton Scott, and Ambuj Tewari. Mixture proportion estimation via kernel embedding of distributions. In /CML-2016, 2016.
Jing Ren, Qian Liu, John Ellis, and Jinyan Li. Positive-unlabeled learning for the prediction of conformational b-cell epitopes. BMC bioinformatics, 16(18):S12, 2015.
Tomoya Sakai, Marthinus C. du Plessis, Gang Niu, and Masashi Sugiyama. Semi-supervised classification based on classification from positive and unlabeled data. In Proceedings of the 34th International Conference on Machine Learning, pp. 2998-30990000 2017.
Emanuele Sansone, Francesco GB De Natale, and Zhi-Hua Zhou. Efficient training for positive unlabeled learning. IEEE Trunsuctions on Pattern Analysis and Machine Intelligence, 2018.
Clayton Scott and Gilles Blanchard. Novelty detection: Unlabeled data definitely help. In Proceedings of Artificial Intelligence and Statistics, pp. 464 2009.
Konstantinos Sechidis and Gavin Brown. Markov blanket discovery in positive-unlabelled and semisupervised data. In Proceedings of Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 351366, 2015.
Hong Shi, Shaojun Pan, Jian Yang, and Chen Gong. Positive and unlabeled learning via loss decomposition and centroid estimation. In ISCAI, pp. 2699-2695, 2018.
12

---

Under review as a conference paper at ICLR 2020
Richard S Sutton, David A McAllester, Satinder P Singh, and Yishay Mansour. Policy gradient methods for reinforcement learning with function approximation. Proceedings of Advances in neural information processing systems, pp. 1057-1063, 2000.
Gill Ward, Trevor Hastie, Simon Barry, Jane Elith, and John Leathwick. Presence-only data and the em algorithm. Biometrics, 2009.
Jia Wu, Shirui Pan, Xingquan Zhu, Chengqi Zhang, and Xindong Wu. Positive and unlabeled multi multi-graph learning. IEEE transactions on cybernetics, 47(4):818 8299, 2017.
Yixing Xu, Chang Xu, Xu Chao, and Dacheng Tao. Multi-positive and unlabeled learning. In Proceedings of the Twenty-Sixh International Joint Conference Artificial Intelligence (I/CAI- 17), 2017.
Jinfeng Yi. Kush R. Varshney, Lijun Zhang, and Yao Li. Scalable demand-awas aware recommendation. In Proceedings of Advances neural information processing systems, pp. 24122421, 2017.
Hwanjo Yu, Jiawei Han, and Kevin Chen-Chuan Chang. Pebl: Web page classification without negative examples. ieee transactions on knowledge and data engineering 16(1), 7081 (2004). IEEE Transactions on Knowledge and Data Engineering, 16(1):70-81, 2004.
Jing Zhang, Zewei Ding, Wanqing Li, and Philip Ogunbona. Importance weighted adversarial nets for partial domain adaptation. In Proceedings of the IEEE Conference Computer Vision and Puttern Recognition, pp. 81 8156-8164, 2018.
Kaja Zupanc and Jesse Davis. Estimating rule quality for knowledge base completion with the relationship between coverage assumption. Proceedings of the Web Conference, 2018.
13

---

Under review as a conference paper at ICLR 2020
Appendix
A ASYMMETRY OF KL(D.IIC. FOR POSITIVE AND NEGATIVE DATA
In this section, we use the gradient asymmetry for positive and negative of KL-divergence to show the need for the term marked III in Eq. (also see Eq. [eblww). The term marked II in Eq. will produce asymmetric gradients for positive and negative examples for both D()) and C(-) due to asymmetry of (D(D:IIC) for positive and negative data (explained below). If we don't 't have the term marked III, gradient of D(.) is:
VDV(D,C) Vol- Σ KL(P AC
E KL(D.IIC))
log ((a7)(1- - C(+))
(5)
(
(
((a)))
(e)
(b)
where no and nh the size of positive and unlabeled set respectively. Term marked (a) is symmetric for positive and unlabeled data as they can obtain gradients with the same scale for the corresponding position, e.g., D(= Dxx") 1. But it is asymmetric for positive and negative data positive data exist in the unlabeled set. That cause the positive being over optimized toward negative. Unfortunately.
marked (b) is also asymmetric for positive and negative data. We can see, the zero point of
gradient term marked (b) is:
D( - O(+)))
log (1 ((x;)) D(x)) Cxx))
(6)
which means that the zero point is moved according to C(x')). In the worst case, if C overfitted to give small probability instances in the unlabeled set, then D(-) is not easy to escape from overfitting, In summary, that will cause high precision and low recall.
Asymmetric phenomenon also exist in Eq. below without the term marked as the gradient for C()) is: VcV(D, vel KL(P!" D. A
KL(D.
C(a))- log (1- ((x)))
Clearly, it is asymmetric for positive and negative data, as positives have different gradient scale compared to negatives. And that can cause the unbalanced training problem. this case, we propose to use the flipped distribution of CH denoted by Ct, to address the problem, and please refer to the term marked III in Eq. g After adding the term marked III, the asymmetric gradient problem caused by the term marked is eliminated. The gradient for D(.) now is Σ log C(af)
(I-C(="))
which be regarded constant when optimizing D(-)- And the gradient for C(-) now is 2-(x))-1 which is symmetric between positive and negative. log ([CC())))((T))
B SIMPLIFICATION
Recall the loss function Eq. 3
n5 KL(D (C)
min max V(D,C) C
KL(D"I(;))
(8)
D
m
14

---

Under review as a conference paper at ICLR 2020
KL-divergence is defined as:
KL(PIIQ) = Σ P(z) log P(x) P(x) log Q(x)
=ER
X' denotes the probability space, it is 1 or 0 (X' = {1. 0}) in our scenario. We first address term I
in Eq. 88 if we use D to denote D (1) (the probability for ith instance being positive judged by
discriminator) and P to denote P! (1), then (0) = D and P. (0) 1-P. Then we have:
[I(PP""| (!"")
-o P + P Plog D (1 P) log(1 P) + (1 P) D) log( Plog D + -1 D)
(10)
Due to the fact that (0) = 0 and (1) 1 if the ith instance is positive and P!" (1) 0 and pp (0) = if the ith instance is unlabeled, we can rewrite the result as:
Pp(r)) [log D(x")] + Px (l=) D(x
(11)
Similarly, for term ① Eq 8 we use D to denote (1) and C to denote C1), then we get:
KL(D"|(C)) - KLLE)]))))
))
Dlog D - Dog + D) log(1 D) (1 Do C) - D log D
+ Dlog(1 C) - (1 D) log(1 D) + D) log C
(12)
Dog C - (1
log(1 C) D log(1 C) + (1 D) log
= (log(1 - C) - log (2D
Then we have:
(log(1 C) log (2D
= (x" [loggl (log(1-((x")) -og((("))))20 (13)
Combining Eqs. 11 and 13
we get Eq. 14
min max V(D,C)
C D
= Expx (x)) [log D(x"))
W.-H(PL.DDO ((eer))
(14)
+ Ex (log(1 C(xx log(C(("))) 1)]
where PP denotes the distribution of the positive data.
C THEORETICAL ANALYSIS ABOUT THE LEARNED CLASSIFIER
In this section, we analyze the properties of the learned classifier and show why Eq. 3 can perform PU learning. Intuitively, from Eq. 8 (same Eq. 3 in the paper).
see that ()-) is biased if we only consider term I because the unlabeled set contains both positive and negative examples. However, terms II and III help correct the bias. Eq. 14 (same as Eq. 4 the paper), which is derived from Eq. 8 for training, shows the property more clearly than Eq. 8 Notice that the bias in term in Eq. g will result high precision and low recall for the positive class. Now back to Eq. 14 and let us imagine that most examples in the unlabeled set are regarded as negative by C(-) (meaning low recall). From Eq. [4] we can see that the value of term V will below zero. But when optimizing D(-), term VI will push D(-) up for these data points, and thus the bias is reduced and the low recall problem is mitigated because in the next optimization iteration, C(-) will follow D(-) to go up for these data points.
We now theoretically discuss the optimal decision surface of the classifier C(.) learned by the proposed PAN.
Proposition 1. Let T(x) = log[1 C(x)) log(C(x)), =(x) = f(T(x)) the learned optimal decision surface of C(.) is:
(x)
exx) 2 + + (x)
(15)
f(-) is type of function that satisfies E(x) T(x) 0.
From Proposition 1, we can see that PAN finds the decision surface by combining (x(x) and pa (x)... The combination is controlled by exx). exx) is function of C(x).
15

---

Under review as a conference paper at ICLR 2020
C.I PROOF OF PROPOSITION
Proof: The training criterion for discriminator D, given any classifier C, is to maximize the quantity V(C, D),
/ P2(xx) log D(x)dx / (x(x D(x))
V(C, D)
(16)
+* / (x)))]] C(x)) log (2D)) 1)de
Clearly, the maximum point appears at the point with derivative 0. Then calculate the partial derivative of V(C, D) to and get:
(x x ("(x) ((x) - ((x)
AP (x)[lox - log C(x)]= =
(17)
Directly computing the solution is complex. If we omit term b for the time being, the solution for (*(x)
Eq. 17 is D(x) p=(x)- (x)) After bringing back b this solution should be revised as follows. ((x)) and e(x) f(T(x)), re-write the
With the definition of (x) log] C(x)] loggg
solution after revision:
(x) (x) + =
D' (x) P*(x)
(18)
where =(x) a function of Txx) since (x( and pd p=(x) are decided by the dataset and e(x) changes
with the change of (x).. The exact expression of exx) is difficult but we can show s(x) x T(x) in our case. Eq. 17 term @ decrcases monotonously when (xx) E (0, (),1) and both À and P
greater than O. In this case, if (x) > 0 (T() O), to keep Eq. 17 equal to 0, D(x) must move toward the positive (negative) direction, which indicates s(x) > (E(x) < 0). Formally, we have:
(xx) x T(x): (xx) > 0
(19)
Note that the training objective of D be interpreted as using the training data (PP(x) and P P*(x)) to find discrimination bound and utilizing the learned knowledge C to adapt it. The mini-max
game in Eq. can now be reformulated as:
L(C)= max V(C,D)
= =R P" [log(( - C(x)) log(C(x))) 1)] 2P"
(20)
=E [T((x)( (x(x) (x(x) 2=(x) 1)]
Clearly, because the range of T(x) (-e,,), L(C) achieves its minimum when T(x) and (2D*(x)
1) have opposite signs.3 Then, the optimal T (x) satisfies:
2P
(x) px(x) (x(x 2x(x) <
(21)
As introduced in Footnote 5, C(x) (0, 1). In this case, C(x) 0.5 the decision
surface to perform classification. Clearly, this decision surface equals to T (x) log - 0.5)
log(0.5) O. In summary, we get the optimal decision surface:
(x) (xx) 2 + + (x)
(22)
WWe force ((x) satisfy the condition adding Sigmoid function the end of D. We also force the output range of C into (0, using the same method.
'The original range of (xx should be (-o0, +oo). However, such range can cause stability problems for training. We adopt a standard trick, i.c., adding small value to the log function, e.g., log(C(x)) 1 1e)"), to change the range of T(x) to (-8,8).
16

---

Under review as a conference paper at ICLR 2020
D MORE EXPERIMENTS AND ANALYSIS
In the paper we showed that PAN gets significant improvement comparing with state-of-the-art baselines. Here we give more detailed analysis of PAN terms of dealing with varied positive ratio and the selection of wapphh-paraaeee parameter À in Eq. (also 4
D.1 VARYING KNOWN POSITIVE RATIO
In this section, we analyse the performance of PAN when dealing with varied ratios of known positive examples. We vary the ratio of known positive examples from 5% to 30%, and show the accuracy and F-score of PAN and baseline on MNIST and CIFAR1O 100 datasets. To verify the ability of PAN working in extreme conditions, we tested PAN on the situation with only 1% positive examples. The results are reported in Table and 5
Table Varying the ration of known positive data on MNIST.
MNIST results given as F Acc
Model
1%
5%
10%
20%
30%
NNPU
88.34/88.51
93.96/94.09
95.60/96.51
96.89/96.96
97.51/ 97.57
PAN 90.45/90.30
95.27 /95.36 96.51/96.42
97.38/97.49
97.90/97.95
Table 5: Varying the ration of known positive data on I0.
CIFARIO results given F Acc
Model
1%
5%
10%
20%
30%
NNPU
81,41/84.22
86.09/88.84
87.84/90.
89.05 99.04
90.01/91.66
PAN
82.70/86.10
87 87.22/89.70
88.37/90.77
89.74/91.85
90.65 92.49
From Table 4 and 5 data and with extremely few known positive examples. We note that that the margin between PAN and NNPU goes large with the decrease of the ratio of known positive examples, which indicates that PAN is more effective than NNPU. The margin is smaller when the known positive ratio enlarges, that is because if we have enough positive data, the limitation of getting better results is no longer the PU learning method, but the performance of the classifier.
we can see that PAN can do well with different proportions of known positive
D.2 HYPER- PARAMETER SELECTION
is the value but it not too sensitive when it around 0.0001. In our case, we set to 0.0001.
that balances the KI-divergences, Here, we show that À should be small
Table 6: Sensitivity of on MNIST.
Varying À results are Acc
Model
0.01
0.001
0.0001
0.00001
0.000001
PAN
82.70
88.90
90.30
88.23
86.44
D.3 ABLATION STUDY FOR TERM MARKED BY III IN EQ.
Table 7: Accuracy (%) on different datasets for PAN with and without term III.
Model
YELP
RT
IMDB
20News
MNIST
CIFARIO
PAN without term III
80.67
60.32
78.45
72.63
96.30
89.38
PAN full model
83.56
64.10 10
78.84
81.00
96.42
89.70
17

---

Under review as a conference paper at ICLR 2020
Table F-score comparison on different datasets with and without term III.
Model
YELP
RT
IMDB
20News
MNIST
CIFARIO
PAN without term III
81.86
66.28
78.52
73.59
96.27
86.68
PAN full model
83.45
66.58
77.10
81.06
96.51
87.22
Table 7 and show PAN's ablation results in accuracy and F-score with or without term III respectively, From the two tables, we can see that adding term III indeed improves the performance of PAN on of datasets.
18


Reviews:
[{"review_id": "SylW40kRFH", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper considers the problem of learning a binary classifier from only positive and unlabeled data (PU learning), where they develop a Predictive Adversarial Networks (PAN) method by using the GAN-like network architecture with a KL-divergence based objective function. Experiments and comparisons with SOTA are provided.\n\nPros:\nTheir idea of making an adaption to GAN architecture by replacing the generator by a classifier to select P from U and using the discriminator to distinguish whether the selected data is from P or U for PU learning is interesting, and benefits from not relying on the class prior estimation.\n\nQuestion:\nThe authors claim that Eq. (2) cannot be trained in an end-to-end fashion directly, this statement may need some modification since there are some existing works replacing c(x) by a score function or some other continuous function and then this direct adaptation can be trained, for example, see Eq. (5) in “Discriminative adversarial networks for positive-unlabeled learning. arXiv:1906.00642, 2019”. Can any explanation be given on this?\n\nRemarks:\nThe clarity of the paper could be improved in multiple places. For example, the data generation processes can be mathematically defined in the problem setting part, now it is quite confusing to me. And more details on experimental protocol may be needed: e.g. what kind of hyperparameter tuning was done?\n \nIn general, the paper proposed an interesting GAN-like network architecture to learn from PU data, but some unclear parts need to be improved. ", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "HkxCbL-e9H", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposed an interesting idea of using two adversarial classifiers for PU learning. The first classifier tries to introduce samples from unlabeled data that are similar to the existing labeled positive data, and the second one tries to detect if a sample has a ground truth label or drawn from unlabeled data by the first classifier (fake). The idea of the paper is interesting; it is well-motivated and well supported with a range of experiments from NLP and computer vision tasks. The paper is a good read but required a pass of proofreading; some rearrangement of the concepts (for example, in second paragraph C(.) and D(.) is used, but they are introduced properly in section 4. Also, the paper could use some clarifications.\n\n* How the proposed method handles unlabeled positive samples that have a different distribution from P or has a similar distribution with some of the negative samples that might exist in unlabeled samples. \n\n* The experiment section could have enjoyed from an ablation study in which a system that only implements terms I and II from Eq(3). The authors mentioned that such an objective function is asymmetric but didn't explore the implications of such an objective function in the empirical experiments.\n\n* PGAN's results are not compared in the fair condition since the PU version of CIFAR 10 is different from PGAN's version.\n\n* Using MLP classifier for the text classification (e.g., for YELP) makes a very weak baseline for the system. Also, training the word embedding by the system itself is unrealistic. Therefore, the sentence might need to be rewritten.\n\n* Some readability issues: \n(i) C(.) and D(.) needs an introduction in the section \"I Introduction\" before their usage.\n(ii) The idea could be illustrated easily. Such a figure significantly improves the readability of the system.\n(iii) be careful with the use of \\cite{} and \\citep{} interchangeably (\"{Liu et al., 2003; Shi et al., 2018\" -> Liu et al. (2003) and Shi et al. (2018) ..., \n(iv) The first paragraph of section 2 should be split into two from this phrase \"None of these works...\"\n(v) Please rewrite the latter half of paragraph 2 in section 2. Also, please rewrite the beginning sentences of section 4.1 and the final paragraph of section 3.\n(vi) right after equation (2), please change x_s to \\mathbf(x)^s for the consistency of your formulation.\n(vii) favorible -> favorable, radio (in section 5.1) -> ratio,\n(viii) please add a reference for this statement. \"This is one of the best architectures for CIFAR10.\"", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "Bke8e-RTFS", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "<Paper summary>\nThe authors proposed a novel method for positive-unlabeled learning. In the proposed method, adversarial training is adopted to extract positive samples from unlabeled data. In the experiments, the proposed method achieves better performance compared with state-of-the-art methods. \n\n<Review summary>\nAlthough the idea to utilize adversarial training for PU learning is interesting, the proposed method is not sufficiently validated in theory. In addition, the manuscript is hard to follow due to confusing notations and lack of figures. I vote for rejection.\n\n<Details>\n* Strength\n + The main idea is simple and interesting.\n + The proposed method performs well in the experiments.\n\n* Weakness and concerns\n - Confusing notations and lack of figures.\n  -- Lack of mathematical definition of C and D.\n  -- The argument of P^p and that of P^u are different (x^p and x^u), which implies that those distributions are defined at different space (but actually same).\n  -- Shared index ``i\" for positive and unlabeled data in Eq. (3).\n  -- The notation with ``hat\" often imply the empirically estimated (or approximated) value in the field of ML. \n  -- No figures about the proposed method. Specifically, it is hard to understand the relationship between C and D. \n\n - Since Eq. (3) looks totally different from Eq. (2), why Eq. (3) is reasonable remains unclear. \n  -- About I: first, P^{pu} cannot be calculated, because it requires unavailable labels of x^u. If you treat unlabeled data as negative, it should not be called ``ground-truth,\" and the term I cannot help D correctly recognize positive samples. Second, the positive samples are almost ignored in this term, because the number of positive data should be substantially small in a common setting of PU learning. \n  -- About II: the authors explain the role of this term by min-max game between C and D during optimization, but the most important point here is what will happen when we obtain the optimal C and D after the optimization. What property or behavior do the optimal C and D have? \n\n - What do the authors want to claim with Proposition 1? The right-hand side of Eq. (5) cannot be easily calculated due to the density ratio between P^p and P^u. There is no explanation about what f and eps mean. What ``optimal\" means is also ambiguous. \n\n\n* Minor concerns that do not have an impact on the score\n - Although the problem setting is quite different, the idea of this paper is partially similar to the importance weighting technique adopted in some recent domain adaptation methods [R1, R2]. Do you have any comment on that?\n\n[R1] ``Reweighted adversarial adaptation network for unsupervised domain adaptation,\" CVPR2018 \n[R2] ``Importance weighted adversarial nets for partial domain adaptation,\" CVPR2018\n\n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: Thanks for your feedback to the reviewers, which helped us a lot to better understand your paper.
Through the discussion, the overall evaluation of this paper was significantly improved.
However, given the very high competition at ICLR2020, this submission is still below the bar unfortunately.
We hope that the discussion with the reviewers will help you improve your paper for potential future publication.
Decision: Reject