Year: 2019

Abstract:
Positive-unlabeled (PU) learning learns a binary classifier using only positive and unlabeled examples without labeled negative examples. This paper shows that the GAN (Generative Adversarial Networks) style of adversarial training is quite suitable for PU learning. GAN learns a generator to generate data (e.g., images) to fool a discriminator which tries to determine whether the generated data belong to a (positive) training class. PU learning is similar and can be naturally casted as trying to identify (not generate) likely positive data from the unlabeled set also to fool a discriminator that determines whether the identified likely positive data from the unlabeled set (U) are indeed positive (P). A direct adaptation of GAN for PU learning does not produce a strong classifier. This paper proposes a more effective method called Predictive Adversarial Networks (PAN) using a new objective function based on KL-divergence, which performs much better. Empirical evaluation using both image and text data shows the effectiveness of PAN.

heading: 1 INTRODUCTION
text: Positive-unlabeled (PU) learning learns a binary classifier from only Positive (P) and Unlabeled (U ) examples with no labeled negative examples (Denis, 1998; Liu et al., 2002; Elkan & Noto, 2008; du Plessis et al., 2014; Bekker & Davis, 2018b). It has many applications in text analysis, biomedicine, spam detection, recommendation, remote sensing, matrix and knowledge base completion, graph learning, etc (Li & Liu, 2003; Yu et al., 2004; Fusilier et al., 2015; Scott & Blanchard, 2009; Li et al., 2010; Cerulo et al., 2010; Calvo et al., 2007; Ren et al., 2015; Hsieh et al., 2015; Wu et al., 2017; Zupanc & Davis, 2018). This paper proposes an adversarial PU learning method inspired by GAN (generative adversary networks) (Goodfellow et al., 2014), as we found that the GAN framework naturally suits PU learning. GAN aims to generate data of a particular training class, which is like the positive class P in PU learning. GAN works by generating likely positive data using a generator G(·) to fool a discriminator D(·), which determines whether the generated data indeed belong to the training (positive) class. For PU learning, this is like choosing likely positive instances from the unlabeled set U (achieved by a classifier) also to fool a discriminator D(·). Thus, we can simply replace GAN’s generator with a classifier C(·) to produce a PU learner.1 Note that the classifier is also a discriminator, but we use the term classifier here to distinguish it from the original discriminator of GAN. Note also both C(·) and D(·) are neural networks.2
This paper first proposes a direct adaptation of GAN, called a-GAN (adapted GAN), for PU learning following the above idea (see Section 3). However, this simple method is unable to produce stateof-the-art results. It led us to propose a new objective function based on Kullback-Leibler (KL) divergence. Our final proposed method is called PAN (Predictive Adversary Networks) due to the use of the classifier to replace the generator in GAN along with a new adversarial training method. On the new objective function, it is clearly desirable to consider the overall performance of C(·) on the entire unlabeled set. We use a distance metric to measure whether C(·) can produce similar
1PU learning is analogous to GAN and vice versa. That is, if we could put all the data (e.g., images) that can be generated by GAN’s generator in a set, the set should be regarded as unlabeled as it contains both good (positive) and bad (negative) images. Then what the generator does is like selecting good images to fool the discriminator, which is exactly what our classifier does with a set of given unlabeled data.
2Their exact architectures are given in the experiment section (see Training Details in Section 5.1) since C(·) and D(·) have different architectures for text and image.
predictions to those of D(·) for all the examples in U . If C(·) gives similar predictions, it means that the examples obtaining high probabilities from C(·) also get high probabilities from D(·), achieving the goal of fooling D(·), which will give us a good final PU classifier. We employ KL-divergence as the distance metric to measure the difference between C(·) and D(·). One important advantage of PAN is that it does not need the input of class prior probability, which many state-of-the-art systems need (see Section 2). In practice, the class prior is unknown (there are some existing methods to estimate it, see below). Hou et al. (2018) and Chiaroni et al. (2018) have employed GAN to generate positive and/or negative data and then use a separate learner (e.g., CNN) to learn the final PU classifier using the generated data, they are not adaptations of GAN like PAN and their generators generate only images. PAN can be applied to any data as it has no generator. We evaluate PAN using both text and image classification datasets and show it outperforms start-of-the-art PU learning baselines even when we give them the perfect class prior probabilities. We also show that when the class prior probability estimation is off, the results can be quite poor.

heading: 2 RELATED WORK
text: Research in PU learning started in early 2000s (Denis, 1998; Liu et al., 2002; Lee & Liu, 2003; Yu et al., 2004; Liu et al., 2003; Elkan & Noto, 2008). Denis (1998) proposed it under statistical query framework. Liu et al. (2002) studied sample complexity. Elkan & Noto (2008) showed that if ranking is the goal of PU learning rather than classification, then PU learning is equivalent to the conventional binary learning. Due to numerous applications, there has been a recent surge of interest in PU learning (du Plessis et al., 2014; Mordelet & Vert, 2014; du Plessis et al., 2015; Sechidis & Brown, 2015; Claesena et al., 2015; Chang et al., 2016; Niu et al., 2016a; Yi et al., 2017; Liu et al., 2017; Jian et al., 2017; Northcutt et al., 2017; Kiryo et al., 2017; Sakai et al., 2017; Hou et al., 2018; Xu et al., 2017; Gong et al., 2018; Bekker & Davis, 2018a; Chiaroni et al., 2018; Hsieh et al., 2018; Bao et al., 2018; Shi et al., 2018; Sansone et al., 2018; Kato et al., 2019).
Early approaches for PU learning use 2-steps. Step 1 finds some highly probable negative examples from the unlabeled set. Step 2 uses the positive set, highly probable negative set, and the remaining unlabeled set to build a classifier (Liu et al., 2002; Yu et al., 2004; Li & Liu, 2003). Liu et al. (2003); Shi et al. (2018) also regarded the unlabeled data to have noisy labels. Several techniques applied training data re-weighting via regularization as well (Lee & Liu, 2003; Elkan & Noto, 2008). Work in (du Plessis et al., 2014; 2015; Kiryo et al., 2017) avoided parameter tuning by using unbiased risk estimators. Kato et al. (2019) deals with sample selection bias.
None of these works explores the idea of adversarial learning as we do in PAN. Hou et al. (2018) used GAN to generate positive and negative examples and then used them to build the final classifier employing a separate classification model. Chiaroni et al. (2018) proposed a method to use GAN to generate negative training examples. Both papers are for image classification. Generating text and other forms of data using GAN is relatively more challenging. PAN does not generate data. It replaces the generator with a classifier to directly train the required PU classifier in the adversarial learning framework using positive and unlabeled data. Our formulation and objective function are also quite different from those of GAN. Moreover, Xu et al. (2017) and Gong et al. (2018) used traditional margin-based methods.
Our a-GAN method is also related to the studies of weighted adversarial nets (WAN) (Chen et al., 2018; Zhang et al., 2018) as a-GAN also weights the discriminator, but the proposed PAN differs significantly from WAN. That is because although WAN weights the discriminator but the adversarial training procedure is the same as the original GAN (similar to our a-GAN). Our a-GAN is also similar to the method DAN recently posted on arXiv (Liu et al., 2019) as we can see from their Eq. 5 and our Eq. 2. The training methods are slightly different. PAN takes a very different approach as we will see in the next two sections.
Other related works include leveraging biased negative examples (Hsieh et al., 2018; Sakai et al., 2017), studying the random assumption of PU learning (Bekker & Davis, 2018a), multi-instance PU learning (Bao et al., 2018), scalable PU learning (Sansone et al., 2018), etc. More details can be found in the survey (Bekker & Davis, 2018b).
A weakness with these recent state-of-the-art systems is that they need the class prior probability (du Plessis et al., 2015; Kiryo et al., 2017; Hou et al., 2018; Xu et al., 2017; Chiaroni et al., 2018;
Kato et al., 2019), which is hard for the user to provide. Although there are methods that try to estimate the class prior (Menon et al., 2015; Ramaswamy et al., 2016; Jain et al., 2016; du Plessis et al., 2017; Gong et al., 2019), we will show that if the estimate is off, the results can be quite poor.

heading: 3 BACKGROUND
text: GAN is an adversarial learner that learns a generator for generating data such as images. It composes of two networks, a generator and a discriminator. The generator generates new data instances, and the discriminator evaluates them for authenticity, i.e. deciding whether each generated data instance it reviews belongs to the actual training dataset or not. Through an iterative and adversarial process, the generator can generate new data instances that the discriminator has hard time to distinguish from the real training data. GAN is formulated as a minmax game as follows:
min G max D V(D,G) = Ex∼Pdata(x)[logD(x)] + Ez∼Pz(z)[log(1−D(G(z)))] (1)
where G(·) is the generator that aims to generate data that can approximate real data to fool the discriminator D(·), while D(·) tries its best to discriminate the generated data from real/training data. Pdata is the data generating distribution of the real data, and Pz is the data generating distribution of the generator. Through an adversarial training, we expect to learn a good generator.

heading: 3.1 DIRECT ADAPTATION OF GAN FOR PU LEARNING
text: We now present the direct adaptation of GAN (a-GAN) for PU learning. The real training data in GAN is our labeled positive data P . As illustrated in Figure 1, the discriminator D(·) in a-GAN still plays the same role as that in GAN, but the generator in a-GAN is replaced with another discriminator, which we call the classifier (or predictor) C(·). The goal of C(·) is to identify likely positives from the unlabeled set U to give to the discriminator for it to decide whether these are real positive data. The following equation shows this approach:
min C max D
V(D,C) = Exp∼Pp(xp)[logD(xp)]
+ Exs=argxu∼Pu(xu) C(xu)=1[log(1−D(x s))]
(2)
where xs denotes the example judged as a likely positive example from U by C(·). P p and Pu are the data generating distributions of the known positive data and the unlabeled data in PU learning, respectively. The known positive examples are randomly selected from the positive population. The hidden positives in the unlabeled set is also a random sample of the same positive population.
Due to the discreteness of the last term in the equation, we use the Policy Gradient method (Sutton et al., 2000) from reinforcement learning to train it, where the last term is regarded as the reward for optimizing C(·). We call this adapted version of the system a-GAN (adapted GAN). a-GAN does reasonably well (see Section 5.2). However, since it focuses on the positive data only, its learning is not balanced, which can cause some confusion with the separation of positive and negative data. Next, we present the final proposed method PAN, which emphasizes both positive and negative and is able to produce a better separation for them.

heading: 4 PROPOSED PAN
text: PAN adopts the same adversarial learning idea to build a PU classifier C(·). However, as the right part of Figure 1 shows, instead of using D(·) to directly discriminate the known positive data and the selected positive data by C(·), we propose to use the adversarial learning idea on the probability distributions of D(·) and C(·) on each example (or instance). Specifically, in the part surrounded by the red dash-lined box in Figure 1, D(·) and C(·) produce a score for each input example xui from the unlabeled set in parallel with different optimization objectives. D(·) tries to give xui the opposite prediction score to that of C(·) in order to identify it as a “fake” example; C(·) tries to give xui a similar score to that of D(·) to fool D(·). The adversarial learning is performed through a distance metric, D(·) tries to enlarge the distance with D(·) but C(·) tries to shrink the distance, which is applied to each example in the unlabeled set (no sampling is used). We choose KL-divergence as the metric, which minimizes the information loss between the two probability distributions as it has been shown to be able to learn and suit complex distributions (Kingma & Welling, 2013; Goodfellow et al., 2014). The green links in Figure 1 show the procedure of optimizing the known positive data. Note that the unlabeled data are regarded as the negative data in PAN to endow D(·) the ability to recognizing negative samples to some extent.

heading: 4.1 PREDICTIVE ADVERSARY NETWORKS (PAN)
text: Unlike GAN, which only generates positive examples that are hard to distinguish by the discriminator D(·), we also want the remaining unlabeled examples to be easily distinguishable (as possible negatives) by the discriminator. To this end, C(·) tries to separate positive and negative examples in the unlabeled data with a large margin. That is, C(·) not only gives high probabilities to examples that D(·) has difficulty to distinguish (meaning D(·) also gives high probabilities to those examples) but also low probabilities to examples that are easy to distinguish by D(·) (meaning D(·) also gives low probabilities to those examples because of the easy separation). Note, when we say C(·) or D(·) gives high/low probability, we mean the probability of being positive. We propose to achieve our goal by controlling the distance (similarity) between the predictions of C(·) and D(·) on the unlabeled set. We use the sum of KL-divergences on the predictions of all examples (or instances) in U as the distance. In detail, PAN assumes the output of D(·) (respectively, C(·)) on ith instance as a discrete distribution over binary outcomes (or classes) of positive and negative. For example, if D(·) (likewise, C(·)) gives an instance the probability of 0.3. It means that for the positive outcome or class, the probability is 0.3, and for the negative outcome, the probability is 0.7. We use Di and Ci to denote the two distributions and KL-divergence is employed to measure their distance. Superscripts pu and u denote the corresponding datasets. PAN’s objective is defined as follows:
min C max D V(D,C) = − n∑ i=1
KL(P pui ||D pu i )︸ ︷︷ ︸
I
+λ ( n0∑ i=1
KL(Dui ||Cui )︸ ︷︷ ︸ II
− n0∑ i=1
KL(Dui ||C̃ui ))︸ ︷︷ ︸ III
(3)
where P pui is the probability distribution of positive and unlabeled of the ith instance (we basically treat unlabeled as negative, which is an issue to be addressed shortly) in the given PU data Xpu (including both positive Xp and unlabeled Xu data), and n and n0 are the total numbers of training examples in Xpu and Xu respectively. C̃ui denotes the opposite distribution of C u i , i.e., 1−Cui (with a slight abuse of notation). λ is a hyper-parameter for balancing the distances.
We marked three terms in Eq. 3. The adversarial learning of Eq. 3 works as follows: The first term marked by I is to minimize the sum of the divergences between Dpui and P pu i (notice the minus sign in front). It aims to achieve the goal of helping D(·) recognize positive instances (it is necessary especially at the beginning of training). When optimizing C(·), the term marked by II minimizes the sum of the KL-divergences from Cui to D u i , indicating that C(·) tries to give the same probability to the input xui as D(·). In this case, the instances/examples getting high probabilities (chosen by C(·) as positive) can also get high probabilities from D(·). This achieves the goal of fooling D(·) by C(·). When optimizing D(·), the term marked II maximizes the sum of the KL-divergences between Dui and Cui , meaning that D(·) tries to give low probabilities to the instances that get high probabilities from C(·) in order to detect ‘fake’ positive examples, and vice versa. Using the first two terms can already perform the function of PAN. An advantage of PAN is that it can consider and optimize both positive and negative examples in the unlabeled set. However, we
show that the term marked II produces asymmetric gradient for positive and negative examples for both D(·) and C(·) in Appendix A. That means the term marked II can cause unbalanced training between positive and negative examples and lead to high precision and low recall. To this end, we propose the term marked III which can eliminate the concern (see Appendix A). Ablation study also shows the effectiveness of term III in Appendix D.3. With all three terms, we build an adversarial learning approach for PU learning through the minimizing and maximizing operations mentioned above, i.e., a minmax game between D(·) and C(·).

heading: 4.2 SIMPLIFICATION OF EQUATION 3
text: To facilitate the optimization of the objective function in Eq. 3, we use D to denote D(xpu) and C to denote C(xpu) and simplify Eq. 3 to (see Appendix B for derivations):
min C max D V(D,C) = Exp∼Pp(xp)[logD(xp)] + Exu∼Pu(xu)[log(1−D(xu))]︸ ︷︷ ︸ IV:−H(PL,D(xpu)) + λ · Exu∼Pu(xu)[(log(1− C(xu))− log(C(xu)))︸ ︷︷ ︸ V (2D(xu)− 1)]
︸ ︷︷ ︸ VI
(4)
where P p denotes the distribution of the positive data. As we marked in Eq. 4, term IV is the cross entropy loss between D(xpu) and the ground-truth label distribution P pu of the PU data, denoted by H(P pu, D(xpu)). About the term marked VI, we elaborate with the following two points:
(1). Term VI can be viewed approximately as a policy gradient reinforcement learning algorithm for training C(·) but with no sampling operation, if we regard D(xu) as the reward and term V as the policy. Clearly, if D(·) outputs a high ‘reward’ that exceeds 0.5, meaning that D(·) judges the current input as a positive instance with high probability, Eq. 4 will maximize the probability of C(·) over the current input to fit the distribution of D(·). However, if D(·) outputs a low ‘reward’ below 0.5, minimizing Eq. 4 is equivalent to minimizing the probability of C(·) over the current input. As a consequence, the distribution of C(·) is made closer to D(·). (2). The term marked with V in Eq. 4 is a comparison game between the likelihood log(C(xu)) of choosing an example or the likelihood log(1 − C(xu)) of not choosing an example xu. If the choosing probability is greater than the not choosing probability, the value of term V is less than 0. Then to optimize D(·), maximizing Eq. 4 is equivalent to minimizing the probability of D(xu). On the contrary, maximizing Eq. 4 is equivalent to maximizing the probability of D(xu). Clearly, this is an adversarial learning method: for the case that term V is less than 0, D(·) tries to distinguish examples selected by C(·) (to give low probabilities to these examples). For the case that term VII is greater than 0, D(·) tries to give high probabilities to examples not selected by C(·), which helps the system move away from local training optimal.
Analysis of the Learned Classifier (C(·)): Although the proposed PAN is quite different from original GAN, its running follows the adversarial procedure. The behaviors of optimal C(·) and D(·) are that C(·) gives the same prediction as D(·) while D(·) cannot move away from C(·), which means D(·) also give positive (or negative) scores to examples that get positive (or negative) scores from C(·).3 The reason that D(·) cannot move away from C(·) is because moving away from C(·) will let D(·) make errors on known positive data. We give theoretical analysis of the optimal decision surface learned by PAN in Appendix C. Due to the complexity of PAN, we cannot give the precise optimal decision surface expression, but we show its properties in the Appendix C.

heading: 4.3 TRAINING ALGORITHM OF PAN
text: Algorithm 1 gives the training algorithm of PAN using stochastic gradient descent for conciseness. Note, our method is not limited to using stochastic gradient descent. In this work, we use Adam for optimization. The algorithm alternately trains the discriminator D(·) and the classifier C(·). In each step or iteration, the lines between 5 to 10 (not including 10) are for training D(·) and the lines after 10 are for training C(·). The details of the algorithm are self-explanatory.
3Positive (or negative) score means the score is greater (or lower) than 0.5.
Algorithm 1 PAN training by minibatch stochastic gradient descent method. Input: given positive training data Xp; given unlabeled training data Xu; Initial: Randomly initialize D(·) and C(·); for number of training steps do
// Training D(·) k steps, we set k = 1. 5: for k steps do
• Sample a mini-batch of m positive examples {xp1, . . . ,xpm} from Xp; • Sample a mini-batch of m unlabeled examples {xu1 , . . . ,xum} from Xu; • Update D(·) by ascending its stochastic gradient:
∇θd m∑ i=1 [logD(xpi ) + log(1−D(x u i )) + λ(log(1− C(xui ))− logC(xui ))(2D(xui )− 1)]
end for 10: • Sample a mini-batch of m unlabeled examples {xu1 , . . . ,xum} from Xu;
• Update C(·) by descending its stochastic gradient: ∇θc m∑ i=1 [λ(log(1− C(xui ))− logC(xui ))(2D(xui )− 1)]
end for

heading: 5 EXPERIMENTS
text: We now evaluate the proposed technique PAN and compare it with state-of-the-art baselines.Three text and two image classification datasets are used in our experiments.
(1). YELP: a collection of online reviews from Yelp. Each review is labeled with a star rating ranging from 1 to 5. The dataset is extracted from the Yelp Dataset Challenge 2015. (2). RT: a collection of online reviews from rotten tomatoes with sentiment labels good and bad. (3). IMDB: another collection of online review for binary sentiment classification. (4). 20News: a collection of about 20,000 newsgroup documents, partitioned (nearly) evenly across 20 different news topics. (5). MNIST: a collection of 70,000 images of handwritten digits from 0 to 9. (6). CIFAR10: a collection of 60000 32x32 colour images of 10 classes, with 6000 images per class. See 4 for all datasets.

heading: 5.1 EXPERIMENT SETTINGS
text: Data Preparation: Since the five datasets are for traditional supervised learning with class labels, we need to prepare positive P and unlabeled U data for PU learning. We use two steps, after which we obtain the training and testing data for each dataset (on the left of the dataset name in Table 1).
Step 1 - Constructing positive and negative data. As not all datasets have 2 classes, we need to make each of them a two-class (positive and negative) dataset. RT and IMDB are already two-class datasets. For YELP, which has 5 classes, we remove the reviews with the class label of 3-stars, and split the remaining classes into two: one as the positive data (4 or 5 stars) and the other as the negative data (1 or 2 stars) (this is commonly done for sentiment classification (Pang & Lee, 2008)). Following the baseline (Kiryo et al., 2017), for 20News, topics ‘alt.’, ‘comp.’, ‘misc.’, and ‘rec.’ form the positive data, and topics ‘sci.’, ‘soc.’ and ‘talk.’ form the negative data. For MNIST, all images labeled with even numbers form the positive data and all images labeled with odd numbers form the negative data. For CIFAR10, classes airplane, automobile, ship and truck are used as the positive data and the rest as the negative data.
Step 2 - Creating PU learning datasets. After step 1, we get positive and negative training data for each dataset. We then build the PU learning training dataset, which includes positive and unlabeled data as follows. For each dataset (except CIFAR10), we randomly select 10% (5% for CIFAR10 for diversity). We also show more results by varying the ratio in Appendix D.1, the percent of positive
4See http://www.yelp.com/dataset challenge for YELP, http://www.cs.cornell.edu/people/pabo/movie-review-data/ for RT, https://www.imdb.com/interfaces/ for IMDB, http://qwone.com/∼jason/20Newsgroups/ for 20NEWS, http://yann.lecun.com/exdb/mnist/ for MNIST, http://www.cs.toronto.edu/ kriz/cifar-10-python.tar.gz for CIFAR10.
examples from the whole positive set as the known positive data P for PU learning. The unlabeled data U consists of the negative data and the remaining positive data in the dataset.5
Baselines: We use our a-GAN and five state-of-the art representative approaches as the baselines.
a-GAN. This is the direct adaptation of GAN given in Section 3.
UPU (du Plessis et al., 2015). This method proposed a general unbiased estimator that is also convex for loss functions meeting certain linear-odd conditions.
NNPU (Kiryo et al., 2017). This is a non-negative risk estimator for PU learning. When minimized, it is more robust against overfitting, and is able to use flexible models even given limited P data. Note that NNPU has two versions, the linear and the MLP versions. We give the results of the MLP version as it does better.
NNPUSB (Kato et al., 2019). This is a recent algorithm that extended NNPU with an additional mechanism for handling sample selection bias.
GenPU (Hou et al., 2018): This system uses the GAN framework and an array of generators and discriminators to generate both positive and negative data for PU learning.
PMPU (Gong et al., 2018): this is a traditional SVM based PU learning method.
It is important to note that both UPU and NNPU need the input of the class prior probability, which is often not available in practice. In our experiments, we provide them the correct class prior probabilities. Even with this favorable condition, they are still weaker than PAN, which does not need the class prior probability input. For UPU and NNPU, we use the opensource code from the authors and a third party6, respectively. For NNPUSB, we use the original code provided by the authors. Note also we use the same network as these baselines, including architecture, number of parameters, and the optimization method. We also give them exactly the same positive and unlabeled data and the test data. For GenPU, we again use the code provided by the authors. For PGAN’s results mentioned above, since there is no source code available, we used the best results reported in the paper.
Training Details: For fair comparison, we use the same classifier C for all systems following the baselines. For text, a 2-layer convolutional network (CNN), with 5 * 100 and 3 * 100 convolutions for layers 1 and 2 respectively, and 100 filters for each layer, is used as the classifier C(·) and discriminator D(·). The word embeddings are also trained by the system. For MNIST, the classifier is a 3-layer MLP (with 2 hidden layers, more specifically, d-512-256-1) as it is fairly simple. The classifier for the CIFAR10 dataset was an all convolutional net: (32 × 32 × 3)-[C(3 × 3, 96)] - C(3 × 3, 96, 2) - [C(1 × 1, 192)] - C(1 × 1, 10) - 1000 - 1000 - 1, where each input is a 32 × 32 RGB image, C(3×3, 96) means 96 channels of 3×3 convolutions followed by ReLU, C(3 × 3, 96, 2) means a similar layer but with stride 2, etc. We set λ in Eq. 3 and 4 to 0.0001, please see more details in Appendix D.2. We also balance the impact of positive and unlabeled data for term I in Eq. 3 in training; otherwise the positive examples will be dominated by the unlabeled data. We use 1:1 ratio of positive data and unlabeled data in each mini-batch in training. The network parameters are updated using the Adam algorithm with a learning rate of 0.0001. For baseline a-GAN, it needs pre-training of D(·). We use the original positive and unlabeled (regarded as negative) data to pre-train D(·) in order to enable its ability to classify positive and unlabeled data. We pre-train D(·) 3 epochs for the datasets.
5Note that PU learning has two data sampling settings, we use the one-pass (Niu et al., 2016b) or singletraining-set (Elkan & Noto, 2008) setting, not the case-controlled or two-pass (Ward et al., 2009) setting.
6https://github.com/GarrettLee/nnpu tf

heading: 5.2 RESULTS AND ANALYSIS
text: Figures 2-7 show the test accuracy, precision and recall in each epoch of each method. The usual first-order exponential weighted moving average smoothing with weight 0.7 is applied to the figures. Note that due to space limitations, we only show the curves of 100 epochs, but the final results in Table 1 are produced with more epochs as NNPU takes slightly longer to reach the peak (see below).
The final accuracy and F-score results of A-GAN, UPU, NNPU, NNPUSB and PAN are given in Table 1 (on the right side of dataset names) and the results for GenPU and PMPU are given Table 2. Since different epochs give different results, for a fair comparison, we give the average of both the best F-score (F) and best accuracy (Acc) for each system on each dataset over 200 epochs (all systems converged before 200 epochs) over 5 runs. The F-score is measured on the positive data/class, as in PU learning the user is normally interested in identifying the positive data. The last row in the table gives the average result for each column.
From the Figures and Table 1, we can make the following observations.
(1). From the results in Table 1 (on the right side of dataset names in the table), we see that PAN outperforms all baselines on all datasets (the last row gives the average of each algorithm for all datasets). Among the baselines, NNPU and NNPUSB are the strongest and their results are very similar. PAN outperforms them markedly. Given that PAN does not need class prior probability input, this is even more significant. The direct adaptation of GAN a-GAN is weaker than both NNPU and PAN, but is better than UPU. Although NNPUSB extends NNPU, it did not do better than NNPU. The reason could be that our data do not have sample selection bias, which NNPUSB tries to address.
(2). Figures 2 to 7 show that PAN and NNPUSB are robust across all datasets with both high precision and high recall. And clearly PAN is better than NNPUSB. NNPU is rather unbalanced for precision and recall for YELP and RT, either very high precision but very low recall, or vice versa. a-GAN and UPU have the same problem, which is highly undesirable. PAN also outperforms baselines consistently in accuracy for all six datasets.
(3). From the 6 figures, we also see that a-GAN is unstable. Precision, recall, and accuracy fluctuate greatly from one epoch to another. Stability problem of GAN is well documented (Metz et al., 2017; Berthelot et al., 2017). Our adaptation requiring reinforcement learning to train it is likely to have made the problem worse.
(4). NNPU converges slowly (Figure 3). It didn’t converge even at 100 epoch. We report its best accuracy and F-score in Table 1 in 200 epochs (it converged earlier than 200).
Table 2 shows the results of GenPU and PMPU. Since GenPU’s data generator cannot generate text data and thus no results for the text datasets. We can see that GenPU’s results are dramatically worse. Hou et al. (2018) showed that GenPU does well with few classes as the positive and negative, e.g, 1 class as positive and 1 class as negative. However, in our case, both positive and negative consists of many classes, which probably make GenPU work poorly. PMPU is also significantly poorer than PAN.
In summary, we can conclude that PAN markedly outperforms the baselines in accuracy, F-score, robustness, and stability. Given that PAN does not need the class priors, this is more significant.
Varying Positive Data and NNPU’s Sensitivity to Class Prior: We use MNIST and CIFAR10 as representatives to study these issues. For each dataset, we randomly select 1 or 2 classes in the original data to form the positive set, and the rest to form the negative set to generate 2 PU learning datasets as discussed above. Since both MNIST and CIFAR have 10 classes, for the 1-class positive PU data, the class prior probability is 10% for positive and 90% for negative (or 1:9 for short). For the 2-class positive PU data, it is 20% for positive and 80% for negative (or 2:8). The results are given in Table 2. We see similar improvements from PAN with the exact class prior given to NNPU
(1:9 or 2:8). For the 1:9 (respectively, 2:8) case for both MNIST and CIFAR10, if we change the class prior from the correct 1:9 (2:8) to the wrong 2:8 (3:7), NNPU’s result drops are small (not in Table 2). So NNPU has some robustness. However, if we change to the wrong 3:7 or 4:6 (for the correct 1:9), and 4:6 or 5:5 (for the correct 2:8), the drops are dramatic for MNIST. For CIFAR10, they are smaller, even a small increase in F for the wrong 4:6 (correct 2:8), likely an anomaly as this data is hard, but still poorer than PAN. We conclude although it is possible to estimate the class prior (see Section 2), if the estimate is off, the results can be quite poor.

heading: 6 CONCLUSIONS
text: This paper proposed a new GAN style PU learning method PAN based on adversarial training. PAN is also significantly different from GAN as PAN does not use a generator but a classifier in its place. The objective function of PAN is also entirely different as it is based on KL-divergence. PAN represents a new way to do PU learning. Empirical evaluation using both text and image datasets showed that PAN outperformed the state-of-the-art baselines. Also importantly, PAN obtained the better results without using any class prior probability information.

heading: B SIMPLIFICATION
text: Recall the loss function Eq. 3:
min C max D V(D,C) = − n∑ i=1
KL(P pui ||D pu i )︸ ︷︷ ︸
I
+λ 1©︷ ︸︸ ︷ (
n0∑ i=1
KL(Dui ||Cui )︸ ︷︷ ︸ II
− n0∑ i=1
KL(Dui ||C̃ui ))︸ ︷︷ ︸ III
(8)
KL-divergence is defined as: KL(P ||Q) = ∑ x∈X P (x) logP (x)− P (x) logQ(x) (9)
X denotes the probability space, it is 1 or 0 (X = {1, 0}) in our scenario. We first address term I in Eq. 8, if we use D to denote Dpui (1) (the probability for ith instance being positive judged by discriminator) and P to denote P pui (1), then D pu i (0) = 1−D and P pu i (0) = 1− P . Then we have:
−KL(P pui ||D pu i ) = −P logP + P logD − (1− P ) log(1− P ) + (1− P ) log(1−D) = P logD + (1− P ) log(1−D)
(10)
Due to the fact that P pui (0) = 0 and P pu i (1) = 1 if the ith instance is positive and P pu i (1) = 0 and P pui (0) = 1 if the ith instance is unlabeled, we can rewrite the result as: Exp∼Pp(xp)[logD(xp)] + Exu∼Pu(xu)[log(1−D(xu))] (11)
Similarly, for term 1© in Eq. 8, if we use D to denote Dui (1) and C to denote Cui (1), then we get: KL(Dui ||Cui )−KL(Dui ||(1− Cui )) = D logD −D logC + (1−D) log(1−D)− (1−D) log(1− C)−D logD +D log(1− C)− (1−D) log(1−D) + (1−D) logC
= −D logC − (1−D) log(1− C) +D log(1− C) + (1−D) logC = (log(1− C)− logC)(2D − 1)
(12)
Then we have: (log(1− C)− logC)(2D − 1) = Exu∼Pu(xu)[(log(1− C(xu))− log(C(xu)))(2D(xu)− 1)] (13)
Combining Eqs. 11 and 13, we get Eq. 14: min C max D V(D,C)
= Exp∼Pp(xp)[logD(xp)] + Exu∼Pu(xu)[log(1−D(xu))]︸ ︷︷ ︸ IV:−H(PL,D(xpu)) + λ · Exu∼Pu(xu)[(log(1− C(xu))− log(C(xu)))︸ ︷︷ ︸ V (2D(xu)− 1)]
︸ ︷︷ ︸ VI
(14)
where P p denotes the distribution of the positive data.

heading: C THEORETICAL ANALYSIS ABOUT THE LEARNED CLASSIFIER
text: In this section, we analyze the properties of the learned classifier and show why Eq. 3 can perform PU learning. Intuitively, from Eq. 8 (same as Eq. 3 in the paper), we can see that D(·) is biased if we only consider term I because the unlabeled set contains both positive and negative examples. However, terms II and III help correct the bias. Eq. 14 (same as Eq. 4 in the paper), which is derived from Eq. 8 for training, shows the property more clearly than Eq. 8. Notice that the bias in term I in Eq. 8 will result in high precision and low recall for the positive class. Now back to Eq. 14 and let us imagine that most examples in the unlabeled set are regarded as negative by C(·) (meaning low recall). From Eq. 14, we can see that the value of term V will be below zero. But when optimizing D(·), term VI will push D(·) up for these data points, and thus the bias is reduced and the low recall problem is mitigated because in the next optimization iteration, C(·) will follow D(·) to go up for these data points.
We now theoretically discuss the optimal decision surface of the classifier C(·) learned by the proposed PAN.
Proposition 1. Let T (x) = log[1 − C(x)] − log[C(x)], ε(x) = f(T (x)), the learned optimal decision surface of C(·) is:
ε(x) = 1 2 − P
p(x)
P p(x) + Pu(x) (15)
f(·) is a type of function that satisfies ε(x) · T (x) > 0. From Proposition 1, we can see that PAN finds the decision surface by combining P p(x) and Pu(x). The combination is controlled by ε(x). ε(x) is a function of C(x).
C.1 PROOF OF PROPOSITION 1
Proof: The training criterion for discriminator D, given any classifier C, is to maximize the quantity V (C,D),
V (C,D) = ∫ x P p(x) logD(x)dx+ ∫ x Pu(x) log(1−D(x))dx
+λ ∫ x Pu(x)[log(1− C(x))− logC(x)](2D(x)− 1)dx (16)
Clearly, the maximum point appears at the point with derivative 0. Then we calculate the partial derivative of V (C,D) to D and get:
P p(x)
D(x) − P
u(x)
1−D(x)︸ ︷︷ ︸ a© +λPu(x)[log(1− C(x))− logC(x)]︸ ︷︷ ︸ b© = 0 (17)
Directly computing the solution is complex. If we omit term b© for the time being, the solution for Eq. 17 is D(x) = P
p(x) Pp(x)+Pu(x) . After bringing back b©, this solution should be revised as follows.
With the definition of T (x) = log[1− C(x)]− log[C(x)] and ε(x) = f(T (x)), we can re-write the solution after revision:
D∗(x) = P p(x)
P p(x) + Pu(x) + ε(x) (18)
where ε(x) is a function of T (x) since P p(x) and Pu(x) are decided by the dataset and ε(x) changes with the change of T (x). The exact expression of ε(x) is difficult but we can show ε(x) ∝ T (x) in our case. In Eq. 17, term a© decreases monotonously when D(x) ∈ (0, 1),7 and both λ and P p(x) are greater than 0. In this case, if T (x) > 0 (T (x) < 0), to keep Eq. 17 equal to 0, D(x) must move toward the positive (negative) direction, which indicates ε(x) > 0 (ε(x) < 0). Formally, we have:
ε(x) ∝ T (x); ε(x)T (x) > 0 (19)
Note that the training objective of D can be interpreted as using the training data (P p(x) and Pu(x)) to find a discrimination bound and utilizing the learned knowledge in C to adapt it. The mini-max game in Eq. 4 can now be reformulated as:
L(C) =max D V(C,D)
=Ex∼Pu(x)[(log(1− C(x))− log(C(x)))(2D∗(x)− 1)]
=Ex∼Pu(xu)[T (x)( 2P p(x)
P p(x) + Pu(x) + 2ε(x)− 1)]
(20)
Clearly, because the range of T (x) is (− , ), L(C) achieves its minimum when T (x) and (2D∗(x)− 1) have opposite signs.8 Then, the optimal T ∗(x) satisfies:
T ∗(x)[ 2P p(x)
P p(x) + Pu(x) + 2ε(x)− 1] < 0 (21)
As we introduced in Footnote 5, C(x) ∈ (0, 1). In this case, we use C(x) = 0.5 as the decision surface to perform classification. Clearly, this decision surface equals to T ∗(x) = log(1− 0.5)− log(0.5) = 0. In summary, we get the optimal decision surface:
ε(x) = 1 2 − P
p(x)
P p(x) + Pu(x) (22)
7We force D(x) to satisfy the condition by adding a Sigmoid function to the end of D. We also force the output range of C into (0, 1) using the same method.
8The original range of T (x) should be (−∞,+∞). However, such range can cause stability problems for training. We adopt a standard trick, i.e., adding a small value to the log function, e.g., log(C(x) + 1e−8), to change the range of T (x) to (−8, 8).

heading: D MORE EXPERIMENTS AND ANALYSIS
text: In the paper we showed that PAN gets significant improvement comparing with state-of-the-art baselines. Here we give more detailed analysis of PAN in terms of dealing with varied positive ratio and the selection of hyper-parameter λ in Eq. 3 (also 4).
D.1 VARYING KNOWN POSITIVE RATIO
In this section, we analyse the performance of PAN when dealing with varied ratios of known positive examples. We vary the ratio of known positive examples from 5% to 30%, and show the accuracy and F-score of PAN and baseline on MNIST and CIFAR10 datasets. To verify the ability of PAN working in extreme conditions, we tested PAN on the situation with only 1% positive examples. The results are reported in Table 4 and 5.
From Table 4 and 5, we can see that PAN can do well with different proportions of known positive data and with extremely few known positive examples. We note that that the margin between PAN and NNPU goes large with the decrease of the ratio of known positive examples, which indicates that PAN is more effective than NNPU. The margin is smaller when the known positive ratio enlarges, that is because if we have enough positive data, the limitation of getting better results is no longer the PU learning method, but the performance of the classifier.
D.2 HYPER-PARAMETER SELECTION
λ is the hyper-parameter that balances the KL-divergences. Here, we show that λ should be a small value but it is not too sensitive when it is around 0.0001. In our case, we set it to 0.0001.
D.3 ABLATION STUDY FOR TERM MARKED BY III IN EQ. 3
Table 8: F-score comparison on different datasets with and without term III.
Model YELP RT IMDB 20News MNIST CIFAR10 PAN without term III 81.86 66.28 78.52 73.59 96.27 86.68
PAN full model 83.45 66.58 77.10 81.06 96.51 87.22
Table 7 and 8 show PAN’s ablation results in accuracy and F-score with or without term III respectively. From the two tables, we can see that adding term III indeed improves the performance of PAN on 5 out of 6 datasets.

Review summaries:
The paper proposed an interesting idea of using two adversarial classifiers for PU learning. The first classifier tries to introduce samples from unlabeled data that are similar to the existing labeled positive data, and the second one tries to detect if a sample has a ground truth label or drawn from unlabeled data by the first classifier (fake). The idea of the paper is interesting; it is well-motivated and well supported with a range of experiments from NLP and computer vision tasks. The paper is a good read but required a pass of proofreading; some rearrangement of the concepts (for example, in second paragraph C(.) and D(.) is used, but they are introduced properly in section 4. Also, the paper could use some clarifications.