L EARNING FROM P OSITIVE AND U NLABELED D ATA WITH A DVERSARIAL T RAINING
Anonymous authors
Paper under double-blind review
A BSTRACT
Positive-unlabeled (PU) learning learns a binary classiﬁer using only positive and unlabeled examples without labeled negative examples.
This paper shows that the GAN (Generative Adversarial Networks) style of adversarial training is quite suitable for PU learning.
GAN learns a generator to generate data (e g., images) to fool a discriminator which tries to determine whether the generated data belong to a (positive) training class.
PU learning is similar and can be naturally casted as trying to identify (not generate) likely positive data from the unlabeled set also to fool a discriminator that determines whether the identiﬁed likely positive data from the unlabeled set ( U ) are indeed positive ( P ).
A direct adaptation of
GAN for PU learning does not produce a strong classiﬁer.
This paper proposes a more effective method called Predictive Adversarial Networks (PAN) using a new objective function based on KL-divergence, which performs much better.
Empirical evaluation using both image and text data shows the effectiveness of PAN.
I NTRODUCTION
Positive-unlabeled (PU) learning learns a binary classiﬁer from only Positive ( P ) and Unlabeled ( U ) examples with no labeled negative examples (Denis, 1998;
Liu et al., 2002; Elkan & Noto, 2008; du Plessis et al., 2014; Bekker & Davis, 2018b).
It has many applications in text analysis, biomedicine, spam detection, recommendation, remote sensing, matrix and knowledge base completion, graph learning, etc (Li & Liu, 2003;
Yu et al., 2004; Fusilier et al., 2015; Scott & Blanchard, 2009;
Li et al., 2010; Cerulo et al., 2010; Calvo et al., 2007; Ren et al., 2015; Hsieh et al., 2015; Wu et al., 2017; Zupanc & Davis, 2018).
This paper proposes an adversarial PU learning method inspired by GAN (generative adversary networks) (Goodfellow et al., 2014), as we found that the GAN framework naturally suits PU learning.
GAN aims to generate data of a particular training class, which is like the positive class P in PU learning.
GAN works by generating likely positive data using a generator G ( · ) to fool a discriminator D ( · ) , which determines whether the generated data indeed belong to the training (positive) class.
For PU learning, this is like choosing likely positive instances from the unlabeled set U (achieved by a classiﬁer) also to fool a discriminator D ( · ) .
Thus, we can simply replace GAN’s generator with a classiﬁer C ( · ) to produce a PU learner.
1 Note that the classiﬁer is also a discriminator, but we use the term classiﬁer here to distinguish it from the original discriminator of GAN.
Note also both C ( · ) and D ( · ) are neural networks.
This paper ﬁrst proposes a direct adaptation of GAN, called a-GAN ( adapted GAN ), for PU learning following the above idea (see Section 3).
However, this simple method is unable to produce stateof-the-art results.
It led us to propose a new objective function based on Kullback-Leibler (KL) divergence.
Our ﬁnal proposed method is called PAN ( Predictive Adversary Networks ) due to the use of the classiﬁer to replace the generator in GAN along with a new adversarial training method.
On the new objective function, it is clearly desirable to consider the overall performance of C ( · ) on the entire unlabeled set.
We use a distance metric to measure whether C ( · ) can produce similar predictions to those of D ( · ) for all the examples in U .
If C ( · ) gives similar predictions, it means that the examples obtaining high probabilities from C ( · ) also get high probabilities from D ( · ) , achieving the goal of fooling D ( · ) , which will give us a good ﬁnal PU classiﬁer.
We employ KL-divergence as the distance metric to measure the difference between C ( · ) and D ( · ) .
One important advantage of PAN is that it does not need the input of class prior probability, which many state-of-the-art systems need (see Section 2).
In practice, the class prior is unknown (there are some existing methods to estimate it, see below).
Hou et al.
and Chiaroni et al.
(2018) have employed GAN to generate positive and/or negative data and then use a separate learner (e.g.,
CNN) to learn the ﬁnal PU classiﬁer using the generated data, they are not adaptations of GAN like PAN and their generators generate only images.
PAN can be applied to any data as it has no generator.
We evaluate PAN using both text and image classiﬁcation datasets and show it outperforms start-of-the-art PU learning baselines even when we give them the perfect class prior probabilities.
We also show that when the class prior probability estimation is off, the results can be quite poor.
R ELATED W ORK
Research in PU learning started in early 2000s (Denis, 1998; Liu et al., 2002; Lee & Liu, 2003;
Yu et al., 2004; Liu et al., 2003; Elkan & Noto, 2008).
Denis (1998) proposed it under statistical query framework.
Liu et al.
(2002) studied sample complexity.
Elkan & Noto (2008) showed that if ranking is the goal of PU learning rather than classiﬁcation, then PU learning is equivalent to the conventional binary learning.
Due to numerous applications, there has been a recent surge of interest in PU learning (du Plessis et al., 2014; Mordelet & Vert, 2014; du Plessis et al., 2015; Sechidis &
Brown,
[Table]
Early approaches for PU learning use 2-steps.
Step 1 ﬁnds some highly probable negative examples from the unlabeled set.
Step 2 uses the positive set, highly probable negative set, and the remaining unlabeled set to build a classiﬁer (Liu et al., 2002; Yu et al., 2004; Li & Liu, 2003).
Liu et al.
Shi et al.
(2018) also regarded the unlabeled data to have noisy labels.
Several techniques applied training data re-weighting via regularization as well (Lee & Liu, 2003; Elkan & Noto, 2008).
Work in (du Plessis et al., 2014; 2015; Kiryo et al., 2017) avoided parameter tuning by using unbiased risk estimators.
Kato et al.
(2019) deals with sample selection bias.
None of these works explores the idea of adversarial learning as we do in PAN.
Hou et al.
(2018) used GAN to generate positive and negative examples and then used them to build the ﬁnal classiﬁer employing a separate classiﬁcation model.
Chiaroni et al.
(2018) proposed a method to use GAN to generate negative training examples.
Both papers are for image classiﬁcation.
Generating text and other forms of data using GAN is relatively more challenging.
PAN does not generate data.
It replaces the generator with a classiﬁer to directly train the required PU classiﬁer in the adversarial learning framework using positive and unlabeled data.
Our formulation and objective function are also quite different from those of GAN.
Moreover, Xu et al.
(2017) and Gong et al.
(2018) used traditional margin-based methods.
Our a-GAN method is also related to the studies of weighted adversarial nets (WAN) (Chen et al., 2018; Zhang et al., 2018) as a-GAN also weights the discriminator, but the proposed PAN differs signiﬁcantly from WAN.
That is because although WAN weights the discriminator but the adversarial training procedure is the same as the original GAN (similar to our a-GAN).
Our a-GAN is also similar to the method DAN recently posted on arXiv (Liu et al., 2019) as we can see from their Eq.
5 and our
Eq. 2.
The training methods are slightly different.
PAN takes a very different approach as we will see in the next two sections.
Other related works include leveraging biased negative examples (Hsieh et al., 2018;
Sakai et al., 2017), studying the random assumption of PU learning (Bekker & Davis, 2018a), multi-instance PU learning (Bao et al., 2018), scalable PU learning (Sansone et al., 2018), etc.
More details can be found in the survey (Bekker & Davis, 2018b).
A weakness with these recent state-of-the-art systems is that they need the class prior probability (du Plessis et al., 2015; Kiryo et al., 2017;
Hou et al., 2018;
Xu et al., 2017; Chiaroni et al., 2018;
[Figure]
Kato et al., 2019), which is hard for the user to provide.
Although there are methods that try to estimate the class prior (Menon et al., 2015; Ramaswamy et al., 2016; Jain et al., 2016; du Plessis et al., 2017;
Gong et al., 2019), we will show that if the estimate is off, the results can be quite poor.
B ACKGROUND
GAN is an adversarial learner that learns a generator for generating data such as images.
It composes of two networks, a generator and a discriminator .
The generator generates new data instances, and the discriminator evaluates them for authenticity, i.e. deciding whether each generated data instance it reviews belongs to the actual training dataset or not.
Through an iterative and adversarial process, the generator can generate new data instances that the discriminator has hard time to distinguish from the real training data.
GAN is formulated as a minmax game as follows:
[Equation]
where G ( · ) is the generator that aims to generate data that can approximate real data to fool the discriminator D ( · ) , while D ( · ) tries its best to discriminate the generated data from real/training data.
P data is the data generating distribution of the real data, and P z is the data generating distribution of the generator.
Through an adversarial training, we expect to learn a good generator.
3. D IRECT A DAPTATION OF GAN FOR PU L EARNING
We now present the direct adaptation of GAN (a-GAN) for PU learning.
The real training data in
GAN is our labeled positive data P .
As illustrated in Figure 1, the discriminator D ( · ) in a-GAN still plays the same role as that in GAN, but the generator in a-GAN is replaced with another discriminator, which we call the classiﬁer (or predictor ) C ( · ) .
The goal of C ( · ) is to identify likely positives from the unlabeled set U to give to the discriminator for it to decide whether these are real positive data.
The following equation shows this approach:
[Equation]
where x s denotes the example judged as a likely positive example from U by C ( · ) .
P p and P u are the data generating distributions of the known positive data and the unlabeled data in PU learning, respectively.
The known positive examples are randomly selected from the positive population.
The hidden positives in the unlabeled set is also a random sample of the same positive population.
Due to the discreteness of the last term in the equation, we use the Policy Gradient method (Sutton et al., 2000) from reinforcement learning to train it, where the last term is regarded as the reward for optimizing C ( · ) .
We call this adapted version of the system a-GAN ( adapted GAN ).
a-GAN does reasonably well (see Section 5.2).
However, since it focuses on the positive data only, its learning is not balanced, which can cause some confusion with the separation of positive and negative data.
Next, we present the ﬁnal proposed method PAN, which emphasizes both positive and negative and is able to produce a better separation for them.
P ROPOSED PAN
PAN adopts the same adversarial learning idea to build a PU classiﬁer C ( · ) .
However, as the right part of Figure 1 shows, instead of using D ( · ) to directly discriminate the known positive data and the selected positive data by C ( · ) , we propose to use the adversarial learning idea on the probability distributions of D ( · ) and C ( · ) on each example (or instance).
Speciﬁcally, in the part surrounded by the red dash-lined box in Figure 1, D ( · ) and C ( · ) produce a score for each input example x ui from the unlabeled set in parallel with different optimization objectives.
D ( · ) tries to give x ui the opposite prediction score to that of C ( · ) in order to identify it as a “fake” example; C ( · ) tries to give x ui a similar score to that of D ( · ) to fool D ( · ) .
The adversarial learning is performed through a distance metric, D ( · ) tries to enlarge the distance with D ( · ) but C ( · ) tries to shrink the distance, which is applied to each example in the unlabeled set (no sampling is used).
We choose KL-divergence as the metric, which minimizes the information loss between the two probability distributions as it has been shown to be able to learn and suit complex distributions (Kingma & Welling, 2013; Goodfellow et al., 2014).
The green links in Figure 1 show the procedure of optimizing the known positive data.
Note that the unlabeled data are regarded as the negative data in PAN to endow D ( · ) the ability to recognizing negative samples to some extent.
4. P REDICTIVE A DVERSARY N ETWORKS (PAN)
Unlike GAN, which only generates positive examples that are hard to distinguish by the discriminator
D ( · ) , we also want the remaining unlabeled examples to be easily distinguishable (as possible negatives) by the discriminator.
To this end, C ( · ) tries to separate positive and negative examples in the unlabeled data with a large margin.
That is, C ( · ) not only gives high probabilities to examples that D ( · ) has difﬁculty to distinguish (meaning D ( · ) also gives high probabilities to those examples) but also low probabilities to examples that are easy to distinguish by D ( · ) (meaning D ( · ) also gives low probabilities to those examples because of the easy separation).
Note, when we say C ( · ) or
D ( · ) gives high/low probability, we mean the probability of being positive.
We propose to achieve our goal by controlling the distance (similarity) between the predictions of C ( · ) and D ( · ) on the unlabeled set.
We use the sum of KL-divergences on the predictions of all examples (or instances) in
U as the distance.
In detail, PAN assumes the output of D ( · ) (respectively, C ( · ) ) on i th instance as a discrete distribution over binary outcomes (or classes) of positive and negative .
For example, if D ( · ) (likewise, C ( · ) ) gives an instance the probability of 0.3.
It means that for the positive outcome or class, the probability is 0.3, and for the negative outcome, the probability is 0.7.
We use D i and C i to denote the two distributions and KL-divergence is employed to measure their distance.
Superscripts pu and u denote the corresponding datasets.
PAN’s objective is deﬁned as follows:
[Equation]
where P pui is the probability distribution of positive and unlabeled of the i th instance (we basically treat unlabeled as negative, which is an issue to be addressed shortly) in the given PU data X pu (including both positive X p and unlabeled X u data), and n and n 0 are the total numbers of training examples in X pu and X u respectively.
(cid:101) C ui denotes the opposite distribution of C ui , i.e., 1 − C ui (with a slight abuse of notation).
λ is a hyper-parameter for balancing the distances.
We marked three terms in Eq.
The adversarial learning of Eq.
3 works as follows:
The ﬁrst term marked by I is to minimize the sum of the divergences between D pui and P pui (notice the minus sign in front).
It aims to achieve the goal of helping D ( · ) recognize positive instances (it is necessary especially at the beginning of training).
When optimizing C ( · ) , the term marked by II minimizes the sum of the KL-divergences from C ui to D ui , indicating that C ( · ) tries to give the same probability to the input x ui as D ( · ) .
In this case, the instances/examples getting high probabilities (chosen by C ( · ) as positive) can also get high probabilities from D ( · ) .
This achieves the goal of fooling D ( · ) by C ( · ) .
When optimizing D ( · ) , the term marked II maximizes the sum of the KL-divergences between D ui and C ui , meaning that D ( · ) tries to give low probabilities to the instances that get high probabilities from C ( · ) in order to detect ‘fake’ positive examples, and vice versa.
Using the ﬁrst two terms can already perform the function of PAN.
An advantage of PAN is that it can consider and optimize both positive and negative examples in the unlabeled set.
However, we show that the term marked II produces asymmetric gradient for positive and negative examples for both D ( · ) and C ( · ) in Appendix A.
That means the term marked II can cause unbalanced training between positive and negative examples and lead to high precision and low recall.
To this end, we propose the term marked III which can eliminate the concern (see Appendix A).
Ablation study also shows the effectiveness of term III in Appendix D.3.
With all three terms, we build an adversarial learning approach for PU learning through the minimizing and maximizing operations mentioned above, i.e., a minmax game between D ( · ) and C ( · ) .
4. S IMPLIFICATION OF E QUATION
To facilitate the optimization of the objective function in Eq. 3, we use D to denote D ( x pu ) and C to denote C ( x pu ) and simplify Eq.
3 to (see Appendix B for derivations):
[Equation]
where P p denotes the distribution of the positive data.
As we marked in Eq. 4, term IV is the cross entropy loss between D ( x pu ) and the ground-truth label distribution P pu of the PU data, denoted by
H ( P pu , D ( x pu )) .
About the term marked VI, we elaborate with the following two points:
Term VI can be viewed approximately as a policy gradient reinforcement
learning algorithm for training C ( · ) but with no sampling operation, if we regard D ( x u ) as the reward and term V as the policy.
Clearly, if D ( · ) outputs a high ‘reward’ that exceeds 0.5, meaning that D ( · ) judges the current input as a positive instance with high probability, Eq. 4 will maximize the probability of C ( · ) over the current input to ﬁt the distribution of D ( · ) .
However, if D ( · ) outputs a low ‘reward’ below 0.5, minimizing Eq.
4 is equivalent to minimizing the probability of C ( · ) over the current input.
As a consequence, the distribution of C ( · ) is made closer to D ( · ) .
The term marked with V in Eq.
4 is a comparison game between the likelihood log( C ( x u )) of choosing an example or the likelihood log(1 − C ( x u )) of not choosing an example x u .
If the choosing probability is greater than the not choosing probability, the value of term V is less than 0.
Then to optimize D ( · ) , maximizing Eq.
4 is equivalent to minimizing the probability of D ( x u ) .
On the contrary, maximizing Eq. 4 is equivalent to maximizing the probability of D ( x u ) .
Clearly, this is an adversarial learning method: for the case that term V is less than 0, D ( · ) tries to distinguish examples selected by C ( · ) (to give low probabilities to these examples).
For the case that term VII is greater than 0, D ( · ) tries to give high probabilities to examples not selected by C ( · ) , which helps the system move away from local training optimal.
Analysis of the Learned Classiﬁer ( C ( · ) ):
Although the proposed PAN is quite different from original GAN, its running follows the adversarial procedure.
The behaviors of optimal C ( · ) and
D ( · ) are that C ( · ) gives the same prediction as D ( · ) while D ( · ) cannot move away from C ( · ) , which means D ( · ) also give positive (or negative) scores to examples that get positive (or negative) scores from C ( · ) .
The reason that D ( · ) cannot move away from C ( · ) is because moving away from C ( · ) will let D ( · ) make errors on known positive data.
We give theoretical analysis of the optimal decision surface learned by PAN in Appendix C. Due to the complexity of PAN, we cannot give the precise optimal decision surface expression, but we show its properties in the Appendix C.
4. T RAINING A LGORITHM OF PAN
Algorithm 1 gives the training algorithm of PAN using stochastic gradient descent for conciseness.
Note, our method is not limited to using stochastic gradient descent.
In this work, we use Adam for optimization.
The algorithm alternately trains the discriminator D ( · ) and the classiﬁer C ( · ) .
In each step or iteration, the lines between 5 to 10 (not including 10) are for training D ( · ) and the lines after 10 are for training C ( · ) .
The details of the algorithm are self-explanatory.
E XPERIMENTS
We now evaluate the proposed technique PAN and compare it with state-of-the-art baselines.
Three text and two image classiﬁcation datasets are used in our experiments.
YELP : a collection of online reviews from Yelp.
Each review is labeled with a star rating ranging from 1 to 5.
The dataset is extracted from the Yelp Dataset Challenge 2015.
RT : a collection of online reviews from rotten tomatoes with sentiment labels good and bad .
IMDB : another collection of online review for binary sentiment classiﬁcation.
20News : a collection of about 20,000 newsgroup documents, partitioned (nearly) evenly across 20 different news topics.
MNIST : a collection of 70,000 images of handwritten digits from 0 to 9.
CIFAR10 : a collection of 60000 32x32 colour images of 10 classes, with 6000 images per class.
See 4 for all datasets.
5. E XPERIMENT S ETTINGS
Data Preparation: Since the ﬁve datasets are for traditional supervised learning with class labels, we need to prepare positive P and unlabeled U data for PU learning.
We use two steps, after which we obtain the training and testing data for each dataset (on the left of the dataset name in Table 1).
Step 1 - Constructing positive and negative data .
As not all datasets have 2 classes, we need to make each of them a two-class (positive and negative) dataset.
RT and IMDB are already two-class datasets.
For YELP, which has 5 classes, we remove the reviews with the class label of 3-stars, and split the remaining classes into two: one as the positive data (4 or 5 stars) and the other as the negative data ( or 2 stars) (this is commonly done for sentiment classiﬁcation (Pang & Lee, 2008)).
Following the baseline (Kiryo et al., 2017), for 20News, topics ‘alt.’
, ‘comp.’
, ‘misc.’, and ‘rec.’ form the positive data, and topics ‘sci.’, ‘soc.’
and ‘talk.’
form the negative data.
For MNIST, all images labeled with even numbers form the positive data and all images labeled with odd numbers form the negative data.
For CIFAR10, classes airplane, automobile, ship and truck are used as the positive data and the rest as the negative data.
Step 2 - Creating PU learning datasets.
After step 1, we get positive and negative training data for each dataset.
We then build the PU learning training dataset, which includes positive and unlabeled data as follows.
For each dataset (except CIFAR10), we randomly select 10% (5% for CIFAR10 for diversity).
We also show more results by varying the ratio in Appendix D.1, the percent of positive examples from the positive
P for PU learning.
The unlabeled data U consists of the negative data and the remaining positive data in the dataset.
Baselines: We use our a-GAN and ﬁve state-of-the art representative approaches as the baselines.
a-GAN .
This is the direct adaptation of GAN given in Section 3.
UPU (du Plessis et al., 2015).
This method proposed a general unbiased estimator that is also convex for loss functions meeting certain linear-odd conditions.
NNPU (Kiryo et al., 2017).
This is a non-negative risk estimator for PU learning.
When minimized, it is more robust against overﬁtting, and is able to use ﬂexible models even given limited P data.
Note that NNPU has two versions, the linear and the MLP versions.
We give the results of the MLP version as it does better.
NNPUSB (Kato et al., 2019).
This is a recent algorithm that extended NNPU with an additional mechanism for handling sample selection bias.
GenPU (Hou et al., 2018):
This system uses the GAN framework and an array of generators and discriminators to generate both positive and negative data for PU learning.
PMPU (Gong et al., 2018): this is a traditional SVM based PU learning method.
It is important to note that both UPU and NNPU need the input of the class prior probability, which is often not available in practice.
In our experiments, we provide them the correct class prior probabilities.
Even with this favorable condition, they are still weaker than PAN, which does not need the class prior probability input.
For UPU and NNPU, we use the opensource code from the authors and a third party 6 , respectively.
For NNPUSB, we use the original code provided by the authors.
Note also we use the same network as these baselines, including architecture, number of parameters, and the optimization method.
We also give them exactly the same positive and unlabeled data and the test data.
For GenPU, we again use the code provided by the authors.
For PGAN’s results mentioned above, since there is no source code available, we used the best results reported in the paper.
Training Details: For fair comparison, we use the same classiﬁer C for all systems following the baselines.
For text, a 2-layer convolutional network (CNN), with 5 * 100 and 3 * 100 convolutions for layers 1 and 2 respectively, and 100 ﬁlters for each layer, is used as the classiﬁer C ( · ) and discriminator D ( · ) .
The word embeddings are also trained by the system.
For MNIST, the classiﬁer is a 3-layer MLP (with 2 hidden layers, more speciﬁcally, d-512-256-1) as it is fairly simple.
The classiﬁer for the CIFAR10 dataset was an all convolutional net: (32 × 32 × 3)-[C(3 × 3, 96)] - C( × 3, 96, 2) - [C(1 × 1, 192)] - C(1 × 1, 10) - 1000 - 1000 - 1, where each input is a 32 × 32 RGB image, C(3 × 3, 96) means 96 channels of 3 × 3 convolutions followed by ReLU, C(3 × 3, 96, 2) means a similar layer but with stride 2, etc.
We set λ in Eq.
3 and 4 to 0.0001, please see more details in Appendix D.2.
We also balance the impact of positive and unlabeled data for term I in Eq.
3 in training; otherwise the positive examples will be dominated by the unlabeled data.
We use 1:1 ratio of positive data and unlabeled data in each mini-batch in training.
The network parameters are updated using the Adam algorithm with a learning rate of 0.0001.
For baseline a-GAN, it needs pre-training of D ( · ) .
We use the original positive and unlabeled (regarded as negative) data to pre-train D ( · ) in order to enable its ability to classify positive and unlabeled data.
We pre-train D ( · ) 3 epochs for the datasets.
[Figure]
[Table]
[Figure] [Figure]
[Figure] [Figure]
5. R ESULTS AND A NALYSIS
Figures 2-7 show the test accuracy, precision and recall in each epoch of each method.
The usual ﬁrst-order exponential weighted moving average smoothing with weight 0.7 is applied to the ﬁgures.
Note that due to space limitations, we only show the curves of 100 epochs, but the ﬁnal results in
Table 1 are produced with more epochs as NNPU takes slightly longer to reach the peak (see below).
[Figure]
The ﬁnal accuracy and F-score results of A-GAN, UPU, NNPU, NNPUSB and PAN are given in
Table 1 (on the right side of dataset names) and the results for GenPU and PMPU are given Table 2.
Since different epochs give different results, for a fair comparison, we give the average of both the best F-score (F) and best accuracy (Acc) for each system on each dataset over 200 epochs (all systems converged before 200 epochs) over 5 runs.
The F-score is measured on the positive data/class, as in
PU learning the user is normally interested in identifying the positive data.
The last row in the table gives the average result for each column.
From the Figures and Table 1, we can make the following observations.
From the results in Table 1 (on the right side of dataset names in the table), we see that PAN outperforms all baselines on all datasets (the last row gives the average of each algorithm for all datasets).
Among the baselines, NNPU and NNPUSB are the strongest and their results are very similar.
PAN outperforms them markedly.
Given that PAN does not need class prior probability input, this is even more signiﬁcant.
The direct adaptation of GAN a-GAN is weaker than both NNPU and
PAN, but is better than UPU.
Although NNPUSB extends NNPU, it did not do better than NNPU.
The reason could be that our data do not have sample selection bias, which NNPUSB tries to address.
Figures 2 to 7 show that PAN and NNPUSB are robust across all datasets with both high precision and high recall.
And clearly PAN is better than NNPUSB.
NNPU is rather unbalanced for precision and recall for YELP and RT, either very high precision but very low recall, or vice versa.
a-GAN and UPU have the same problem, which is highly undesirable.
PAN also outperforms baselines consistently in accuracy for all six datasets.
From the 6 ﬁgures, we also see that a-GAN is unstable.
Precision, recall, and accuracy ﬂuctuate greatly from one epoch to another.
Stability problem of GAN is well documented (Metz et al., 2017;
Berthelot et al., 2017).
Our adaptation requiring reinforcement learning to train it is likely to have made the problem worse.
NNPU converges slowly (Figure 3).
It didn’t converge even at 100 epoch.
We report its best accuracy and F-score in Table 1 in 200 epochs (it converged earlier than 200).
[Table]
Table 2 shows the results of GenPU and PMPU.
Since
GenPU’s data generator cannot generate text data and thus no results for the text datasets.
We can see that GenPU’s results are dramatically worse.
Hou et al.
(2018) showed that GenPU does well with few classes as the positive and negative, e.g, 1 class as positive and 1 class as negative.
However, in our case, both positive and negative consists of many classes, which probably make GenPU work poorly.
PMPU is also signiﬁcantly poorer than PAN.
In summary, we can conclude that PAN markedly outperforms the baselines in accuracy, F-score, robustness, and stability.
Given that PAN does not need the class priors, this is more signiﬁcant.
Varying Positive Data and NNPU’s Sensitivity to Class Prior: We use MNIST and CIFAR10 as representatives to study these issues.
For each dataset, we randomly select 1 or 2 classes in the original data to form the positive set, and the rest to form the negative set to generate 2 PU learning datasets as discussed above.
Since both MNIST and CIFAR have 10 classes, for the 1-class positive
PU data, the class prior probability is 10% for positive and 90% for negative (or 1:9 for short).
For the 2-class positive PU data, it is 20% for positive and 80% for negative (or 2:8).
The results are given in Table 2.
We see similar improvements from PAN with the exact class prior given to NNPU
(1:9 or 2:8).
For the 1:9 (respectively, 2:8) case for both MNIST and CIFAR10, if we change the class prior from the correct 1:9 (2:8) to the wrong 2:8 (3:7), NNPU’s result drops are small (not in
Table 2).
So NNPU has some robustness.
However, if we change to the wrong 3:7 or 4:6 (for the correct 1:9), and 4:6 or 5:5 (for the correct 2:8), the drops are dramatic for MNIST.
For CIFAR10, they are smaller, even a small increase in F for the wrong 4:6 (correct 2:8), likely an anomaly as this data is hard, but still poorer than PAN.
We conclude although it is possible to estimate the class prior (see Section 2), if the estimate is off, the results can be quite poor.
[Table]
C ONCLUSIONS
This paper proposed a new GAN style PU learning method PAN based on adversarial training.
PAN is also signiﬁcantly different from GAN as PAN does not use a generator but a classiﬁer in its place.
The objective function of PAN is also entirely different as it is based on KL-divergence.
PAN represents a new way to do PU learning.
Empirical evaluation using both text and image datasets showed that
PAN outperformed the state-of-the-art baselines.
Also importantly, PAN obtained the better results without using any class prior probability information.
Appendix A A SYMMETRY OF KL ( D i || C i ) FOR P OSITIVE AND N EGATIVE D ATA
In this section, we use the gradient asymmetry for positive and negative of KL-divergence to show the need for the term marked III in Eq.
3 (also see Eq.
8 below).
The term marked II in Eq.
3 will produce asymmetric gradients for positive and negative examples for both D ( · ) and C ( · ) due to asymmetry of KL ( D i || C i ) for positive and negative data (explained below).
If we don’t have the term marked III, gradient of D ( · ) is:
[Equation]
where n p and n u are the size of positive and unlabeled set respectively.
Term marked (a) is symmetric for positive and unlabeled data as they can obtain gradients with the same scale for the corresponding position, e.g., D ( x pi )
+ D ( x uj )
= 1 .
But it is asymmetric for positive and negative data as positive data exist in the unlabeled set.
That cause the positive being over optimized toward negative.
Unfortunately, term marked (b) is also asymmetric for positive and negative data.
We can see, the zero point of gradient term marked (b) is:
[Equation]
which means that the zero point is moved according to C ( x ui ) .
In the worst case, if C overﬁtted to give small probability to instances in the unlabeled set, then D ( · ) is not easy to escape from overﬁtting.
In summary, that will cause high precision and low recall.
Asymmetric phenomenon also exist in Eq.
8 below without the term marked III as the gradient for
C ( · ) is:
[Equation]
Clearly, it is asymmetric for positive and negative data, as positives have different gradient scale compared to negatives.
And that can cause the unbalanced training problem.
In this case, we propose to use the ﬂipped distribution of C ui , denoted by (cid:101) C ui , to address the problem, and please refer to the term marked III in Eq. 8.
After adding the term marked III, the asymmetric gradient problem caused by the term marked II is eliminated.
The gradient for D ( · ) now is (cid:80) n u i =1 log (1 − C ( x ui ))
C ( x ui ) which can be regarded as a constant when optimizing D ( · ) .
And the gradient for C ( · ) now is
[Equation]
log
[Equation]
x ui ) which is symmetric between positive and negative.
B S IMPLIFICATION
Recall the loss function Eq. 3:
[Equation]
KL-divergence is deﬁned as:
[Equation]
X denotes the probability space, it is 1 or 0 ( X = { 1 , 0 } ) in our scenario.
We ﬁrst address term I in Eq.
8, if we use D to denote D pui (1) (the probability for i th instance being positive judged by discriminator) and P to denote P pui (1) , then D pui (0) =
1 − D and P pui (0) =
1 − P .
Then we have:
[Equation]
Due to the fact that P pui (0) = 0 and P pui (1) =
if the i th instance is positive and P pui (1) = 0 and
P pui (0) = 1 if the i th instance is unlabeled, we can rewrite the result as:
[Equation]
Similarly, for term 1 (cid:13) in Eq.
8, if we use D to denote D ui (1) and C to denote C ui (1) , then we get:
[Equation]
Then we have:
[Equation]
Combining Eqs.
11 and 13, we get Eq.
[Equation]
where P p denotes the distribution of the positive data.
C T HEORETICAL A NALYSIS A BOUT THE L EARNED C LASSIFIER
In this section, we analyze the properties of the learned classiﬁer and show why Eq. 3 can perform PU learning. Intuitively, from Eq.
8 (same as Eq.
3 in the paper), we can see that D ( · ) is biased if we only consider term I because the unlabeled set contains both positive and negative examples.
However, terms II and III help correct the bias.
Eq.
14 (same as Eq.
4 in the paper), which is derived from Eq.
8 for training, shows the property more clearly than Eq. 8.
Notice that the bias in term I in
Eq. 8 will result in high precision and low recall for the positive class.
Now back to Eq.
14 and let us imagine that most examples in the unlabeled set are regarded as negative by C ( · ) (meaning low recall).
From Eq.
14, we can see that the value of term V will be below zero.
But when optimizing
D ( · ) , term VI will push D ( · ) up for these data points, and thus the bias is reduced and the low recall problem is mitigated because in the next optimization iteration, C ( · ) will follow D ( · ) to go up for these data points.
We now theoretically discuss the optimal decision surface of the classiﬁer C ( · ) learned by the proposed PAN.
Proposition 1.
Let T ( x ) =
log[1 − C ( x )] − log[ C ( x )] , ε ( x ) =
f ( T ( x )) , the learned optimal decision surface of C ( · ) is:
[Equation]
[Equation] f ( · ) is a type of function that satisﬁes ε ( x ) · T ( x ) > 0 .
From Proposition 1, we can see that PAN ﬁnds the decision surface by combining P p ( x ) and P u ( x ) .
The combination is controlled by ε ( x ) .
ε ( x ) is a function of C ( x ) .
C. P ROOF OF P ROPOSITION
Proof: The training criterion for discriminator D , given any classiﬁer C , is to maximize the quantity
V ( C, D ) ,
[Equation]
Clearly, the maximum point appears at the point with derivative 0.
Then we calculate the partial derivative of V ( C, D ) to D and get:
[Equation]
Directly computing the solution is complex.
If we omit term b (cid:13) for the time being, the solution for
Eq.
17 is D ( x ) =
P
[Equation] x [Equation] P u ( x ) .
After bringing back b (cid:13) , this solution should be revised as follows.
With the deﬁnition of T ( x ) = log[1 − C ( x )] − log[ C ( x )] and ε ( x ) =
f ( T ( x )) , we can re-write the solution after revision:
[Equation]
(18) where ε ( x ) is a function of T ( x ) since P p ( x ) and P u ( x ) are decided by the dataset and ε ( x ) changes with the change of T ( x ) .
The exact expression of ε ( x ) is difﬁcult but we can show ε ( x ) ∝ T ( x ) in our case.
In Eq.
17, term a (cid:13) decreases monotonously when D ( x ) ∈ (0 , 1) , 7 and both λ and P p ( x ) are greater than 0.
In this case, if T ( x ) > 0 ( T ( x ) < 0 ), to keep Eq.
17 equal to 0, D ( x ) must move toward the positive (negative) direction, which indicates ε ( x ) > 0 ( ε ( x ) < 0 ).
Formally, we have:
[Equation]
[Equation]
Note that the training objective of D can be interpreted as using the training data ( P p ( x ) and P u ( x ) ) to ﬁnd a discrimination bound and utilizing the learned knowledge in C to adapt it.
The mini-max game in Eq.
4 can now be reformulated as:
[Equation]
Clearly, because the range of T ( x ) is ( − (cid:15), (cid:15) ) , L ( C ) achieves its minimum when T ( x ) and (2 D ∗ ( x ) − 1) have opposite signs.
Then, the optimal T ∗ ( x ) satisﬁes:
[Equation]
As we introduced in Footnote 5, C ( x ) ∈ (0 , 1) .
In this case, we use C ( x ) = 0 .
5 as the decision surface to perform classiﬁcation.
Clearly, this decision surface equals to T ∗ ( x ) =
log(1 − 0 .
5) − log(0 .
= 0 .
In summary, we get the optimal decision surface:
[Equation]
D M ORE E XPERIMENTS AND A NALYSIS
In the paper we showed that PAN gets signiﬁcant improvement comparing with state-of-the-art baselines.
Here we give more detailed analysis of PAN in terms of dealing with varied positive ratio and the selection of hyper-parameter λ in Eq.
3 (also 4).
D. V ARYING K NOWN P OSITIVE R ATIO
In this section
[Table]
we analyse the performance of PAN when dealing with varied ratios of known positive examples.
We vary the ratio of known positive examples from 5% to 30% [Table] and show the accuracy and F-score of PAN and baseline on MNIST and CIFAR10 datasets.
To verify the ability of PAN working in extreme conditions, we tested PAN on the situation with only 1% positive examples.
The results are reported in Table 4 and 5.
[Table]
[Table] [Table] [Table]
From Table 4 and
[Table]
we can see that PAN can do well with different proportions of known positive data and with extremely few known positive examples.
We note that that the margin between PAN and NNPU goes large with the decrease of the ratio of known positive examples, which indicates that
PAN is more effective than NNPU.
The margin is smaller when the known positive ratio enlarges, that is because if we have enough positive data, the limitation of getting better results is no longer the
PU learning method, but the performance of the classiﬁer.
D. H YPER - PARAMETER S ELECTION
λ is the hyper-parameter that balances the KL-divergences.
Here, we show that λ should be a small value but it is not too sensitive when it is around 0.0001.
In our case, we set it to 0.0001.
[Table]
D. A BLATION S TUDY FOR T ERM M ARKED BY III IN E Q .
[Table] [Table]
Table 7 and 8 show PAN’s ablation results in accuracy and F-score with or without term III respectively.
From the two tables, we can see that adding term III indeed improves the performance of PAN onout of 6 datasets.