Learning from Positive and Unlabeled Data  with Adversarial Training

1 INTRODUCTION . Positive-unlabeled ( PU ) learning learns a binary classifier from only Positive ( P ) and Unlabeled ( U ) examples with no labeled negative examples ( Denis , 1998 ; Liu et al. , 2002 ; Elkan & Noto , 2008 ; du Plessis et al. , 2014 ; Bekker & Davis , 2018b ) . It has many applications in text analysis , biomedicine , spam detection , recommendation , remote sensing , matrix and knowledge base completion , graph learning , etc ( Li & Liu , 2003 ; Yu et al. , 2004 ; Fusilier et al. , 2015 ; Scott & Blanchard , 2009 ; Li et al. , 2010 ; Cerulo et al. , 2010 ; Calvo et al. , 2007 ; Ren et al. , 2015 ; Hsieh et al. , 2015 ; Wu et al. , 2017 ; Zupanc & Davis , 2018 ) . This paper proposes an adversarial PU learning method inspired by GAN ( generative adversary networks ) ( Goodfellow et al. , 2014 ) , as we found that the GAN framework naturally suits PU learning . GAN aims to generate data of a particular training class , which is like the positive class P in PU learning . GAN works by generating likely positive data using a generator G ( · ) to fool a discriminator D ( · ) , which determines whether the generated data indeed belong to the training ( positive ) class . For PU learning , this is like choosing likely positive instances from the unlabeled set U ( achieved by a classifier ) also to fool a discriminator D ( · ) . Thus , we can simply replace GAN ’ s generator with a classifier C ( · ) to produce a PU learner.1 Note that the classifier is also a discriminator , but we use the term classifier here to distinguish it from the original discriminator of GAN . Note also both C ( · ) and D ( · ) are neural networks.2 This paper first proposes a direct adaptation of GAN , called a-GAN ( adapted GAN ) , for PU learning following the above idea ( see Section 3 ) . However , this simple method is unable to produce stateof-the-art results . It led us to propose a new objective function based on Kullback-Leibler ( KL ) divergence . Our final proposed method is called PAN ( Predictive Adversary Networks ) due to the use of the classifier to replace the generator in GAN along with a new adversarial training method . On the new objective function , it is clearly desirable to consider the overall performance of C ( · ) on the entire unlabeled set . We use a distance metric to measure whether C ( · ) can produce similar 1PU learning is analogous to GAN and vice versa . That is , if we could put all the data ( e.g. , images ) that can be generated by GAN ’ s generator in a set , the set should be regarded as unlabeled as it contains both good ( positive ) and bad ( negative ) images . Then what the generator does is like selecting good images to fool the discriminator , which is exactly what our classifier does with a set of given unlabeled data . 2Their exact architectures are given in the experiment section ( see Training Details in Section 5.1 ) since C ( · ) and D ( · ) have different architectures for text and image . predictions to those of D ( · ) for all the examples in U . If C ( · ) gives similar predictions , it means that the examples obtaining high probabilities from C ( · ) also get high probabilities from D ( · ) , achieving the goal of fooling D ( · ) , which will give us a good final PU classifier . We employ KL-divergence as the distance metric to measure the difference between C ( · ) and D ( · ) . One important advantage of PAN is that it does not need the input of class prior probability , which many state-of-the-art systems need ( see Section 2 ) . In practice , the class prior is unknown ( there are some existing methods to estimate it , see below ) . Hou et al . ( 2018 ) and Chiaroni et al . ( 2018 ) have employed GAN to generate positive and/or negative data and then use a separate learner ( e.g. , CNN ) to learn the final PU classifier using the generated data , they are not adaptations of GAN like PAN and their generators generate only images . PAN can be applied to any data as it has no generator . We evaluate PAN using both text and image classification datasets and show it outperforms start-of-the-art PU learning baselines even when we give them the perfect class prior probabilities . We also show that when the class prior probability estimation is off , the results can be quite poor . 2 RELATED WORK . Research in PU learning started in early 2000s ( Denis , 1998 ; Liu et al. , 2002 ; Lee & Liu , 2003 ; Yu et al. , 2004 ; Liu et al. , 2003 ; Elkan & Noto , 2008 ) . Denis ( 1998 ) proposed it under statistical query framework . Liu et al . ( 2002 ) studied sample complexity . Elkan & Noto ( 2008 ) showed that if ranking is the goal of PU learning rather than classification , then PU learning is equivalent to the conventional binary learning . Due to numerous applications , there has been a recent surge of interest in PU learning ( du Plessis et al. , 2014 ; Mordelet & Vert , 2014 ; du Plessis et al. , 2015 ; Sechidis & Brown , 2015 ; Claesena et al. , 2015 ; Chang et al. , 2016 ; Niu et al. , 2016a ; Yi et al. , 2017 ; Liu et al. , 2017 ; Jian et al. , 2017 ; Northcutt et al. , 2017 ; Kiryo et al. , 2017 ; Sakai et al. , 2017 ; Hou et al. , 2018 ; Xu et al. , 2017 ; Gong et al. , 2018 ; Bekker & Davis , 2018a ; Chiaroni et al. , 2018 ; Hsieh et al. , 2018 ; Bao et al. , 2018 ; Shi et al. , 2018 ; Sansone et al. , 2018 ; Kato et al. , 2019 ) . Early approaches for PU learning use 2-steps . Step 1 finds some highly probable negative examples from the unlabeled set . Step 2 uses the positive set , highly probable negative set , and the remaining unlabeled set to build a classifier ( Liu et al. , 2002 ; Yu et al. , 2004 ; Li & Liu , 2003 ) . Liu et al . ( 2003 ) ; Shi et al . ( 2018 ) also regarded the unlabeled data to have noisy labels . Several techniques applied training data re-weighting via regularization as well ( Lee & Liu , 2003 ; Elkan & Noto , 2008 ) . Work in ( du Plessis et al. , 2014 ; 2015 ; Kiryo et al. , 2017 ) avoided parameter tuning by using unbiased risk estimators . Kato et al . ( 2019 ) deals with sample selection bias . None of these works explores the idea of adversarial learning as we do in PAN . Hou et al . ( 2018 ) used GAN to generate positive and negative examples and then used them to build the final classifier employing a separate classification model . Chiaroni et al . ( 2018 ) proposed a method to use GAN to generate negative training examples . Both papers are for image classification . Generating text and other forms of data using GAN is relatively more challenging . PAN does not generate data . It replaces the generator with a classifier to directly train the required PU classifier in the adversarial learning framework using positive and unlabeled data . Our formulation and objective function are also quite different from those of GAN . Moreover , Xu et al . ( 2017 ) and Gong et al . ( 2018 ) used traditional margin-based methods . Our a-GAN method is also related to the studies of weighted adversarial nets ( WAN ) ( Chen et al. , 2018 ; Zhang et al. , 2018 ) as a-GAN also weights the discriminator , but the proposed PAN differs significantly from WAN . That is because although WAN weights the discriminator but the adversarial training procedure is the same as the original GAN ( similar to our a-GAN ) . Our a-GAN is also similar to the method DAN recently posted on arXiv ( Liu et al. , 2019 ) as we can see from their Eq . 5 and our Eq . 2 . The training methods are slightly different . PAN takes a very different approach as we will see in the next two sections . Other related works include leveraging biased negative examples ( Hsieh et al. , 2018 ; Sakai et al. , 2017 ) , studying the random assumption of PU learning ( Bekker & Davis , 2018a ) , multi-instance PU learning ( Bao et al. , 2018 ) , scalable PU learning ( Sansone et al. , 2018 ) , etc . More details can be found in the survey ( Bekker & Davis , 2018b ) . A weakness with these recent state-of-the-art systems is that they need the class prior probability ( du Plessis et al. , 2015 ; Kiryo et al. , 2017 ; Hou et al. , 2018 ; Xu et al. , 2017 ; Chiaroni et al. , 2018 ; Kato et al. , 2019 ) , which is hard for the user to provide . Although there are methods that try to estimate the class prior ( Menon et al. , 2015 ; Ramaswamy et al. , 2016 ; Jain et al. , 2016 ; du Plessis et al. , 2017 ; Gong et al. , 2019 ) , we will show that if the estimate is off , the results can be quite poor . 3 BACKGROUND . GAN is an adversarial learner that learns a generator for generating data such as images . It composes of two networks , a generator and a discriminator . The generator generates new data instances , and the discriminator evaluates them for authenticity , i.e . deciding whether each generated data instance it reviews belongs to the actual training dataset or not . Through an iterative and adversarial process , the generator can generate new data instances that the discriminator has hard time to distinguish from the real training data . GAN is formulated as a minmax game as follows : min G max D V ( D , G ) = Ex∼Pdata ( x ) [ logD ( x ) ] + Ez∼Pz ( z ) [ log ( 1−D ( G ( z ) ) ) ] ( 1 ) where G ( · ) is the generator that aims to generate data that can approximate real data to fool the discriminator D ( · ) , while D ( · ) tries its best to discriminate the generated data from real/training data . Pdata is the data generating distribution of the real data , and Pz is the data generating distribution of the generator . Through an adversarial training , we expect to learn a good generator . 3.1 DIRECT ADAPTATION OF GAN FOR PU LEARNING . We now present the direct adaptation of GAN ( a-GAN ) for PU learning . The real training data in GAN is our labeled positive data P . As illustrated in Figure 1 , the discriminator D ( · ) in a-GAN still plays the same role as that in GAN , but the generator in a-GAN is replaced with another discriminator , which we call the classifier ( or predictor ) C ( · ) . The goal of C ( · ) is to identify likely positives from the unlabeled set U to give to the discriminator for it to decide whether these are real positive data . The following equation shows this approach : min C max D V ( D , C ) = Exp∼Pp ( xp ) [ logD ( xp ) ] + Exs=argxu∼Pu ( xu ) C ( xu ) =1 [ log ( 1−D ( x s ) ) ] ( 2 ) where xs denotes the example judged as a likely positive example from U by C ( · ) . P p and Pu are the data generating distributions of the known positive data and the unlabeled data in PU learning , respectively . The known positive examples are randomly selected from the positive population . The hidden positives in the unlabeled set is also a random sample of the same positive population . Due to the discreteness of the last term in the equation , we use the Policy Gradient method ( Sutton et al. , 2000 ) from reinforcement learning to train it , where the last term is regarded as the reward for optimizing C ( · ) . We call this adapted version of the system a-GAN ( adapted GAN ) . a-GAN does reasonably well ( see Section 5.2 ) . However , since it focuses on the positive data only , its learning is not balanced , which can cause some confusion with the separation of positive and negative data . Next , we present the final proposed method PAN , which emphasizes both positive and negative and is able to produce a better separation for them .

Summary: The paper proposed an interesting idea of using two adversarial classifiers for PU learning. The first classifier tries to introduce samples from unlabeled data that are similar to the existing labeled positive data, and the second one tries to detect if a sample has a ground truth label or drawn from unlabeled data by the first classifier (fake). The idea of the paper is interesting; it is well-motivated and well supported with a range of experiments from NLP and computer vision tasks. The paper is a good read but required a pass of proofreading; some rearrangement of the concepts (for example, in second paragraph C(.) and D(.) is used, but they are introduced properly in section 4. Also, the paper could use some clarifications.