Year: 2021

Abstract:
Positive and Unlabeled (PU) learning targets inducing a binary classifier from weak training datasets of positive and unlabeled instances, which arise in many real-world applications. In this paper, we propose a novel PU learning method, namely Positive and unlabeled learning with Partially Positive Mixup (PMix), which simultaneously benefits from data augmentation and supervision correction with a heuristic mixup technique. To be specific, we take inspiration from the decision boundary deviation phenomenon observed in our preliminary experiments, where the learned PU boundary tends to deviate from the fully supervised boundary towards the positive side. For the unlabeled instances with ambiguous predictive results, we select their mixup partners from the positive instances around the learned PU boundary, so as to transform them into augmented instances near to the boundary yet with more precise supervision. Accordingly, those augmented instances may push the learned PU boundary towards the fully supervised boundary, thereby improving the classification performance. Comprehensive experimental results demonstrate the effectiveness of the heuristic mixup technique in PU learning and show that PMix can consistently outperform the state-of-theart PU learning methods.

heading: 1 INTRODUCTION
text: Positive and Unlabeled (PU) learning refers to a specific binary classification problem, where only a small number of positive training instances are manually annotated but all other instances are unlabeled (Liu et al., 2002). Such kind of datasets naturally arise in many significant real-world scenarios such as product recommendation (Hsieh et al., 2015), deceptive reviews detection (Ren et al., 2014), and medical diagnosis (Yang et al., 2012). For specific example, many diseases, e.g., Alzheimer’s disease, Amyotrophic Lateral Sclerosis, and Parkinson’s disease, are very infrequent and with long latency, hence only few diagnosed patients are known but a much larger population of undiagnosed individuals may be either diseased or healthy. Treating the diagnosed ones as positive instances and the undiagnosed ones as unlabeled instances results in such PU datasets of medical diagnosis. To meet those practical demands, PU learning has drawn increasing interest from the machine learning community (Bekker & Davis, 2020).
Formally, let x ∈ Rd and y ∈ {0, 1} be the feature representation and category label, respectively, where the positive instance is indicated by y = 1 and the negative one by y = 0. In the context of PU learning, the training dataset is composed of the sets of positive instances P = {(xi, yi = 1)} np i=1 and unlabeled instances U = {xi} np+nu i=np+1
, where U contains both positive and negative instances. The target is to learn a binary classifier based on such weak training dataset P ∪ U . During the past decades, many PU learning methods have been proposed, where, naturally, the essential idea is to estimate the negative instances from the set of unlabeled instances U . Generally, most of existing PU learning methods can be divided into two categories, termed as sample-selection methods and cost-sensitive methods. The sample-selection methods, as the name suggests, mainly select reliable negative instances from U using various heuristic strategies, e.g., Naı̈ve Bayes (Liu et al., 2002), kNN (Zhang & Zuo, 2009), k-means (Chaudhari & Shevade, 2012), and reinforcement learning (Luo et al., 2021); and then apply supervised methods over positive and those reliable negative instances. In contrast, the cost-sensitive methods treat all unlabeled instances as corrupted negative ones, and correct the estimation bias of the objective by employing well-designed misclas-
sification risks such as unbiased risk estimator (du Plessis et al., 2014; 2015; Kiryo et al., 2017) and maximum margin loss (Shi et al., 2018; Gong et al., 2019b; Zhang et al., 2019; Gong et al., 2019a).
Orthogonal to the aforementioned techniques, we note that some PU learning methods such as (Chen et al., 2020a; Wei et al., 2020) have made preliminary attempts to integrate with the art of mixup, i.e., an economic-yet-effective data augmentation method (Zhang et al., 2018). Formally, mixup generates an augmented instance (x̂, ŷ) with the convex combination of any pair of instances {(xi, yi), (xj , yj)} drawn from the training dataset:
x̂ = λxi + (1− λ)xj , ŷ = λyi + (1− λ)yj , λ ∼ Beta(α, α), α ∈ (0,∞).
Previous studies have indicated that mixup is approximately equivalent to applying adversarial training (Zhang et al., 2021), enabling to improve robustness with even scarce and noisy supervision (Thulasidasan et al., 2019; Carratino et al., 2020; Zhang et al., 2021). Accordingly, it has been successfully used to solve various learning problems with weak supervision, e.g., semi-supervised learning (Berthelot et al., 2019), noisy label learning (Li et al., 2020), and partial label learning (Yan & Guo, 2020).
Our story and contribution. Inspired by the recent success of mixup in learning with weak supervision, our original goal is to thoroughly investigate the impact of mixup in PU learning. To this end, we begin with a naive disambiguation-free objective of PU learning, where all unlabeled instances are treated as pseudo-negative instances, denoted by Ũ = {(xi, yi = 0)} np+nu i=np+1 , and the binary classifier is trained based on P ∪ Ũ . In preliminary experiments, we found an interesting phenomenon, where the number of training instances predicted as positive by the disambiguationfree classifier tends to be smaller than usual as illustrated in Fig.1. This phenomenon implies that the disambiguation-free boundary tends to deviate from the fully supervised boundary towards the positive side, expressed by a toy example shown in Fig.2(a). We consider that the decision boundary deviation is mainly caused by the marginal pseudo-negative instances, which lie between the two boundaries. Such instances are more likely to be positive but actually annotated by negative.
1Specially, we apply the early learning regularization (Liu et al., 2020) into the objective to keep it stable.
Motivated by this observation, we extend mixup to a specific heuristic version for PU learning, enabling to achieve data augmentation and supervision correction simultaneously. Its basic idea is to transform the marginal pseudo-negative instances into augmented instances which are partially positive and yet also lie between the two boundaries, so as to push the learned boundary towards the fully supervised one. This can be achieved by selecting the mixup partners for marginal pseudonegative instances from the positive instances that are around the learned boundary, as expressed in Fig.2(b). With this insight, we propose a novel PU method, namely Positive and unlabeled learning with Partially Positive Mixup (P3Mix). Generally, P3Mix is easy-to-implement, where, specifically, we can define the marginal pseudo-negative instances using the predictive results and the positive instances around the boundary using the entropy values of predictive results. To evaluate the effectiveness of P3Mix, we conduct a number of experiments on benchmark datasets. Experimental results demonstrate that P3Mix can consistently outperform the state-of-the-art PU methods.
2 THE PROPOSED P3MIX METHOD
In this section, we introduce the proposed P3Mix method for PU learning. We first revisit and clarify some important notations: the set of positive instances P = {(xi, yi = 1)} np i=1 and the set of unlabeled instances U = {xi} np+nu i=np+1 . By treating all unlabeled instances as negative, we translate U into the set of pseudo-negative instances Ũ = {(xi, yi = 0)} np+nu i=np+1
. Given batches Xp ⊂ P and Xu ⊂ Ũ , the disambiguation-free objective of PU learning can be formulated as follows:
L(Xp,Xu;Θ) = 1 |Xp| ∑
(x,y)∈Xp
` ( f(x;Θ), y ) + β |Xu| ∑
(x,y)∈Xu
` ( f(x;Θ), y ) , (1)
where f(·;Θ) is a trainable neural network, i.e., the binary classifier, parameterized by Θ; `(·, ·) is the loss function; and β is the coefficient parameter.
To achieve data augmentation and supervision correction simultaneously, P3Mix transforms Xp and Xu into the batches of augmented instances X̂p and X̂u using the proposed heuristic mixup technique. Accordingly, the objective of P3Mix is then expressed as follows:
L(X̂p, X̂u;Θ) = 1
|X̂p| ∑ (x̂,ŷ)∈X̂p ` ( f(x̂;Θ), ŷ ) + β |X̂u| ∑ (x̂,ŷ)∈X̂u ` ( f(x̂;Θ), ŷ ) , (2)
X̂p, X̂u = HeuristicMixup(Xp,Xu, α), (3)
where α ∈ (0,∞) is a hyperparameter of mixup. Next, we describe the details of heuristic mixup.
Algorithm 1 Training procedure of P3Mix, P3Mix-E and P3Mix-C Input: P ∪ U : training instances; β: coefficient parameter; γ: thresholding parameter; k: size of the candidate mixup pool; α: hyperparameter of mixup; η: coefficient parameter of early-learning regularization . η = 0 for P3Mix and P3Mix-C
Output: Θ: binary classifier parameters
1: Initialize Θ, the mean-teacher parameters Θ̃ and the candidate mixup pool Xcnd randomly, translate U into Ũ ; 2: for t = 1, 2, · · · ,MaxEpoch do 3: Shuffle P ∪ Ũ into I mini-batches and denote the i-th mini-batch by (X ip,X iu); 4: for i = 1, 2, · · · , I do 5: Estimate marginal pseudo-negative instances Xmpn using Eq.(6); 6: Select the mixup partners for each instance within X ip ∪ X iu using Eq.(5); 7: Set labels of { (x, y = 0)|(x, y = 0) ∈ X iu, f(x;Θ) > γ } to 1; . Optional for P3Mix-C 8: Construct X̂ ip ∪ X̂ iu by applying Eq.(4) to X ip ∪ X iu and their mixup partners; 9: Estimate {ỹj} |X̂ ip|+|X̂ i u| j=1 for instances in X̂ ip ∪ X̂ iu by f(·; Θ̃); . Optional for P3Mix-E
10: Update Θ by ∇Θ ( L(X̂ ip, X̂ iu;Θ) + ηRelr({(x̂j , ỹj)} |X̂ ip|+|X̂ i u| j=1 ;Θ) )
with Adam; 11: end for 12: Update Xcnd using Eq.(7); 13: Update Θ̃ by Θ with the move-average; . Optional for P3Mix-E 14: end for

heading: 2.1 TRAINING WITH HEURISTIC MIXUP
text: Basically, for each instance (xi, yi) ∈ Xp ∪ Xu we select a mixup partner (xj , yj) to generate an augmented instance (x̂i, ŷi) using the modified mixup operator2(Berthelot et al., 2019):
x̂i = λ ′xi + (1− λ′)xj , ŷi = λ′yi + (1− λ′)yj , λ′ = max(λ, 1− λ),
λ ∼ Beta(α, α), α ∈ (0,∞), (4)
accordingly forming the augmented instance sets X̂p and X̂u. Our heuristic mixup refers to a guidance of mixup partner selection to refine the imprecise supervision within Xu. We take inspiration from the phenomenon, where the boundary learned by Eq.(1) tends to deviate from the fully supervised boundary towards the positive side as illustrated in Fig.2(a). The marginal pseudo-negative instances Xmpn ⊂ Xu lie between the two boundaries, and they are more likely to be positive but actually annotated by negative. To resolve this problem, for each of them we uniformly select a mixup partner from the candidate mixup pool Xcnd ⊂ P of positive instances that are around the current learned boundary, so as to generate an augmented instance which is partially positive and yet also lies between the two boundaries as expressed in Fig.2(b). Besides, for positive instances Xp and other pseudo-negative instances Xu \ Xmpn, we uniformly choose their mixup partners from Xp ∪ Xu The overall mixup partner selection is formulated as follows:
(xj , yj) ∼  Uniform(Xcnd) if (xi, yi) ∈ Xmpn,
Uniform(Xp ∪ Xu) if (xi, yi) ∈ Xp ∪ Xu \ Xmpn. (5)
In what follows, we introduce how to estimate the marginal pseudo-negative instances Xmpn and construct the candidate mixup pool Xcnd.
2Because we compute individual loss terms for positive instances and pseudo-negative ones in Eq.(2) appropriately, we define λ′ = max(λ, 1−λ) to guarantee that the feature of each augmented instance x̂i is closer to xi than the mixup partner xj . Consequently, (x̂i, ŷi) is assigned into X̂p if (xi, yi) ∈ Xp, or X̂u otherwise.
Marginal pseudo-negative instance estimation. Because the fully supervised boundary is exactly unknown, we have to estimate the set of marginal pseudo-negative instances Xmpn from Xu. In this work, we define them as the “unreliable” pseudo-negative instances measured by the predictive scores with thresholding parameter γ ∈ [0.5, 1]:
Xmpn = { (x, y = 0)|(x, y = 0) ∈ Xu, 1− γ ≤ f(x;Θ) ≤ γ } , (6)
where γ = 0.5 implies Xmpn = ∅, and γ = 1 means Xmpn = Xu.
Candidate mixup pool. We maintain a candidate mixup pool Xcnd containing the positive instances around the current learned boundary from P . To be specific, for each positive instance we compute its entropy value of the predictive score, and update the candidate mixup pool with the top-k positive instances as follows:
Xcnd = { (x, y = 1)|(x, y = 1) ∈ P,H(f(x;Θ)) ∈ Rank({H(f(xi;Θ))} np i=1) } , (7)
where H(·) is the entropy, and Rank(·) outputs a set of positive instances with the top-k maximum entropy values. For efficiency, we update Xcnd in every epoch. The full training procedure is shown in Algorithm 1.

heading: 2.2 ROBUSTNESS
text: The augmented instances within X̂u also suffer from imprecise supervision even using the heuristic mixup. To make P3Mix more robust, we employ two tricks, i.e., early-learning regularization (Liu et al., 2020) and pseudo-negative instance correction. We call the versions with early-learning regularization and pseudo-negative instance correction as P3Mix-E and P3Mix-C, respectively.
Early-learning regularization. We employ the early learning regularization to prevent the memorization of imprecise supervision (Liu et al., 2020). For each mixup instance within X̂p ∪ X̂u, we estimate an auxiliary target vector ỹ, and formulate the early-learning regularization below:
Relr({(x̂i, ỹi)} |X̂p|+|X̂u| i=1 ;Θ) =
1
|X̂p|+|X̂u|
∑|X̂p|+|X̂u| i=1 log ( 1− 〈f(x̂i;Θ), ỹi〉 ) (8)
Here, we estimate the target vector ỹ of each instance by using the mean teacher technique (Tarvainen & Valpola, 2017), and incorporate Eq.(8) to Eq.(2).
Pseudo-negative instance correction. We concentrate on the pseudo-negative instances with high confidence to be positive { (x, y = 0)|(x, y = 0) ∈ Xu, f(x;Θ) > γ } . We directly revise their labels to positive before their corresponding mixup operators.

heading: 3 EXPERIMENT

heading: 3.1 EXPERIMENTAL SETTINGS
text: Datasets. In the experiments, we employ three prevalent benchmark datasets, including FashionMNIST (F-MNIST) (Xiao et al., 2017),3 CIFAR-10 (Krizhevsky, 2016),4 and STL-10 (Coates et al., 2011).5 The dataset statistics are described in Table 1. Note that all benchmark datasets have 10 category labels, and we denote them with integers ranging from 0 to 9 following the default settings in torchvision 0.10.0. For each dataset, we group those category labels into two disjoint sets as positive or negative, and generate two synthetic PU datasets by reversing the definitions of positive and negative labels. Following the protocol of (Chen et al., 2020a), the specific definitions of labels (“positive” vs “negative”) are as follows: F-MNIST-1: “1,4,7” vs “0,2,3,4,5,6,8,9”, F-MNIST-2: “0,2,3,5,6,8,9” vs “1,4,7”; CIFAR-10-1: “0,1,8,9” vs “2,3,4,5,6,7”, CIFAR-10-2: “2,3,4,5,6,7” vs “0,1,8,9”; STL-10-1: “0,2,3,8,9” vs “1,4,5,6,7”, STL-10-2: “1,4,5,6,7” vs “0,2,3,8,9”. For each dataset, we randomly select 1,000 positive instances from the training set, and 500 instances as the validation set.
3https://github.com/zalandoresearch/fashion-mnist 4http://www.cs.toronto.edu/˜kriz/cifar.html 5https://cs.stanford.edu/˜acoates/stl10
Baseline methods. To verify the effectiveness of P3Mix, we utilize eight PU learning baselines and also the supervised method for comparison. The details of baseline methods are presented below.
• unbiased PU learning (uPU) (du Plessis et al., 2014): A cost-sensitive method based on unbiased risk estimation. We use the public code from the net.6
• non-negative PU learning (nnPU) (Kiryo et al., 2017): A cost-sensitive method based on non-negative risk estimation. We use the public code from the net.6 [suggested settings: β = 0 and γ = 1.0]
• nnPU+mixup: A cost-sensitive method incorporating the typical mixup technique into the nnPU method by mixing positive instances and unlabeled ones separately.
• Self-PU (Chen et al., 2020b): A cost-sensitive method with self-supervision scheme. We use the public code from the net.7 [suggest settings: α = 10.0, β = 0.3, γ = 1/16, Pace1 = 0.2 and Pace2 = 0.3]
• Predictive Adversarial Networks (PAN) (Hu et al., 2021): A GAN-based PU learning method with a discriminator and a classifier. We use the public code from the net.8 [suggested settings: λ = 1e− 4]
• Variational PU learning (VPU) (Chen et al., 2020a): A PU learning method based on the variational principle. We use the public code from the net.9 [suggested settings: α = 0.3, β ∈ {1e− 4, 3e− 4, 1e− 3, · · · , 1, 3}]
• MIXPUL Wei et al. (2020): A PU learning method based on the consistency regularization with the mixup technique. We use the public code from the net.10 [suggested settings: alpha = 1.0, β = 1.0, η = 1.0]
• Positive-Unlabeled Learning with effective Negative sample Selector (PULNS) (Luo et al., 2021): A sample-selection method with reinforcement learning. We implement an in-house python code with a 3-layer MLP selector suggested by the paper. [suggested settings: α = 1.0 and β ∈ {0.4, 0.6, 0.8, 1.0}]
• Supervised: The classifiers trained on the fully supervised datasets.
For all comparing methods, we adopt the classifiers (including the discriminator of PAN) by LeNet-5 for F-MNIST, and 7-layer CNN for CIFAR-10 and STL-10. Specially, the baseline methods of uPU, nnPU and Self-PU require the prior knowledge of class proportion, however, the prior is actually unknown for STL-10 since it contains many “real” unlabeled instances. Accordingly, we estimate the class proportion of STL-10 by using the SOTA KM2 method (Ramaswamy et al., 2016) before evaluating uPU, nnPU, and Self-PU.
Implementation details of P3Mix. We implement P3Mix, P3Mix-E and P3Mix-C by using Pytorch (Paszke et al., 2019) with the Adam algorithm (Kingma & Ba, 2014). We employ the cross entropy function as the loss function ` of Eq.(2), fix the mixup hyperparameter α to 1 and the size k of the candidate mixup pool Xcnd to 100, and choose the coefficient parameter β from {0.8, 0.9, 1.0}, the thresholding parameter γ from {0.8, 0.85, 0.9}. We will make the sensitivity analysis on {β, γ} later. Specially, the early-learning regularization parameter of P3Mix-E is chosen from {1.0, 2.0, 3.0, 4.0, 5.0}.
6https://github.com/kiryor/nnPUlearning 7https://github.com/TAMU-VITA/Self-PU 8https://github.com/morning-dews/PAN 9https://github.com/HC-Feynman/vpu
10https://github.com/Stomach-ache/MixPUL

heading: 3.2 CLASSIFICATION PERFORMANCE
text: For each dataset, we independently run each comparing method 5 times and report the average classification accuracy in Table 2. Generally, our P3Mix-E and P3Mix-C consistently outperform all PU learning baselines on all benchmark datasets, indicating their superior performance. Compared with nnPU+mixup, VPU and MIXPUL, which utilize the typical mixup technique, both P3Mix-E and P3Mix-C achieve significant performance gain in most cases, i.e., about 1% ∼ 5% improvements. These results imply that our proposed heuristic mixup benefits to the supervision correction within marginal pseudo-negative instances. Compared with the three cost-sensitive PU learning methods uPU, nnPU and Self-PU, P3Mix-E and P3Mix-C also outperform them by about 1% ∼ 4% in all cases. Besides, we observe that all discriminative PU learning methods except uPU perform better than the GAN-based PU learning method PAN in most cases. The possible reason is that the imprecise supervision within unlabeled instances makes the identification of fake instances for the discriminator more difficult, resulting in a worse classifier.

heading: 3.3 ABLATION STUDY
text: To evaluate the effectiveness of our proposed heuristic mixup, we conduct the ablation experiments on all benchmark datasets. Specifically, we compare P3Mix, P3Mix-E and P3Mix-C with the Disambiguation-Free (DF) objective of Eq.(1), DF+mixup, augmented by the typical mixup technique, and their versions with early-learning regularization (“-E”) and pseudo-negative instance correction (“-C”). The experimental results are reported in Table 3. It clearly demonstrates that the proposed heuristic mixup can significantly improve the classification performance. This result is expected because the heuristic mixup can simultaneously achieve data augmentation and supervision correction by refining the imprecise supervision within marginal pseudo-negative instances. Besides, we can also observe that both early-learning regularization and pseudo-negative instance correction contribute to the improvement of the classification performance in all cases, proving the effectiveness of those two tricks in improving the robustness of models.

heading: 3.4 SENSITIVITY ANALYSIS
text: In this section, we examine the sensitivities of the coefficient parameter β and the thresholding parameter γ on F-MNIST-1, CIFAR-10-1 and STL-10-1.
Sensitivity of β. We examine the impact of different β values over the set {0.1, 0.2, · · · , 1.0} by P3Mix-C and plot the experimental results in Fig.3. We omit the results of P3Mix and P3Mix-E due to their similar performance curves and also page-limitation. Obviously, the performance achieves the highest and is relatively stable when β ≥ 0.8, and it sharply drops as the values become smaller especially on CIFAR-10-1 and STL-10-1. Notice that the coefficient parameter β is used to balance the importance of the positive and pseudo-negative parts in Eq.(2). The larger or smaller value of
β will result in the indecent importance of the pseudo-negative part, leading to a unstable classifier. Therefore, we suggest tuning β over the set {0.8, 0.9, 1.0} in practice.
Sensitivity of γ. Generally, the thresholding parameter γ is utilized to estimate the set of marginal pseudo-negative instances Xmpn from pseudo-negative instances. However, both the early-learning regularization and pseudo-negative instance correction may be affected to γ since they are mainly used to control the corrupted negative instances withinXu\Xmpn. Accordingly, we perform the sensitivity analysis of γ by P3Mix. Specifically, we vary the value of γ over the set {0.5, 0.55, · · · , 1.0}. As shown in Fig.4, P3Mix performs relatively worse when γ is too small or too large especially on F-MNIST-1 and CIFAR-10-1. This result is expected because the smaller value of γ implies that Xmpn contains fewer marginal pseudo-negative instances, e.g., Xmpn = ∅ when γ = 0.5, and it will hurt the supervision correction of heuristic mixup. Besides, when γ becomes too large, too many pseudo-negative instances will be treated as marginal pseudo-negative ones, e.g., Xmpn = Xu when γ = 1.0, resulting in the reduction of the variety of augmented instances. In summary, its suggested setting is given by {0.8, 0.85, 0.9, 0.95}.

heading: 4 RELATED WORK
text: In this section, we review the representative studies on PU learning, especially the ones most related to P3Mix. Besides, we briefly introduce the recent studies of mixup for data augmentation.

heading: 4.1 PU LEARNING
text: As the name suggests, the basic idea of sample-selection methods is to select reliable negative instances from unlabeled instances to form pseudo-binary dataset before applying supervised methods. Existing methods have proposed various heuristic strategies for negative sample selection, e.g., 1- DNF (Yu et al., 2002; 2004; Peng et al., 2008), Naı̈ve Bayes (Liu et al., 2002), Rocchio extraction (Li & Liu, 2003), kNN (Zhang & Zuo, 2009), k-means (Chaudhari & Shevade, 2012), large margin method (Gong et al., 2018) and reinforcement learning (Luo et al., 2021). Early works based on sample-selection spirit mainly focus on exploiting various traditional classification and clustering
approaches to construct the heuristic strategy. Recently, PULNS (Luo et al., 2021) employs a negative selector trained by a reinforcement learning framework, where the selector, the selection of negative instances, and the performance of the classifier are treated as the agent, action and reward, respectively. Then the classifier is induced from the mixture of positive instances and negative ones selected by the selector.
In contrast, the community of PU learning has recently paid more attention to the cost-sensitive methods, which directly treat all unlabeled instances as corrupted negative instances and correct the estimation bias of the objective by employing well-designed misclassificaation risks. The uPU (du Plessis et al., 2014; 2015) made an early attempt of unbiased risk estimation, which reformulates the misclassification risk as an equivalent and ubiased form depending only on PU datasets. However, as reported in (Kiryo et al., 2017), the risk of uPU would become negative due to overfitting when using the flexible and complex models such as deep networks. To remedy this issue, the authors of (Kiryo et al., 2017) suggest the nnPU method, i.e., the non-negative version of uPU. Besides, the Self-PU (Chen et al., 2020b) further considers the learning capability of the model itself, and jointly employs three self-supervision techniques, i.e., a self-paced strategy to discover confident positive and negative instances, a self-calibrated instance-aware loss to explore meaningful supervision over unconfident instances, and a self-supervision consistency by teacher-students learning. Other cost-sensitive methods are based on the maximum margin objective, and refine the bias of corrupted negative instances by various tricks, such as unbiased centroid estimation of unlabeled instances (Shi et al., 2018; Gong et al., 2019b), label calibration with a hat loss (Zhang et al., 2019), and margin-based label disambiguation (Gong et al., 2019a).
In parallel with the aforementioned methods, several GAN-based PU learning methods (Hou et al., 2018; Chiaroni et al., 2018; Guo et al., 2020; Na et al., 2020; Hu et al., 2021) have been proposed. The GenPU (Hou et al., 2018) generates positive and negative instances, and induces a classifier from those generated instances. The PAN (Hu et al., 2021) is based on adversarial learning on the probability distributions of a discriminator and a classifier.
Orthogonal to those PU learning methods, our P3Mix concentrates on data augmentation and extends the well-established mixup technique (Zhang et al., 2018) to a specific heuristic version for PU learning, enabling to achieve data augmentation and supervision correction simultaneously. The recent PU learning works most related to P3Mix are VPU (Chen et al., 2020a) and MixPUL (Wei et al., 2020), in which the mixup is used as a regularization to improve the robustness of classifiers and performed among labeled and unlabeled instances randomly. In contrast, our P3Mix constructs a heuristic mixup partner selection to refine the imprecise supervision within unlabeled instances.

heading: 4.2 MIXUP AUGMENTATION
text: The mixup technique (Zhang et al., 2018) generates augmented instances with convex combinations of training instances. Despite its simplicity, it can effectively improve the robustness with even scarce and noisy supervision (Thulasidasan et al., 2019; Carratino et al., 2020; Zhang et al., 2021). Further, a number of modified mixup versions (Verma et al., 2019; Yun et al., 2019; Guo et al., 2019; Kim et al., 2020; Hendrycks et al., 2020) have been proposed, e.g., manifold mixup that generates convex combinations on the latent feature space (Verma et al., 2019) and puzzle mixup that further
explores the saliency information and underlying statistics of instances (Kim et al., 2020). In this work, we extend the typical mixup to a specific heuristic version for PU learning.

heading: 5 CONCLUSION
text: In this paper, we propose a novel PU learning method named P3Mix, which extends the typical mixup technique to a heuristic version. The story begins with the observation of the decision boundary deviation phenomenon, which inspires us to propose a guidance of mixup partners selection, especially for the marginal pseudo-negative instances. Fortunately, this heuristic mixup technique can simultaneously achieve data augmentation and supervision correction for PU learning. Generally, our P3Mix is easy-to-implement, and we also employ two tricks to improve the robustness of P3Mix. We compare P3Mix against a number of existing PU learning methods on benchmark datasets. Experimental results show the superior performance of P3Mix and the effectiveness of the heuristic mixup technique.

Review summaries:
This paper proposes a variant of the mixup technique for positive-unlabeled learning. Based on the observation that the learned PU boundary tends to deviate towards the positive side, the authors suggest selecting samples between the learned PU and supervised boundaries. The proposed P3MIX method and its variant improve the classification performance of PU learning.

This paper studies an interesting weakly supervised binary classification problem called positive and unlabeled (PU) learning. The authors propose a novel PU learning method inspired by the boundary deviation phenomenon observed in experiments. Specifically, a new mixup method is proposed, which selects the mixup partners for unlabeled examples heuristically to obtain more correct supervised signals. Extensive empirical results, including ablation study and sensitiveness analysis, are provided to evaluate the proposal.

In this paper, the authors focus on the problem of positive and unlabeled learning. They show an interesting phenomenon, where the learned PU boundary tends to deviate the supervised boundary towards the positive side when treating unlabeled examples as pseudo-negative examples. The phenomenon may imply there are a number of marginal pseudo-negative examples that are more likely to be positive but labeled as negative. Based on this, the paper proposes a PU learning approach building on a novel heuristic mixup technique, which can achieve both data augmentation and supervision correction. They also present many empirical results to show the superior performance comparing with SOTA PU learning methods.