A Closer Look to Positive-Unlabeled Learning from Fine-grained Perspectives: An Empirical Study
Authors: Yuanchao Dai
Zhengzhang Hou
Changchun Li
Yuanbo Xu
En Wang
Ximing Li
Venue: NeurIPS 2025 poster ()
Keywords: positive and unlabeled learning
Primary area: general_machine_learning

Abstract: Positive-Unlabeled (PU) learning refers to a specific weakly-supervised learning paradigm that induces a binary classifier with a few positive labeled instances and massive unlabeled instances. To handle this task, the community has proposed dozens of PU learning methods with various techniques, demonstrating strong potential. In this paper, we conduct a comprehensive study to investigate the basic characteristics of current PU learning methods. We organize them into two fundamental families of PU learning, including *disambiguation-free empirical risks*, which approximate the expected risk of supervised learning, and *pseudo-labeling methods*, which estimate pseudo-labels for unlabeled instances. First, we make an empirical analysis on disambiguation-free empirical risks such as uPU, nnPU, and DistPU, and suggest a novel risk-consistent set-aware empirical risk from the perspective of aggregate supervision. Second, we make an empirical analysis of pseudo-labeling methods to evaluate the potential of pseudo-label estimation techniques and widely applied generic tricks in PU learning. Finally, based on those empirical findings, we propose a general framework of PU learning by integrating the set-aware empirical risk with pseudo-labeling. Compared with existing PU learning methods, the proposed framework can be a practical benchmark in PU learning.

A Closer Look to Positive-Unlabeled Learning from Fine-grained Perspectives: An Empirical Study
Yuanchao Dail Zhengzhang Hou! Changchun Li!e, Yuanbo Xu En Wang', Ximing Li
CCollege of Computer Science and Technology. Jilin University, China
Key Laboratory of Symbolic Computation and Knowledge Engineering, Jilin University, China RRIKEN Center for Advanced Intelligence Project fliximing86, yuanchaodai, changchunl193) @amamlmmmmll.oom
Abstract
Positive-Unlabeled (PU) learning refers to specific weakly-supervised learning paradigm that induces a binary classifier with a few positive labeled instances and massive unlabeled instances. To handle this task, the community has proposed dozens of PU learning methods with various techniques, demonstrating strong potential. In this paper, conduct comprehensive study to investigate the basic characteristics current PU learning methods. We organize them into two fundamental families of PU learning, including disambiguation which approximate the expected risk supervised learning, and pseudo- methods, which estimate pseudo-labels for unlabeled instances. First, we make an empirical analysis on disambiguation-fres empirical risks such as uPU, nnPU, and DistPU, and suggest a novel risk-consistent set-aware empirical risk from the perspective of aggregate supervision. Second, we make an empirical analysis pscudo-labeling methods to evaluate the potential of pscudo-label estimation techniques and widely applied generic tricks in PU learning. Finally, based on those empirical findings, we propose a general framework of PU learning by integrating the set-aware empirical risk with pseudo-labeling. Compared with existing PU learning methods, the proposed framework can a practical benchmark PU learning.
empirical risks,
Introduction
Positive-Unlabeled (PU) learning refers to specific weakly-supervised learning paradigm 1 2 for binary classification, which trains a binary classifier with a few positive labeled instances and massive unlabeled instances [4] It arises in various practical scenarios such automatic face tagging, spam detection, and Inlier-based outlier detection [S]. Due to its wide applicability, PU learning has increasingly attracted more attention from the machine learning community.
During the past decades, many emerging practical PU learning methods have been proposed with various advanced techniques [6 77. Because in PU learning, negative labeled instances are unavailable. how to deal with unlabeled instances becomes its key challenge; and from this taxonomic perspective, we organize the existing PU learning methods into two fundamental families, namely disambiguation
free empirical risks 5
10 and pseudo-labeling methods [T] 12
13
14
15 16).
The disambiguation-freg free empirical risks, as the name suggests, directly apply only positive labeled instances and unlabeled instances approximate the expected risk of supervised learning. Under certain data generation assumptions, previous studies suggest unbiased empirical risk uPU [17 and several practical variants such as nnPU with non-negativity constraint (8, abs-PU with absolute-
Cooresponding author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

value constraint 19, and DistPU with positive-class prior constraint [10]. In parallel, the basic idea of aseudo-labeling methods is estimating pseudo-labels for unlabeled instances, and training the binary classifier with them in self-training manner. Analogous to semi-supervised learning, these methods typically estimate pseudo-labels by iteratively updating the current predictions. For example, RP [88] iteratively identifies reliable negative examples from unlabeled data and assigns hard pseudo-labels to them for subsequent training. Another recent Self-PU [I]] utilizes a soft pseudo-labeling strategy that continuously refines label assignments by incorporating the evolving confidence scores throughout the training process, Additionally, they apply several generic tricks such as mixup augmentation, exponential moving average, and knowledge distillation to further improve the classification performance 7 19].
The current PU learning methods have demonstrated strong potential, but we find that most of them, especially pseudo-laheling ones, are commonly complicated by integrating with specific tricks. Accordingly, some of their basic characteristics are still unclear, such as what kind of pseudo-labeling techniques and generic tricks are practical. In this paper, we conduct a comprehensive study to investigate the basic characteristics from a fine-grained perspective of PU learning. First, we make an empirical risks; suggest a novel risk-consistent set-aware
empirical analysis on disambiguation-Fi empirical risk from the perspective of aggregate supervision, and empirically validate that it can be a practical candidate for disambiguation-Fre empirical risks. Second, we turn to pseudo-labeling methods, and make empirical analysis on basic techniques to estimate pseudo-label such as hard pseudo-labeling technique, soft pscudo-labeling technique, and high-confident pseudo-label selection strategies; additionally, we empirically analyze several widely applied generic tricks in PU learning. Finally, based on those empirical findings, we propose a general framework of PU learning by integrating the set-aware empirical risk with pseudo-labeling, namely GPU. We further suggest specification principles within GPU. Compared with existing PU learning methods, the proposed GPU framework can be a practical benchmark in PU learning. In summary, the contributions of this paper is outlined below:
We conduct comprehensive empirical study the current PU learning methods, and make extensive empirical observations on the effectiveness of basic techniques and tricks in PU Learning.
We propose a novel risk-consistent set-aware empirical risk from the perspective of aggregate supervision, which can be practical candidate for disambiguation-fret empirical risks, and then formulate a novel general framework of PU learning by integrating it with pseudo- labeling.
We suggest implementation principles of GPU. Compared with existing PU learning methods, the proposed GPU framework can be a practical benchmark in PU learning.
2 Preliminaries
In this section, we review the problem setting of PU learning and the two main families of PU learning methods.
Problem formulation and notations Formally, under the two-sample problem setting (20) and completely selected at random (SCAR) assumption (21], given a positive dataset D, ((x,+1) i.idd. (x(x) = p(xly = +1) with no instances drawn from the positive-class condi- tional density (x) and an unlabeled dataset D,, ((x,, } i.i.d. p(x) with n., instances drawn from the marginal density p(x), where x E Rd and y E [11,+1) denote the d-dimensional feature vector and the corresponding binary label, respectively. The objective of PU learning is to induce classifier g: d + IR over Dp U D., which can predict labels for unseen instances.
2.1 PU Learning with Disambiguation-free Empirical Risks
PU learning methods with disambiguation-fre empirical risks directly utilize labeled positive data and unlabeled data to approximate the expected risk of supervised learning. this work, we review several representative ones, including uPU (5), nnPU [8), absPU [9, and DistPU [10].

---

Table A summary of basic techniques and widely applied generic tricks for PU learning methods with pseudo-labeling
pseudo- labeling technique
generic trick
PUL. methods
pseudo-lahel
high confidence selection w/o
moving
mixup
hard
soft
with
average
RP 18
AdaSampling
22
GenPU
Self-PU 11
VPU 12
PULNS 24
P'Mix 13
RobustPU [4]
HolisticPU
15
LaGAM
25
PUL-CPBF [6
VQ-Encoder 26
Formally, let 8 X' x - R+ be any loss function, and π = ply = +1) be the positive-class
prior. Since negative samples not directly accessible in PU learning, the SCAR assumption fortu- nately provides a solution. Under this assumption, where the labeled positive examples are selected
completely at random from all positive examples, given R2 (g) Ex,(()) [[(g(x) +1)]. R2 (g)
Ep,(xx [e(g()) 1)]. R (g) = (x,(() [d()(), -1)], and R (g) = Bp(x) [g(()) -1]] we ob- tain that p(x)= = (x) + (1 x)P,. such that (1 π R, (g) = (g) π RR (g). Then,
suppose is known, can formulate (5). which is an unbiased risk of PU learning uPU for the expected risk of supervised learning, formulated as follows:
Raru()) (g) π R (0) R (0) R, (g),
and its empirical risk over D, U D, is given below:
Rapu(g) (9) +
However, uPU suffers from overfitting when using flexible models due to negative empirical risk
R (g) t R, (g). Some methods attempt to impose non-negative constraint on Rg) - π R (g)
prevent the empirical risk from becoming negative. To achieve this, nnPU [ incorporates the max function:
RmaPU (g) नスे (g) max(0, Ri (g) uπ, (g))
and absPU incorporates the absolute value function 9
RataPU (g) n (1) + R. (g) - R2 (4)
Additionally, under the case of symmetric losses where {(2) + ((-2) = 1, we have (g)
1 R5 (g) naturally holds. Leveraging this property, Dist-PU [10] reformulates uPU as follows:
RDiaPU()) P. (g) + Ř. (g) T|
(5)
where the absolute value function is introduced to impose the non-negative constraint on R. (g)
m
2.2 PU Learning with Pseudo-labeling
PU learning methods with -seudo-labeling, the name suggests, estimate srecudo-lbbele for unlabeled data, and train the classifier in a self- self-training manner [18 22 23 semi-supervised learning, we can formulate the generic objective of pseudo-labeling below:
24
13
14
[6].
Referring to
Cig; Dp, Dw) f (g) + C. 0s Dw),
(6)
where Lg 0s D.) is the self-training objective with unlabeled data, and ý denotes the pseudo-label.

---

Table 2: Positive and negative label groups of datasets and the statistics of those PU learning sets.
Dataset
Positive Class
Negative Class
Feature
Train
Backhone
F-NIIT--1
0.4
0, 2, 4,6
1,3, 5, 8,9
28 x 28
60,000
LcNet-5
F-MNIST-2
0.6
1,3,5.7,8,9
0,
28 x 28
60,000
LeNet-5
CIFAR-10-1
0.4
0,1,8,9
5,
x 32 x 32
50,000
7-Layer CNN
CIFAR-10-2
0.6
2,3,4,5,6,7
0,1.9.
x 32 32
50,000
7-Layer CNN
STI-10011
0,2.3.8,9
1,4,5,6, 6,7
x 96
96
105,000
7-Layer CNN
STL-10-2
6,4,5,6,7
0,2.3.8,9 x 96
105,000
7-Layer CNN
We review existing pseudo-labeling methods and summarize the basic techniques and widely applied generic tricks in Table [ Specifically, the basic problem of pseudo-labeling is the techniques to estimate pseudo-labels y for unlabeled data with current predictions, and they typically include hard and soft and secudo-labeling techniques. Let q = (xx and ((g) E [0, 1] denote the prediction of the classifier and the confidence belonging to the positive class, respectively, where Φ transformation function, and here we apply the sigmoid function. Then, the hard and soft pseudo-labels are estimated as U = sign (o(4) - 0.5) (-1,+1} and ý = (()), respectively. In addition, some studies suggest selecting high-confident pseudo-laiee labels, rather than applying all of them [8 22 the representatives include various thresholding strategies.
14,
where
We briefly review two widely applied generic tricks in PU learning studies 11| 12 25), such as mixup and moving average. Mixup is efficient data augmentation trick with the convex combination of instance pairs [7]. Given an instance pair (x,, yi) and (xj, Vý). it generates an augmented instance (&, 5) follows:
Generic tricks
Ax; + (1 A)x;,
ÿ = Ays + (1 A)8s,
X d~ Beta(a, a))
(7)
Here, the moving average refers to updating aseudo-label, with historical predictions during the classifier training process [27. Formally, its update equation for pseudo-labels given below:
((g) cá(g") (1 c))(g),
(8)
where g² denotes the historical prediction, and smoothing parameter.
3 Empirical Findings, Analysis, and Modifications
3.1 Settings of Empirical Study
We conduct empirical evaluations on 3 standard benchmark datasets, i.e. Fashion-MNIST (F-MNIST), CIFAR-10, and STL-10. Following [16). transform them into a set of binary classification problems by partitioning their original 10 classes into positive and negative categories by varying the class prior
{0.4, 0.6]. For all datasets, the number of positive labeled instances fixed as np = 1, 000. The details of datasets are summarized in Table 2
For each PU learning method, We employ dataset-appropriate backbones follows: LeNet-5 for F-MNIST, 7-layer CNN for CIFAR-16 10 and STL-10; the MLP layer is used as the classification layer across all datasets. The mini-batch is fixed 512 and the number of epochs is set to 100 for F-MNIST and 200 for others.
In addition, we employ the classification accuracy (ACC) as the evaluation metric. All experiments are conducted with five different random seeds on a server equipped with two Nvidia RTX4090 GPUs, and we report the mean and standard deviation of the results.
3.2 Disambiguation-free Empirical Risks
In this section, we suggest novel set-aware empirical risk of PU learning and empirically evaluate and existing disambiguation-fra free empirical risks with various surrogate loss functions.

---

Table 3: Properties of commonly used loss functions.
Loss
Formula
Convex
Differ.
Symm.
Lipse.
hinge
ma 1-
logistic
log(1 +e=*)
sigmoid
1/(1 +e²)
/
squared
(1 - =)²
ramp
min(1, max(0, (1- 22] max(0, (1 2)/2, =2
double-hinge
Set-aware empirical risk of PU learning In PU learning, we are given the positive labeled data and unlabeled data Dr U Du. and the positive-class prior я. Inspired by previous weakly-supervised learning studies with aggregate supervision [28). we can arrange the training data as D,, U (Du, T), where we treat (Du, π as a set of instances with its approximate label proportion Accordingly, we can formulate the following set-aware empirical risk of PU learning (SAPU):
REAPU (g) R (g) loE
Thu Σ 9(xi); X,ED
where lcE denotes the cross-entropy loss. Because the size of Du can be too large, directly fitting ' in the second term of (g) may result in smoothing instance-level predictions. To alleviate this potential issue, we can randomly divide D,, into many subsets [S. where S, {xix } n. is the number of subsets, and S is the number of instances in each subset; and if S is large enough, we can also approximate the label proportion of each subset as m. Upon these ideas, we can rearrange the training data as D U {(S,, >))
and then reformulate Eq 9 as follows:
(A)P = Ret (g)
1 Thss
Σ
ECE
1
Σ
Σ
((Xi)),
(10)
5 E.
We consider SAPU as a practical candidate disambiguation frree empirical risks of PU learning. We show the following theorem to indicate that it risk-consistent for the expected risk of supervised learning. The proof is presented in the Appendix.
Lemma 3.1. Let f; s Ei= 18 = +1] be the true proportion of positive instance set S..
When the set size satisfies S > 3x log(2/6) with probability at least - 8, we have [ π|| S for each set St and Vur(ft;) < =(1-a)
the minimizer of the true classification risk and @SPPU = arg mingeg RsAPu(g) denotes the minimizer of the risk form in Eq 10 Suppose that the pseudo- dimensions of {x g(x)]g E G} and (x + (cx(g(x), ()xx))))))))) :)]g E G are finite, and there exist constants Lig, T4 such that 1g(x)l S T.g and ((C(()()) T)) & for all x E and all E G. Then, for any 5 > with probability at least 1-
Theorem 3.2. Let g" = arg
R(GsAru) - R(g') 50
(V 1og(1/0)
+0
log(1/8) ns
+ Te ....
((1 1
T) log (1/8)
Up
S
(11)
Results and analysis We empirically investigate the proposed SAPU and 4 existing disambiguation- free empirical risks with different commonly used loss functions. Table presents 6 loss functions, i.e. hinge, logistic, sigmoid, squared, ramp, and double-hinge, along with their mathematical formulations and theoretical properties. These loss functions are selected to represent diverse characteristics across 4 key properties: convexity, differentiability, symmetry, and Lipschitz continuity. Because the positive prior π for the STL-10 10 dataset is unavailable, we conduct experiments on the CIFAR- 10 and F-MNIST datasets.
We declare that the label proportion approaches n, as You goes
5

---

Table The ACC scores (meantstd) of disambiguation empirical risks with widely used loss functions on F-MNIST and CIFAR- 10. The highest scores are indicated in bold.
Dataset
Method
S
hinge
logistic
sigmoid
squared
ramp
double-hinge
uPU
68.0=0.5
68.5+0.5
69.8+0.9
77.1+2.2
70.8+1.8
68,4+0.6
nnPU
93.820.4
93.020.5
93.940.7
93.241.7
93.8+1.0
94.8+0.3
absPU
94.2+0.4
93.3+0.5
93.3+0.7
93.7=0.4
93.6+0.6
94.1+0.2
F-MNIST-I
Dist-PU
94.320.4
94.020.2
94.720.2
32
93.9+1.0
95.9+0.2
95.9+0.2
94.6#0.6
9..5=1.0
94.5+0.9
64
92.8=0.3
96.0=0.2
96.0+0.1
93.0=0.0
93.8+0.8
94.0#0.9
SAPU
128
92.8+0.1
96.1+0.0
96.240.1
91.0=0.5
93.520.8
92.940.1
256
93.020.5
96.0+0.2
96.2+0.0
90.820.1
92.920.5
92.8+0.3
7u
92.4+0.5
95.4+0.6
96.0+0.0
90.8+0.3
92.9=0.6
92.6+0.1
uPU
47.820.6
47.740.3
49.1
62.4+2.7
50.8+1.4
48.7+0.7
nnPU
92.4+0.4
91.7*11.3
91.0+0.4
92.7=0.5
93.1+0.7
93.4+0.3
absPU
92.6#0.6
91.5+0.7
91.0+1.2
92.0=0.3
92.420.7
93.4+0.4
F-MNIST-2
Dist-PU
91.3++0.9
93.3+0.66
92.2+0.3
32
94.520.5
95.720.0
95.8+0.2
94.720.5
95.320.0
94.920.3
64
94.6#0.1
95.8+0.0
95.8+0.1
92.8+0.5
94.0=#04
94.5+0.5
SAPU
128
93.6+0.4
95.9+0.2
96.0+0.1
90.8+0.2
93.2+0.4
93.9+0.3
256
93.7=0.2
95.8+0.2
96.1=0.0
88.2+1.8
93.4+0.5
93.2+0.0
92.8+0.2
95.9+0.0
96.0+0.1
88.3+1.4
93.620.4
93.5+0.4
uPU
80.5+0.7
81.7+0.9
81.6+1.9
66.1+2.4
77.3+2.3
79.9+0.7
nnPU
86.420.4
84.3+0.7
85.1+1.4
83.2+1.0
86.1
86.420.1
absPU
85.6+0.4
82.920.7
85.7
81.9+1.2
86.3+0.9
85.720.6
CIFAR- 10-1
Dist-PU
86.0+0.9
86.240.6
86,620.5
32
85.3#0.6
86.5+0.5
86.6+0.2
76.6+4.7
86.2+0.7
84.5+0.6
64
83.820.3
86.420.2
86.720.3
77.0+3.4
86.720.7
85,310.4
SAPU
84.7=0.5
85.6#0.4
87.0+0.5
79.5+1.5
85.0=0.0
83.6+0.2
256
83.520.2
86.640.3
86.8+0.2
75.345.5
85.420.6
84.420.6
84.1#0.2
85.3+0.4
86.8+0.3
78.7+2.5
8..
84.1
uPU
76.1
77.3+1.1
76.942.4
55.7+2.0
67.9+2.4
75.
nnPU
84.7+1.0
80.7+1.4
83.7+1..
81.0=1.7
8..3+1.0
83.8+1.4 .4
ahsPU
84.4+0.9
78.0+1.7
83.8+1.4
79.3+2.7
84.4=0.7
82,4+1.4
CIFAR-10-2
Dist-PU
82.1
83.4
85.6+0.7
32
60.7+0.0
85.1+0.7
85.1+0.7
81,340.3
74.746.3
74.7+6.3
62.4+1.7
85.2+0.7
84.9+0.7
81.3+0.1
74.9=6.1
75.7+5.3
SAPU
128
61.620.9
85.2+0.7
84.720.6
81.1+0.0
72.349.1
72.6+8.7
256
60.7+0.0
85.4+0.6
84.5+0.5
81.3+0.0
73.8+7.4
74.0=7.1
61.620.9
85.340.7
84.720.7
81.220.1
74.526.9
74.827.2
The experimental results in Table demoosttte that the choice of loss function significantly influences classification accuracy across different datasets and methods. For F-MNIST dataset, the sigmoid loss consistently delivers superior performance, achieving a remarkable accuracy of 96% with our method. The double-hinge loss also performs exceptionally well, particularly with nnPU method and Dist-PU,, On the more challenging CIFAR-10 dataset, the sigmoid loss still demonstrates robust performance (around 86% with SAPU), while the double-hinge loss excels in several configurations, notably with Dist-PU on CIFAR- 10-2 (85.6%). Interestingly, the effectiveness of each loss function varies substantially across different empirical risk methods and datasets. For example, while SAPU achieves optimal results with sigmoid on CIFAR-10-1 (86.8%) its performance degrades considerably with the squared loss. Additionally, our empirical analysis confirms that smooth, differentiable losses (sigmoid, logistic) achieve better compatibility with SAPU's set-aware architecture, while non-smooth losses (double-hinge, ramp) align better with traditional point-wise optimization methods. This non-uniform behavior suggests complex interaction between loss functions and model architectures

---

that cannot be reduced to simple heuristics, underscoring the importance of careful loss function selection based on specific application contexts.
Based on the above analysis, we can summarize the following guiding principles: (1) The smooth,
differentiable losses (such sigmoid, logistic loss) achieve better compatibility with the set-aware architecture of SAPU, while non-smooth losses (e.g., double-hinge, ramp loss) align better with traditional point-wise optimization methods. (2) Convex losses generally provide better optimization guarantees. (3) Simple datasets (e.g., F-MNIST) benefit from smooth losses, enabling fine-grained optimization, while complex datasets (e.g., CIFAR- 10) may require losses with stronger regularization properties.
Furthermore, the core of SAPU lies in dividing unlabeled data into multiple subsets for set-aware supervision, we further conduct experiments to verify how subset size affects the performance of the model. We systematically tested different subset sizes S = (32, 64, 128, 256, nu. across all datasets and recorded classification accuracy with various loss functions. The experimental results in Table demonstrate that S 256 yields optimal performance on most datasets, particularly when combined with the sigmoid loss function. For example, the model achieved peak accuracies of 96.2% and 96. % respectively on F-MNIST-I and F-MNIST-2 when S 256 with sigmoid loss function. For simpler datasets like F-MNIST, this phenomenon can be explained that when subsets are too small, individual subsets struggle to accurately reflect the overall label distribution; conversely. when subsets become excessively large (approaching n.,), instance- model's discriminative power. Moreover, for more complex datasets like CIFAR- 10, our experiments indicate that larger subset sizes tend to be more effective. Based on our comprehensive analysis across different datasets, we recommend setting medium-sized subsets (e.g.. S 256) as a generally effective configuration for our SAPU method.
predictions become overly smoothed, reducing the
3.3 Pseudo-labeling Methods
In this section, we investigate the pseudo-labeling techniques and thresholding techniques for selecting high-confident pseudo-labels. For comprehensive evaluations, we first suggest several base methods and then discuss the empirical results.
Base methods of pseudo-labeling We specify the generic objective of Eq with specific pseudo- labeling techniques and thresholding strategies, leading to set of base methods. First, we estimate pseudo-labels ý by hard and soft pseudo-labeling techniques; and for clarity, we review them as ý sign (Ø(4) 0.5) (-1,+1} and ÿ = ((g), respectively. Second, the thresholding strategy refers to computing a threshold value r to define the lower bound of high-confident pseudo-labels. Inspired by 22 30), we specify 3 thresholding strategies to compute т, described below:
Fixed thresholding treats the threshold value 7 as a hyper-parameter, and empirically sets it constant value. Here, we fix 0.95.
Adaptive thresholding gradually updates the threshold value during classifier training. Following the idea that the predictions can be more accurate as the classifier continues to be trained 31 we gradually increase as follows:
T Tmar min(L,t/T'.
where is the current epoch, Tmaz the maximum threshold value, and is the ramp-up period.
Class-specific adaptive thresholding gradually updates the threshold values Tp and t, for positive- and negative-classes, respectively. Following (30, we gradually update Tp and Tn as follows:
Tp - Tp x (1)
To f Tn x ((t)
where cg and clO) are the ratios between the pseudo-label accuracies of positive- and negative-classes and their higher accuracy at epoch t.
Based on these specific techniques,
specify 6 base methods of pseudo-labeling
Results and analysis To comprehensively evaluate the effectiveness of different pseudo-label strategies and generic tricks under PU learning, we compare six base pscudo-labeling methods
7

---

Table 5: The ACC scores (mean#std) base methods of pseudo-labeling on F-MNIST and CIFAR- 10. The highest scores are indicated in bold.
Label
Threshold
M.A.
Mixup
F-MNIST-1
F-MNIST-2
CIFAR-I0-1
CIFAR-10-2
90.0+1.7
92.7+0.1
85.140.6
83.1=3.8
89.9=1.6
89.342.3
84.2+1.8
82.0#3.5
Fixed
90.5+0.8
92.8+0.2
85.1+0.3
85.0#1.0
89.6+1.9
89.0+1.7
84.042.1
82.1
91.4+0.7
92.8+0.2
84.3+0.5
83.7=0.8
71.7+4,2
65,5+5,8
80,5+0.2
82.8#3.0
Hard
Adaptive
90.6+0.6
92.940.1
84.5+0.2
83.8+0.2
72.144.3
67.543.2
80.740.0
83.1+3.3
91.5+0.4
89.0+0.0
84.3#0.4
83.5=0.5
71.7+4.2
65.5+5.8
12.0=1.0
12.0=1.5
Class Adaptive
91.0=1.0
89.620.2
84.540.5
84.020.5
72.1
65.5+5.7
80.5+1.0
82.5#1.5
95.4+0.4
93.740.4
84.340.5
83.8+0.2
95.2+0.3
71.1=1.7
13.5
13.0=1.5
Fixed
95.4+0.4
93.9#0.4
84.3+0.5
83.8=0,2
95.2+0.3
70.7+2.2
13.8+1.5
83.3+1.0
95,420.4
94.1+0.3
85.940.5
82.9#3,4
74.0#4.0
69.9+3.0
83.5+1.5
83.0#1.3
Soft
Adaptive
95.5=0.4
94.2+0.4
85.9+0.5
84.8=0.0
82.9+2.4
70.7+2.2
83.9+1.3
83.4
95.0+0.1
94.1#0.4
84.3+0.5
83.8+0.2
77.2+0.1
93.7+1.5
80.8+1.5
83.0#1.5
Class Adaptive
95.5+0.4
94.6+0.4
85.6+0.5
85.8+0,2
82.9+2.4
93.8+1.1
81.0+1.3
83,4+1.1
(including two pseudo-labeling techniques (hard vs. soft labeling) and three thresholding strategies (fixed, adaptive, and class- adaptive )) and two widely used enhancement techniques (mixup and moving average) on CIFAR- 10 and F-MNIST datasets.
The results demonstrate that soft labeling consistently outperforms hard labeling, particularly when combined with class-adaptive thresholding on F-MNIST datasets (achieving up to 95.5%). Mixup proves to be the most consistent generic trick for ACC improvement across all experimental configu- rations, while moving average often leads performance degradation when combined with other techniques. Mixup proves consistently beneficial because it addresses the fundamental challenge of decision boundary uncertainty PU learning. By creating synthetic samples through convex combinations, mixup naturally smooths the decision boundaries in regions. This is particularly crucial PU learning where the model must distinguish between true negatives and mislabeled positives within the unlabeled set. In contrast, the counterintuitive phenomenon of performance degradation with moving average techniques primarily stems from the unstable nature of pseudo-labels in PU learning. Unlike traditional semi-supervised learning where unlabeled data contains truly unlabeled instances, learning involves mislabeled negative samples, making historical predictions unreliable. The self-training process generates systematic biases, and moving average perpetuates rather than corrects these biases. On the other hand, moving average techniques may suppress the model's ability rapidly adapt within the feature space to distinguish between positive and negative samples. Furthermore, the momentum parameter requires careful tuning. which significantly increases the experimental cost for hyperparameter optimization.
Overall, the combination of soft labeling with class- adaptivee thresholds and Mixup yields the best performance across nearly all datasets. The only exception occurs in the 2IFAR-10-1 dataset, likely due to its complex visual diversity as natural image dataset, making the combination of fixed thresholding with moving average and mixup more suitable for handling its complex decision boundaries. These findings suggest that the combination of soft labeling, class-adaptive thresholding. and mixup generally constitutes the most promising universal method.

---

Table 6: The ACC scores (mean:tstd) of existing PU learning methods and GPU.
Method
F-MNIST-1
F-MNIST-2
CIFAR-10-1
CIFAR-10-2
STL-10-1
STL-10-2
Rank
uPU
77.112.2
62.4122.
81.7.1099
77.3111
76.7:10.8
71.514.8
15.3
nnPU
94.8+0.3
93.4+0.3
86,4+0.1
84.7=110
77.1+4.5
81.9++1.0
8.7
absPU
94.2=0.4
93.440.3
86.31:099
84,420.9
75.3+2.2
82.0=1077
9.8
Dist-PU
94.7.10.2
93.310.66
86.7.10.5
85.6.10.7
78.31.0.8
81.5.1111
RP
9..411.6
93.340.55
78.0:1..9
84.2=1.1
71.340.8
75.5#22.6
12.2
AdaSampling
93.6110.3
93.5100.2
79.6:10.5
79.1
74.34:2.2
82.6.1.88
11.3
GenPU
78.110.4
86.2114
71.211.99
68.3
68.51114
57.311.55
16.5
Self-PU
90.8+0.4
89.1+0.7
85.1+0.8
83.9+2.6
78.5+
80.8+22.1
12.2
VPU
92.6=1.2
90.5+0.8
8..8+1.2
82.5=111
78.4111
82.920.7
10.3
PULNS
91.010.5
89.110.8
87.210.66
83.7-2.9
80.2.10.8
83.610.77
9.8
Mix--
92.6=0.4
91.8+002
88.2+004
84.7=00.5
80,2+0..9
83.7+00.7
7.8
M'Mix-C 9.68:10.6
90.440.11
88.7110.4
87.9-0.5
80.7.L0.7
84.1.1..3
6.5
Robust-PU
90.0=0055
85.5+0.7
80.0+0066
85.2=1.1
79.6+0.9
80.400.8
12.3
HolisticPU
96.2=0.11
96.010.3
91.011.3
90.420.5
82.540.5
84.01:122
LaGAM
94.9=0.2
94.1+-0.3
89.9+0.3
88.0=1144
85.3+0..3
85.0+0.3
2.8
PUL-CPBF
96.7=0.3
96,5+0.2
91.4+00.
91.0==03
83.4+0.7
85.4+1.2
1.2
GPU
96.4=0.1
96.1+0.5
88.4+0.1
87.9=0.4
82.7+00.7
84.9+0.4
3.2
PN lcarning
97.7=0.1
97.7+0.1
91.9+t.1
91.9=0.1
86.0+0.66
86.0=#0.6
3.4 Proposed GPU Framework
By integrating SAPU with pseudo-labeling, suggest an efficient PU learning framework GPU. Its generic objective is given as follows:
Lopu(g)= = Rt (g) Culg.0:
Σ na j=1
lce
1
Σ XijES,
g(xis),
(12)
S
where a is coefficient parameter.
We can interpret GPU as regularized pseudo-labeling method PU learning, where the set-aware term is treated as regularization term. Based on the previous evaluations, find that pseudo- labeling methods depend on high quality of pseudo-labels in the early training stage because they are in self-training manner. Accordingly,
suggest warm-up stage by minimizing the objective of SAPU. In addition, we can specify the pseudo-labeling techniques and thresholding strategies according to our empirical observations.
Results and analysis To evaluate the efficacy of our proposed GPU framework compared to existing PU learning methods, we conduct experiments on F-MNIST, CIFAR- 10, and STL-10 datasets to assess its general performance across varying scenarios. For comprehensive comparison, we also include PN learning (i.e., supervised learning) as an upper bound baseline. Our GPU implementation uses subset size 5 = 256 with the sigmoid loss function and employs soft pseudo-labeling with class-adaptive thresholding based on our empirical observations. For the warm-up stage, we train using only SAPU for 20 epochs before introducing the pseudo-labeling component.
As evident from Table 6 our proposed GPU framework demonstrates competitive performance across all benchmark datasets, ranking 3.2 overall, tied with HolisticPU and slightly behind LaGAM (2.8), while remaining competitive with the leading PUL-CPBF. For example, GPU achieves accuracy scores of 96.4% and 96. 16.1% 1% on F-MNIST-1 and F-MNIST-2 respectively, which are comparable to the best- best-performin, PUL-CPBF (96.7% and 96.5%). For CIFAR-I0. 10. GPU obtains 88.4% and 87 87.6%, positioning it among the top-tier methods but slightly below HolisticPU and PUL-CPBF. Similar competitive performance is also demonstrated on the STL- 10 dataset. The performance gap between methods reflects our focus on exploring fundamental techniques and
GPU and the best-performi integrating them with our novel set-aware empirical risk SAPU, rather than employing sophisticated techniques like ensemble methods in PUL-CPBF, trend detection in HolisticPU, meta-learning in

---

LaGAM. GPU provides a general framework that can integrate future advances, while specialized methods may not generalize well. Notably, GPU significantly outperforms traditional PU learning methods across all datasets, demonstrating that combining set-aware empirical risk estimation with pseudo-labeling strategies effectively enhances the discriminative capability of the model.
Discussion and Future Works
In this paper, we comprehensively review the current families of PU learning and investigate their basic characteristics. We review the existing disambiguation-free empirical risks and suggest novel set-aware empirical risk SAPU from the perspective aggregate supervision, which is risk-consistent for the expected risk of supervised learning. We empirically evaluate them with various commonly applied loss functions. In addition, we review the basic techniques and widely applied generic tricks, methods. To empirically evaluate
i.e. mixup and moving average, in the existing them, we formulate a set of base methods specified by hard and soft pscudo-labeling techniques with thresholding strategies for selecting high-confident pseudo-labels such as fixed, adaptive, and class-specific adaptive thresholding strategies. Finally, we propose an efficient PU learning framework GPU by integrating SAPU with pseudo-labeling. GPU involves warm-up stage by minimizing SAPU and specify the framework according to our empirical observations. We compare GPU with the existing PU learning methods, and the empirical results demonstrate that GPU can be practical benchmark in PU learning, and is scalable for future pseudo-labeling techniques.
In the future, there two potential problems that require more attention. One basic problem is how to estimate more accurate pseudo-labels 33 34 since we only investigate the straightforward pseudo-labeling techniques. Some advanced techniques such ensemble learning [35] demonstrate strong potential. Another problem is whether the existing PU learning can be effective for the scenarios with scarce positive labeled instances and how to deal with such scenarios, which can appear in many real-world applications.
Acknowledgements
We would like to acknowledge support for this project from the National Science and Technol- ogy Major Project (N0.20212200115500) and the National Natural Science Foundation of China (No.62276113..
References
[1] Kou, J. Wang, Y. Jia, et al. Progressive label enhancement. Pattern Recognition, 160:111172. 2025.
[2] Inaccurate label distribution learning. IEEE Transactions Circuits and Systems for Video Technology, 34(100)10237-1044 2024.
[3] Instance-dependent inaccurate label distribution learning. IEEE Transactions on Neural Networks and Learning Systems, 36(1) (6(1)):445-14339 2025.
[4] Bekker, J., J. Davis. Learning from positive and unlabeled data: A survey. Machine Learning, 109(4),7 2020.
[5] du Plessis, M. C., G. Niu, M. Sugiyama. Convex formulation for learning from positive and unlabeled data. In International Conference on Machine Learning, pages 1386-1394. 2015.
[6] Goodfellow, I., Pouget-Abadie, M. Mirza, et al. Generative adversarial networks. Communi- cations of the ACM, 1391141139-14 2020.
[7] Zhang, H., M. Cissé, N. Dauphin, et al. mixup: Beyond empirical risk minimization. In International Conference on Learning Representations. 2018.
[8] Kiryo, R., G. NIu, M. C. du Plessis, et Positive-unlabeled learning with non-negative risk estimator. In Neural Information Processing Systems, pages 1675-1685, 2017.
10

---

[9] Hammoudeh, Z., D. Lowd. Learning from positive and unlabeled data with arbitrary positive shift. In Neural Information Processing Systems. 2020.
[10] Zhao, Y. Q. Xu, Y. Jiang, et al. Dist-pu: Positive-umlabeled learning from a label distribution perspective. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14441-14450. 2022.
[11] Chen, X., W. Chen, T. Chen, et al. Self-pu: Self boosted and calibrated positive-unlan unlabeled training. In International Conference on Machine Learning, pages 1510-1519. 2020.
[12] Chen, H.,, F. Liu, Y. Wang, et al. variational approach for learning from positive and unlabeled data. In Neural Information Processing Systems. 2020.
[13] Li, C., X. Li, L. Feng, et al. Who your right mixup partner positive and unlabeled learning. In International Conference on Learning Representations. 2022.
[14] Zhu, Z., L. Wang, P. Zhao, al. Robust positive-unlabeled learning via noise negative sample self-correction. In ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 3663-3675 2023.
[15] Wang, X., W. Wan, C. Geng, et Beyond myopia: Learning from positive and unlabeled data through holistic predictive trends. In Neural Information Processing Systems. 2023.
[16] Li, C., Y. Dai, L. Feng, et al. Positive and unlabeled learning with controlled probability boundary fence. In International Conference on Machine Learning. 2024.
[17] du Plessis, M. C., G. Niu, M. Sugiyama. Analysis of learning from positive and unlabeled data. In Neural Information Processing Systems, pages 703-711. 2014.
[18] Northcutt, C. G., T. Wu, I. L. Chuang. Learning with confident examples: Rank pruning for robust classification with noisy labels. In Conference Uncertainty in Artificial Intelligence. 2017.
[19] Hinton, G., O. Vinyals, J. Dean. Distilling the knowledge in a neural network. arXiv preprint 1xXv::553300231/ 2015.
[20] Niu, G., M. C. du Plessis, T. Sakai, et al. Theoretical comparisons of positive-unlabeled learning against nesitive-negative learning. In Neural Information Processing Systems, pages 1199-1207 2016.
[21] Elkan, C. Noto. Learning classifiers from only positive and unlabeled data. ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 213-220. 2008
K.
[22] Yang, P., W. Liu, J. Y. H. Yang. Positive unlabeled learning via wrapper- adaptive sampling. In International Joint Conference on Artificial Intelligence, pages 3273-3279. 2017.
[23] Hou, M., B. Chaib-Draa, C. Li, et al. Generative adversarial positive-unlabeled learning. In International Joint Conference on Artificial Intelligence, pages 2255 22.1. 2018.
[24] Luo, C., P. Zhao, C. Chen, et al. PULNS: positive-unlabeled learning with effective negative sample selector. AAAI Conference on Artificial Intelligence, pages 8784-8792. 2021.
[25] Long, L., H. Wang, Z. Jiang, et al. Positive-unlabeled learning by latent group-aware meta disambiguation. In JEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20138-23147.00 2024.
[26] Zamzam, O., H. Akrami, M. Soltanolkotabi, al. Learning a disentangling representation for PU learning, arXiv preprint arXiv:2310:03833, 2024.
[27] Wang, H., R. Xiao, Li,, et al. Pico+: Contrastive label disambiguation for robust partial label learning, arXiv preprint arXiv:2201 0299448944 2022.
[28] Busa-Fekete, R., H. Choi, Dick, et al. Easy learning from label proportions. In Neural Information Processing Systems, pages 14957 14968. 2023.

---

[29] Sohn, K. D. Berthelot, N. Carlini, et al. Fixmatch: Simplifying semi-suuprviied learning with consistency and confidence. Neural Information Processing Systems, 33:596-608, 608, 2020.
[30] Zhang, B., Wang, Hou, et al. Flexmatch: Boosting semi-supervised learning with curriculum pseudo labeling. Neural Information Processing Systems, 34: 18408-18818,, 2021.
[31] Berthelot, D., R. Roelofs, K. Sohn, al. Adamatch: A unified approach to semi-supervised learning and domain adaptation. arXiv preprint arXiv:2106.04732, 2021.
[32] Kou, Z., S. Qin, H. Wang, et al. Label distribution learning with biased annotations by learning multi-label representation, 2025.
[33] Kou, Z., Wang, Tang, al. Exploiting multi-label correlation in label distribution learning. 2024.
In International Joint Conference on Artificial Intelligence, pages
[34] Kou, Z., H. Xuan, J. Zhu, et al. Tail-aware reconstruction of incomplete label distributions with low-rank and sparse modeling. IEEE Transactions on Circuits and Systems for Video Technology, pages 1-1, 2025.
[35] Li, C., Y. Dai, L. Feng, et al. Positive and unlabeled learning with controlled probability boundary fence. In International Conference on Machine Learning. 2024.
[36] Hoeffding, W. Probability inequalities for sums of bounded random variables. The collected works of Wassily Hoeffding, pages 409-426, 1994.
12

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We have succinetly outlined the contributions of this paper in both the abstract and introduction sections, and the results presented in the experiments section robustly substantiate the effectiveness our proposed method.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We have created a separate "Limitations" section in our paper, the details can be found in Appendix C.I
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and prool's
Question: For each theoretical result, does the paper provide the full of assumptions and complete (and correct) proof?
13

---

Answer: [Yes]
Justification: We analyze the estimation error of SAPU in Theorem 3.2 All theoretical
results are clearly supported by rigorous mathematical derivations in Appendix A and
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: We have elaborated on the implementation principles and details of the method to facilitate the reproduction of the main experimental results presented in our paper. Additionally, we have submitted our code and datasets in the Supplementary Material.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g.. in the of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
14

---

5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification:
We have submitted our code and datasets in the Supplementary Material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.. cc/
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ //ips.cc/public. wwwpppppc/gguueaeecceeeeeooooepe gewdddeeeooeeeuumiiieeeeeeee Ctw///oooeemmsiiieeeaee www.bbmiiiiiooococcccc. for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide detailed descriptions of all necessary training and testing proce- dures in Section 3.1 Furthermore, all experimental setup details can be readily found the submitted code.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: We report the results regarding the standard errors of the mean our experi- ments and ensure that our paper contains the calculation method for standard errors along with other essential information related them.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
15

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We have provided sufficient information on the computer resources needed reproduce our experiments in Section 3.1
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't fidn't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: /thn iet.eiiieeeeitccccciieeeeeeeee
Answer: [Yes]
Justification: We ensure that research adheres to the NeurlPS Code of Ethics in all aspects.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We have created separate "Broader Impacts" section in our paper, the details can be found in Appendix C.2
Guidelines:
16

---

The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No, they should explain why their work has societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deploymentt of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper pose such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We have provided proper citations for all models, code, and datasets utilized in our paper.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
17

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
If assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, papprsaipphaaeeocccaaappp thcode. com/ com/dataseta has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's ' creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We have submitted the proposed new assets in the Supplementary Material. And the submitted files include structured explanatory documents regarding these new assets.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/codeemmodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and sereenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
18

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does not involve LLMS as any important, original, non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy hatt:p///earrrppcc///ooeeeeeeee/ //t/////ooeeeeeccc////////..1. for what should or should not be described.
19

---

holds with probability at A Proof of Lemma 3.1
The boundary of the bag deviation Under the SCAR assumption 21 each sample in the unlabeled dataset has an independent probability of being positive. Given bag containing samples, since the variance of Bernoulli random variable is x T), we can obtain a tighter bound using Bernstein's s inequality (36] for any 0:
Se²
P([3) - ग| 2 c) < 2 2exp(
S
(13)
2T π) + 2
Then,
S> 3T (1 )
log( (2/))
(14)
26
The variance of ăj random variables:
The variance of fty can be given by the mean of S independent Bernoulli
Var(ft)) T T(1 T
(15)
S
B Proof of Theorem 3.2
decompose the excess risk as:
R(gsAPU) - R(gg) z RR(ōsAPU) RSAPU (ŌsaPU))
RRAPO (ŌsAPo) (saPu(g")
RsaPU (g')
R(g"))
Term
(16)
For Term 1, using the uniform convergence theory and the fact that the deviation in bag proportions bounded by c, we have:
dlgg(n,) log(1/8) + C2p
dlog(n=) log(1/8)
[R(Sspp)) - (Orog) S Ci
+ cle
np
n
(17)
where Le is the Lipschitz constant of the cross-entropy loss; C1 and C2 are universal constants; is the seeudodimennson of the function class.
According to Hoeffding's inequality, for any 8 > 0, | T| < least 1 5. Then, we have:
log (2/5) 28
dlog(n,)) + +og((1/))
dogg(s)) - 1og(1/8)
log (2/8)
[R(ssaPU) REAPU (irag) S scp
C21
Le
np
ns
28
(18)
For Term 2, the deviation comes from the difference between the true positive class prior т and the bag proportions ft.. Using the variance bound from Lemma 3.1 T.Tam and applying Jensen's 's inequality:
T(1
T
RsAPu(g") -R(g") & Ley El( - x)21 = Ley Var(ft)) Le
(19)
S
20

---

Then.
dlog(np) + log(1/6)
dog((n)) + log(1/0)
R(osAru) - R(g") C/
C21
np
ns
(2/()
T(1- TTL (1--n m
Le
+ Lo\
28
S
d llog(mp)
+ log(1/6)
d dllo((n)) + log(1/6)
+ +G2p
"pp
(2/(2
T T))
Le
25
5
=0
(v 10g(1/0)
log(1/6) ns
+ Te (1/20 (1001-1)000 log(1/0)
xt)
Up
5
(20)
C Limitations and Broader Impacts
C.1
Linitations
Despite our comprehensive empirical study, accurately estimating pscudo-labels remains challenging. especially with limited positive samples. Our methods could be further improved by incorporating advanced techniques such as ensemble learning to generate more reliable pseudo-labels. Additionally, the set-aware empirical risk method may face challenges with extremely imbalanced datasets where the positive class prior becomes difficult to estimate accurately.
C.2 Broader Impacts
Our GPU framework introduces novel perspective by integrating empirical risk with pscudo-labeling methods, enhancing PU learning applicability real-world scenarios such as medical diagnoses and fraud detection. The proposed set-aware empirical risk extends the theoretical foundation of PU learning through aggregate supervision, which could inspire further weakly-supervised learning research. By making PU learning more reliable with limited labeled data, our work contributes to reduced annotation costs and broader accessibility of machine learning in resource-comee cesource-oossrriieed environments.
21


Reviews:
[{"review_id": "fh4jmAUUr9", "reviewer": "Reviewer_7j8V", "summary": "This work conducts empirical studies on PU learning methods and proposes a novel PU framework, which combines set-aware empirical risk with pseudo-labeling techniques. Extensive experimental analysis demonstrates the effectiveness of the PU framework.", "questions": "Please refer to the “Weaknesses”.", "limitations": "Yes.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\nThis work innovatively proposes the GPU framework that combines set-aware empirical risk and pseudo-labeling techniques, featuring a modular structure for easy portability and extension. The work provides theoretical proof of risk consistency for the SAPU method with detailed derivations, and is well-written with clear organization.\nWeaknesses:\n1.\tWhen the SCAR assumption is violated, are the methods in this work still applicable?\n2.\tWhile Theorem 3.2 provides convergence guarantees, this work lacks deeper theoretical analysis, such as convergence rates and other fundamental properties.\n3.\tWhy does the incorporation of moving average techniques lead to performance degradation? This phenomenon appears counterintuitive and requires explanation.\n4.\tThe GPU framework's performance improvements over existing methods remain within statistical error margins in most cases, and even fails to achieve optimal performance in some scenarios. What accounts for this limitation?", "was_revised": true, "final_justification": "The authors' rebuttal has addressed my concerns. I maintain a positive score for this work.", "rebuttal": "**Q1. When the SCAR assumption is violated, are the methods in this work still applicable?**\n\nThank you for your valuable comment. Unlike traditional PU methods that heavily rely on instance-level SCAR assumptions, SAPU operates through set-level supervision and the property of the positive sample selection mechanism being relatively independent of the feature distribution, which provides inherent regularization against selection bias. Therefore, when the SCAR assumption is violated, our method demonstrates certain robustness due to its aggregate supervision design. However, we acknowledge that when the SCAR assumption is severely violated, additional importance weighting or bias correction techniques are required. In the future, we plan to extend our work to develop adaptive strategies to handle SCAR violations.\n\n\n&nbsp;\n\n\n**Q2. Theorem 3.2 lacks deeper theoretical analysis.**\n\nThank you for your valuable suggestion. We would like to clarify that our theoretical analysis does provide several important contributions beyond basic convergence. Theorem 3.2 establishes finite-sample bounds showing that the excess risk $R(\\hat{g}\\_{\\text{SAPU}}) - R(g^*)$ converges at rates $ O\\left(\\sqrt{\\frac{\\log(1/\\delta)}{n_p}}\\right) + O\\left(\\sqrt{\\frac{\\log(1/\\delta)}{n_s}}\\right) + L_\\ell \\cdot O\\left(\\sqrt{\\frac{\\pi(1-\\pi)\\log(1/\\delta)}{S}}\\right) $, which explicitly characterizes how the performance depends on the number of positive samples ($n_p$), number of subsets ($n_s$), subset size ($S$), and the positive class prior ($\\pi$).\n\nAdditionally, Lemma 3.1 provides precise characterization of the deviation bounds for our set-aware method, showing that when $S \\geq \\frac{3\\pi(1-\\pi)\\log(2/\\delta)}{2\\epsilon^2}$, we achieve $|\\hat{\\pi}_j-\\pi|\\le \\epsilon $ with high probability, which directly informs the practical choice of subset size.\n\nThe theoretical analysis we provide serves to establish the soundness and risk-consistency of our SAPU method, which is sufficient to support our empirical findings and validate our proposed GPU framework.\n\n\n\n&nbsp;\n\n\n**Q3. Why does the incorporation of moving average techniques lead to performance degradation?**\n\nThank you for your comment. This counterintuitive phenomenon of performance degradation with moving average techniques primarily stems from the unstable nature of pseudo-labels in PU learning. Unlike traditional semi-supervised learning where unlabeled data contains truly unlabeled instances, PU learning involves mislabeled negative samples, making historical predictions unreliable. The self-training process generates systematic biases, and moving average perpetuates rather than corrects these biases. On the other hand, moving average techniques may suppress the model's ability to rapidly adapt within the feature space to distinguish between positive and negative samples. Furthermore, the momentum parameter requires careful tuning, which significantly increases the experimental cost for hyperparameter optimization.\n\n\n&nbsp;\n\n\n**Q4. The GPU framework's performance improvements over existing methods remain within statistical error margins in most cases.**\n\nThank you for this question. Regarding the limited performance improvements of the GPU framework, we acknowledge that compared to state-of-the-art methods, GPU's improvements are modest. This reflects several factors: First, strong baseline methods like PUL-CPBF and HolisticPU employ sophisticated techniques (ensemble learning, meta-learning) that are orthogonal to our contributions; Second, our focus is on fundamental technique analysis and principled integration, rather than achieving performance maximization through complex engineering; Third, GPU provides a general framework that can integrate future advances, while specialized methods may not generalize well. In the future, we will improve upon this by conducting experiments on more challenging real-world datasets and demonstrating GPU's scalability through the integration of ensemble techniques."}, {"review_id": "zUahdxIVH0", "reviewer": "Reviewer_tME1", "summary": "This paper contributes systematic analysis and practical improvement methodologies to the PU learning through fine-grained empirical investigation. The authors systematically organize PU learning methods into two fundamental classes: unbiased empirical risk methods and pseudo-labeling methods. For unbiased empirical risk methods, the authors propose a novel set-aware empirical risk (SAPU) from the perspective of aggregate supervision. For pseudo-labeling methods, systematic ablation analyses are conducted. Finally, the authors present GPU framework, achieved by unifying SAPU with pseudo-labeling strategies.", "questions": "1.\tIn Table 1, why is the P³Mix classified as a \"hard pseudo-labeling\" method? Should not the mixup technique naturally produce \"soft pseudo-labels\"?\n2.\tRegarding Table 4: a) Why do other methods generally demonstrate superior performance with double-hinge loss, while SAPU consistently shows relatively poorer results with double-hinge loss? b) Different loss functions exhibit varying performance across different datasets. How can this phenomenon be explained? Are there general selection guidelines available?\n3.\tThe description about class-specific adaptive thresholding is unclear in lines 210-214. In FlexMatch, $\\tau_p$ is fixed at 0.95, but here it appears to continuously change based on the previous epoch's threshold. Additionally, how is $ \\mathcal{C} $ computed?\n\nMinor Issues:\n\na) Unify notation: SAPU or saPU. \n\nb) Eq.10: $ j=1 $ should be $ i=1 $. \n\nc) Line 145: $ \\mathcal{S}_ i=\\{\\mathbf{x}_ {ij}\\}_ {i=1}^{S} $ should be $ \\mathcal{S}_ i=\\{\\mathbf{x}_ {ij}\\}_ {j=1}^{S} $.", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1.\tThis paper not only categorizes PU learning into unbiased empirical risk methods and pseudo-labeling methods from a novel perspective, but also provides fine-grained analysis of specific techniques within each class, encompassing multiple dimensions including loss functions, pseudo-labeling techniques, threshold strategies, and general tricks. \n\n2.\tThis paper introduces the concept of aggregate supervision into PU learning, proposing the SAPU empirical risk framework, and subsequently presents the GPU framework by integrating it with pseudo-labeling related strategies. \n\n3.\tThe analysis is thorough and the experiments are comprehensive.\n\nWeaknesses: \n\n1.\tThis paper fails to provide a clear classification standard for the distinction between \"soft pseudo-labeling\" and \"hard pseudo-labeling.\" Additionally, this paper lacks clear interpretation of results in the loss function section or general selection guidelines.\n\n2.\tThe technical classification and methodological descriptions are inadequately detailed. Key technical descriptions lack clarity, particularly the specific computational process for class-specific adaptive thresholding, which requires more explicit explanation.\n\n3.\tThis paper exhibits issues with inconsistent symbolic representation (e.g., SAPU vs saPU) and formula writing errors throughout the manuscript.", "was_revised": true, "final_justification": "The authors have provided clear definitions and justifications that resolved my concerns regarding the classification of P$^3$Mix and the adaptive thresholding mechanism. I have increased my score accordingly.", "rebuttal": "**Q1. Why is the P$^3$Mix classified as a \"hard pseudo-labeling\" method? This paper fails to provide a clear classification standard for the distinction between \"soft pseudo-labeling\" and \"hard pseudo-labeling.\"**\n\nThank you for your comments. The P$^3$Mix method indeed falls under the category of \"hard pseudo-labels.\" While mixup techniques inherently generate soft labels, our classification criterion is based on the final form of the labels used for training, rather than the intermediate processing steps. After applying mixup, P$^3$Mix still uses a hard threshold to generate binary pseudo-labels for the final training objective.\nTo address the classification confusion, we formally define the distinction between hard and soft pseudo-labels as follows: pseudo-labels used for training are considered hard pseudo-labels if they are discrete binary values {$-1, +1$}, and soft pseudo-labels if they are continuous probability values $[0, 1]$.\nBased on this criterion, P$^3$Mix is classified as a hard pseudo-label method because its final training objective uses thresholded binary labels, even though the mixup process generates soft labels. We will clarify this definition in the revised version.\n\n\n&nbsp;\n\n\n**Q2. Regarding Table 4: a) Why SAPU consistently shows relatively poorer results with double-hinge loss? b) Different loss functions exhibit varying performance across different datasets. How can this phenomenon be explained? Are there general selection guidelines available?**\n\nThank you for your suggestions. First, SAPU's underperformance with double-hinge loss stems from a fundamental structural incompatibility between our set-aware mechanism and the mathematical properties of loss function. Our SAPU employs cross-entropy loss for the set-level supervision term (Eq.9-10), which expects continuous probability outputs and relies on smooth gradient flow for effective optimization. In contrast, double-hinge loss is piecewise linear and promotes discrete, margin-based decisions through its sharp transitions at $z=1$ and $z=0$. This creates conflicting optimization dynamics where the set-aware component pulls toward smooth probabilistic outputs while the instance-level double-hinge component pushes toward sharp margin-based decisions. Additionally, our empirical analysis confirms that smooth, differentiable losses (sigmoid, logistic) achieve better compatibility with SAPU's set-aware architecture, while non-smooth losses (double-hinge, ramp) align better with traditional point-wise optimization methods.\n\n\n&nbsp;\n\n\n**Q3. The description about class-specific adaptive thresholding is unclear in lines 210-214.**\n\nThank you for your comments. Unlike FlexMatch where $\\tau$ is fixed at 0.95, our method dynamically adjusts both $\\tau_p$ and $\\tau_n$ based on the historical performance of pseudo-label quality. We first compute the accuracy rates for positive and negative pseudo-labels. Specifically, the accuracy of positive pseudo-labels at epoch t $\\text{Acc}_p^{(t)}$ is the ratio of the number of correctly predicted positive pseudo-labels to the total number of positive pseudo-labels (, and similarly, the accuracy of negative pseudo-labels $\\text{Acc}_n^{(t)}$ is the ratio of the number of correctly predicted negative pseudo-labels to the total number of negative pseudo-labels). Then the confidence ratios $C_p^{(t)}$ and $C_n^{(t)}$ are calculated as: \n\n$$ C_p^{(t)} = \\frac{\\text{Acc}_p^{(t)}}{\\max(\\text{Acc}_p^{(1 :t-1)})} , \\quad \\quad C_n^{(t)} = \\frac{\\text{Acc}_n^{(t)}}{\\max(\\text{Acc}_n^{(1:t-1)})} $$\n\nwhere $\\max(\\text{Acc}_p^{(1:t-1)})$ represents the highest positive pseudo-label accuracy achieved in previous epochs. The thresholds are then updated:\n$$\\tau_p = \\tau_p \\times C_p^{(t)} , \\quad \\quad  \\tau_n = \\tau_n \\times C_n^{(t)} $$\nThis mechanism provide adaptive confidence requirements that evolve with the model's improving discrimination capability.\n\n\n&nbsp;\n\n\n**Q4. Minor Issues.**\n\nThank you for your corrections. We will revise them in the next version."}, {"review_id": "ySRws1uodq", "reviewer": "Reviewer_Bw6s", "summary": "This paper categorizes existing PU learning methods into two types and summarizes or improves them, respectively. For the first type of methods, this paper proposes SPU based on aggregate supervision and proves that it is risk-consistent. It also achieves good performance. For the second type of methods, this paper summarizes their techniques and evaluates the tricks used in these methods. Finally, this paper combines these two types of methods and proposes GPU, which demonstrates excellent performance on multiple datasets.", "questions": "1. Why does the choice of loss function in Table 4 have such a significant impact? How should we choose an appropriate loss function?", "limitations": "Yes.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n1. This article provides a thorough discussion of many existing works and offers many valuable tricks.\n2. The performance of GPU in most scenarios is promising.\n3. The article is clearly written.\n\nWeaknesses:\n\n1. The novelty is limited. SAPU is simply a straightforward application of aggregate supervision in PU learning.\n2. This article seems to be merely a summary of existing methods. Although it indicates that certain tricks are effective, it does not explain why.\n3. This article lacks a detailed introduction to aggregate supervision.\n4. The datasets used in this paper are too small and too simple. The authors should conduct experiments on larger and more realistic datasets.", "was_revised": true, "final_justification": "The detailed and thoughtful rebuttal has addressed most of my concerns. My concern about novelty, which is my major concern, is well replied by the authors. The lack of introduction to aggregate supervision is promised to be handled in the revised version. But the choice of dataset still need to be improved. Thus, I decided to keep my score.", "rebuttal": "**Q1. SAPU is simply a straightforward application of aggregate supervision in PUL.**\n\nThank you for this comment. While our SAPU is inspired by aggregate supervision, we argue that this is not a straightforward application but represents a principled innovation based on risk estimation theory. First, SAPU is built upon the theoretical foundation of disambiguation-free empirical risks, where we innovate in the risk estimation term by replacing the estimated negative risk term $\\widehat{R}\\_u^-(g) - \\pi \\widehat{R}\\_p^-(g)$ (which causes negative risk issues in flexible models) with a theoretically grounded set-aware cross-entropy term that avoids optimization instabilities. Second, our subset partitioning strategy with formal variance analysis provides theoretical guarantees on proportion estimation accuracy, which is showed in Lemma 3.1 of our paper. More importantly, this work aims to establish a practical benchmark for PU learning by providing comprehensive empirical analysis and a unified framework. The integration framework GPU demonstrates how to effectively combine risk estimation with pseudo-labeling through careful design principles. So the novelty of our work lies not just in individual components but in establishing methodological principles that can guide future PU learning research.\n\n\n&nbsp;\n\n\n**Q2. Although it indicates that certain tricks are effective, it does not explain why.**\n\nThank you for your suggestions. \n\n**Mixup** proves consistently beneficial because it addresses the fundamental challenge of decision boundary uncertainty in PU learning. By creating synthetic samples through convex combinations, mixup naturally smooths the decision boundaries in regions. This is particularly crucial in PU learning where the model must distinguish between true negatives and mislabeled positives within the unlabeled set.\n\nThe counterintuitive phenomenon of performance degradation with **moving average** techniques primarily stems from the unstable nature of pseudo-labels in PU learning. Unlike traditional semi-supervised learning where unlabeled data contains truly unlabeled instances, PU learning involves mislabeled negative samples, making historical predictions unreliable. The self-training process generates systematic biases, and moving average perpetuates rather than corrects these biases. On the other hand, moving average techniques may suppress the model's ability to rapidly adapt within the feature space to distinguish between positive and negative samples. Furthermore, the momentum parameter requires careful tuning, which significantly increases the experimental cost for hyperparameter optimization.\n\n\n&nbsp;\n\n\n**Q3. This article lacks a detailed introduction to aggregate supervision.**\n\nThank you for your comment. Aggregate supervision refers to learning from group-level constraints rather than instance-level labels. In traditional supervised learning, we have individual labels $y_i$ for each instance $x_i$. In aggregate supervision, we only know aggregate statistics about groups of instances, such as the proportion of positive instances in a set. Our method leverages this principle by treating the unlabeled dataset $D_u$ with class prior $\\pi$ as aggregate supervision. Instead of trying to predict individual labels, we divide $D_u$ into subsets $ \\\\{\\mathcal{S}_i\\\\}\\_{i=1}^S$ and enforce that each subset's predicted positive proportion approximates $\\pi$. This strategy avoids the challenging individual pseudo-labeling problem and provides a natural regularization for overfitting. We will add the explanation about aggregate supervision in the next version.\n\n\n&nbsp;\n\n\n**Q4. Why does the choice of loss function in Table 4 have such a significant impact? How should we choose an appropriate loss function?**\n\nThank you for these comments. The significant impact of loss function on the performance of SAPU reflects a fundamental mathematical incompatibility rather than algorithmic deficiency. Our SAPU employs cross-entropy loss for the set-level supervision term (Eq.9-10), which expects continuous probability outputs and relies on smooth gradient flow for effective optimization. In contrast, double-hinge loss is piecewise linear and promotes discrete, margin-based decisions through its sharp transitions at $z=1$ and $z=0$. This creates conflicting optimization dynamics where the set-aware component pulls toward smooth probabilistic outputs while the instance-level double-hinge component pushes toward sharp margin-based decisions. It explains why SAPU shows sensitivity to loss function compared to traditional disambiguation-free empirical risks (*e.g.,*  uPU, nnPU).\n\nBased on the above analysis, we can summarize the following guiding principles: \n\n(1) The smooth, differentiable losses (such as sigmoid, logistic loss) achieve better compatibility with the set-aware architecture of SAPU, while non-smooth losses (such as double-hinge, ramp loss) align better with traditional point-wise optimization methods. \n\n(2) Convex losses generally provide better optimization guarantees. \n\n(3) Simple datasets (*e.g.,* F-MNIST) benefit from smooth losses, enabling fine-grained optimization, while complex datasets (*e.g.,* CIFAR-10) may require losses with stronger regularization properties."}, {"review_id": "AB47ZF5xsn", "reviewer": "Reviewer_REBb", "summary": "This paper investigates a classic problem: PU Learning. It analyzes and compares two common types of algorithms for this problem (based on ERM and Pseudo-label), and proposes a new algorithm (GPU) by integrating the two categories based on empirical studies. The proposed method is evaluated through experiments on three typical datasets.", "questions": "**1. Problem**\n\nIn real-world applications, one of the key challenges in PU Learning is the unknown positive sample distribution $\\pi$. Although techniques like MPE [1]/BBE [2] can estimate it, their accuracy in high-dimensional data remains uncertain. Could the author please discuss whether this issue was considered in the research?\n\n**2.Related Work**\n\nWhile the review of PU Learning is comprehensive, several representative studies have not been discussed or compared. For example: [2, 3, 4]. Could the author please include a discussion on how these works relate to the research? Comparing the method with these studies would help clarify the unique contributions and positioning of the approach within the field.\n\n**3. Method**\n* What is the specific motivation for combining SAPU and pseudo-labeling in the method? Which particular challenges in PU Learning does this combination address?\n* Similar algorithmic designs (ERM && pseudo-labeling) exist commonly in other weakly supervised learning problems [5] (e.g., UU Learning [6], CL Learning [7, 8], Learning with New Class [9, 10]). Would the method be applicable to these scenarios as well? Elaborating on the generalizability of the approach across related learning problems would add depth to the discussion.\n\n**4. Experiments**\n\nTo enhance the reproducibility and interpretability of the results, could the author please provide the following details:\n* Why were different backbones (LeNet-5 && CNN) used for different datasets instead of a unified one?\n* What was the rationale behind setting different epoch numbers for each dataset?\n* How were the positive and negative categories defined for each dataset? Different classification methods can lead to drastically different problem difficulties, as the difficulty of distinguishing between some classes is low while that of others is high.\n* Could the author elaborate on the process of setting and tuning hyperparameters (e.g., learning rates, weight decay)?\n\nProviding these details would greatly assist readers in understanding the experimental design and reproducing the results, thereby strengthening the credibility of the findings.\n\n[1] Adjusting the outputs of a classifier to new a priori probabilities: a simple procedure\n\n[2] Mixture proportion estimation and PU learning: A modern approach\n\n[3] Positive-unlabeled learning with label distribution alignment\n\n[4] Positive-unlabeled learning using random forests via recursive greedy risk minimization\n\n[5] Machine Learning from Weak Supervision: An Empirical Risk Minimization Approach\n\n[6] On the minimal supervision for training any binary classifier from only unlabeled data\n\n[7] Learning from complementary labels\n\n[8] Unbiased risk estimators can mislead: A case study of learning with complementary labels.\n\n[9] Handling New Class in Online Label Shift\n\n[10] An Unbiased Risk Estimator for Learning with Augmented Classes", "limitations": "yes", "rating": 2, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 1, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\n* The paper is well-structured and written in a fluent manner, making it easy to follow and understand.\n* It provides a comprehensive review of PU Learning, demonstrating a solid grasp of the research landscape in this field.\n\n**Weaknesses**\n\nThe core research objective of this work remains ambiguous: \n* If the study aims to establish a benchmark, expanding the experimental scope with more diverse datasets, detailed parameter configurations (e.g., learning rates) would significantly enhance reproducibility; \n* If the study intends to propose a novel algorithm, the performance needs substantial enhancement, as the method currently fails to outperform state-of-the-art approaches under most experimental settings. \n\nAddressing this ambiguity and reinforcing either the benchmark rigor or algorithmic superiority would strengthen the paper's scientific contribution.", "was_revised": true, "final_justification": "In summary, while the paper provides a comprehensive empirical study and practical implementation guidelines, its core contributions lack the originality and depth required for acceptance at this venue. I encourage the authors to refine their research focus and explore more impactful innovations in future work.", "rebuttal": "**Q1. Could the author discuss whether the issue of estimating the positive sample distribution was considered in the research?**\n\nThank you for your comment. While unknown positive class prior is indeed an important consideration in real-world applications, we would like to clarify our research positioning. Our work focuses on investigating the fundamental characteristics and techniques within existing PU learning paradigms, rather than addressing the prior estimation problem specifically. The assumption of known positive class prior is widely adopted in the PU learning literature.\n\nOur work focuses on investigating the fundamental characteristics and techniques within existing PU learning paradigms, rather than addressing the prior estimation problem specifically. The assumption of known π is widely adopted in recent PU learning methods we compare against, as it allows for focused investigation of other critical aspects such as empirical risk design and pseudo-labeling strategies. Additionally, our proposed SAPU framework is orthogonal to prior estimation techniques, and can readily incorporate prior estimation methods like MPE/BBE as a preprocessing step. We believe combining SAPU with robust prior estimation represents a valuable direction for future work.\n\n\n&nbsp;\n\n\n**Q2. Several representative studies have not been discussed or compared. Could the author include a discussion on how these works relate to the research?**\n\nThank you for your suggestion. [2] addresses the fundamental challenge of estimating the positive class prior $\\pi$. As discussed in our response to the first question, this work is orthogonal to our research, since our method assumes $\\pi$ is known to focus on empirical risk design and pseudo-labeling strategies.\n\n[3] represents an extension of Dist-PU that we discussed in our disambiguation-free empirical risks family. [3] addresses the challenge of distribution mismatch between the positive labeled data and the positive instances within unlabeled data, which can violate the SCAR assumption. The label distribution alignment technique provides a way to handle such distribution shifts, making it a valuable enhancement to the Dist-PU.\n\n[4] presents an alternative algorithmic method using random forests rather than neural networks we primarily focus on.\nWe will expand the related work and discuss their relationship to our proposed method in the next version.\n\n\n&nbsp;\n\n\n**Q3. What is the specific motivation for combining SAPU and pseudo-labeling in the method? Which particular challenges in PU Learning does this combination address?**\n\nThank you for your questions. The motivation for combining SAPU and pseudo-labeling lies in addressing the complementary limitations of existing disambiguation-free empirical risks and pseudo-labeling methods in PU learning. Our empirical analysis revealed that disambiguation-free empirical risks, while theoretically grounded, may suffer from gradient instability, particularly when the negative empirical risk $ R_u^-(g) - \\pi R_p^-(g) $ becomes negative. Conversely, pseudo-labeling methods can capture complex instance-level patterns but are highly dependent on pseudo-label quality, especially in early training stages where predictions are unreliable. By combining SAPU with pseudo-labeling into our GPU framework, we address both challenges simultaneously: SAPU provides stable set-level supervision through aggregate constraints that regularize the learning process, while pseudo-labeling enables fine-grained instance-level discrimination. This combination creates a more robust training objective.\n\n\n&nbsp;\n\n\n**Q4. Elaborating on the generalizability of the approach across related learning problems (e.g., UU Learning, CL Learning, Learning with New Class).**\n\nThank you for this insightful question. The core innovation of our SAPU method lies in leveraging aggregate supervision through set-level label proportions, which naturally extends to various weakly supervised learning paradigms where similar distributional information is available. For instance, in UU learning, where we have two unlabeled datasets from different distributions, SAPU can be adapted by treating each unlabeled set with its estimated class proportion, formulating a cross-entropy loss that encourages the model to match these proportions at the set level. The mathematical foundation remains consistent: instead of $\\pi$ representing the positive class prior in PU learning, we would have $\\pi_1$ and $\\pi_2$ representing the class distributions in the two unlabeled datasets.\n\nFor learning with new class, our GPU framework becomes particularly relevant. The pseudo-labeling component can be adapted to handle the discovery and classification of new classes by incorporating uncertainty estimation and cluster-based pseudo-labeling strategies. The set-aware supervision from SAPU can help maintain class proportion constraints even when new classes emerge.\n\n\n&nbsp;\n\n\n**Q5. Experimental design choices (backbones, epochs, and class definitions).**\n\nThank you for your comment. We want to clarify that our experimental setup follows well-established practices in the PU learning community [1], which we cited and followed for consistency and fair comparison. The different epoch settings reflect established convergence patterns observed across the PU learning community for these standard benchmark datasets. F-MNIST typically reaches convergence faster due to its simpler feature space and lower intra-class variation, while CIFAR-10 and STL-10 require longer training periods to adequately learn the more complex decision boundaries present in natural images. These settings align with the training schedules used in the reference methods we compare against, ensuring that our experimental conditions provide fair and meaningful comparisons.\n\n[1] Positive and unlabeled learning with controlled probability boundary fence.\n\n\n&nbsp;\n\n\n**Q6. Could the author elaborate on the process of setting and tuning hyperparameters (e.g., learning rates, weight decay)?**\n\nThank you for your comment. We set learning rates as $5e-3$ for F-MNIST and $1e-4$ for others, and set weight decay as $1e-6$. We will add them in the next version."}]

Author rebuttal:

Meta-review: This paper makes a strong contribution by conducting a systematic and fine-grained empirical analysis of PU learning methods, revealing several previously unknown patterns such as the consistent advantage of sigmoid loss and the unexpected degradation caused by moving average techniques. These insights provide valuable, evidence-based guidelines for researchers and practitioners working in weakly supervised learning. 

The introduction of the aggregate supervision perspective and the SAPU formulation offers a fresh theoretical lens, with finite-sample guarantees that extend the conceptual foundations of PU learning. The GPU framework further demonstrates a principled integration of risk estimation and pseudo-labeling, presented in a modular and reproducible form. While one reviewer expressed concerns about novelty and dataset scale, other reviewers highlighted the technical soundness, clarity, and systematic nature of the work, with two reviewers recommending acceptance. 

On balance, the benefits to the community from the empirical insights, theoretical perspective, and unified framework outweigh the concerns. We believe this paper will serve as a useful reference point and inspire further innovation in PU learning and broader weakly supervised learning research.
Decision: Accept (poster)