Unbiased Recommender Learning from Implicit Feedback via Weakly Supervised Learning
Authors: Hao Wang
Zhichao Chen
Haotian Wang
Yanchao Tan
Licheng Pan
Tianqiao Liu
Xu Chen
Haoxuan Li
Zhouchen Lin
Venue: ICML 2025 poster (2025)
Keywords: recommendation
collaborative filtering
Primary area: applications->everything_else

Abstract: Implicit feedback recommendation is challenged by the missing negative feedback essential for effective model training. Existing methods often resort to negative sampling, a technique that assumes unlabeled interactions as negative samples. This assumption risks misclassifying potential positive samples within the unlabeled data, thereby undermining model performance. To address this issue, we introduce PURL, a model-agnostic framework that reframes implicit feedback recommendation as a weakly supervised learning task, eliminating the need for negative samples. However, its unbiasedness hinges on the accurate estimation of the class prior. To address this challenge, we propose Progressive Proximal Transport (PPT), which estimates the class prior by minimizing the proximal transport cost between positive and unlabeled samples. Experiments on three real-world datasets validate the efficacy of PURL in terms of improved recommendation quality.  Code is available at https://github.com/HowardZJU/weakrec.

Unbiased Recommender Learning from Implicit Feedback via Weakly Supervised Learning
Hao Wang' Zhichao Chen
Haotian Wang² Yanchao Tan Licheng Pan
Tianqiao Liu Xu Chen
Haoxuan Li Zhouchen Lin 678
Abstract
et al., 2018), and entertainment (Gomez- Uribe and Hunt, 2016). By accurately predicting user preferences, recom- mendation systems play pivotal role in enhancing user experience and increasing business revenues (Gao et al., 2022a;b; Zhang et al., 2025; Zheng et al., 2025).
Implicit feedback recommendation is challenged by the missing negative feedback essential for effective model training. Existing methods of- ten resort to negative sampling, a technique that assumes unlabeled interactions as negative sam- ples. This assumption risks misclassifying poten- tial positive samples within the unlabeled data, thereby undermining model performance. To address this issue, we introduce WeaklyRec, a model-agnostic framework that reframes implicit feedback recommendation as a weakly supervised learning task. climinating the need for negative samples. However, its unbiasedness hinges on the accurate estimation of the class prior. To ad- dress this challenge, we propose Progressive Prox- imal Transport (PPT), which estimates the class prior by minimizing the proximal transport cost between positive and unlabeled samples. Experi- ments on three real-world datasets validate the ef- ficacy of WeaklyRec in terms of improved recom- mendation quality. Code is available at https: //gi thub. com/ Howa rdZJU/ weak/ee..
The core of recommendation system is the handling of user feedback, which can be broadly categorized into explicit and implicit types. Explicit feedback, such as ratings, comments, like/dislike tags, reflects user preferences faithfully but is challenging to collect, as it requires active user participa- tion, potentially diminishing user experience. Conversely, implicit feedback, such as page views or click logs, is read- ily available and does not require active user participation. Therefore, recommendation with implicit feedback, termed as ImplicitRec, has gained significant attention in industrial applications (Togashi al., 2021; Lee et al., 2022).
A unique challenge in ImplicitRec is missing negative feed- back (MNF) (Lim et al., 2015; Yang et al., 2018). Specif- ically, the available data only consists of positive and un- labeled samples. This absence negative feedback com- plicates the task of differentiating between items that are either disliked or simply unexposed to the user across the unlabeled interactions (Zhou al., 2021; Ren al., 2023; Wang et al., 2021). For instance, a user's failure to click on video might not necessarily indicate disinterest; it could sim- ply mean the user was not exposed to that particular video. Treating unlabeled interactions as negative instances (Zhou et al., 2022; Kang and McAuley, 2018) ignores the exis- tence of positive samples within the unlabeled data, thereby introducing bias and compromising recommendation perfor- mance (Wang al., 2021).
1. Introduction
Recommendation systems aim to capture user preferences through their feedback, subsequently offering personalized content or product suggestions. They find extensive applica- tions across various domains, including e-commerce (Smith and Linden, 2017; Wang et al., 2022), advertising (Zhou
To counteract the MNF issue, one line of works advocate downweighting unobserved samples (Hu et al., 2008; Liang et al., 2016), employing pairwise ranking loss (Rendle et al., 2009). While promising, these methods are largely heuristic and lack a solid theoretical foundation for unbiasedness. Recent works have shifted focus towards the construction of theoretically unbiased estimators of the ideal risk, using only positive and unlabeled samples. Representative methods such as RMF (Saito et al., 2020), UBPR (Saito, 2020) and CIMF (Zhu et al., 2020) primarily leverage propensity re-
'Zhejiang University, NNational University of Defense Technol- ogy *TAL Education Group *Remmin University of China 'Center for Data Science, Peking University 'State Key Lab General AI School of Intelligence Science and Technology, Peking Uni- versity 'Institute for Artificial Intelligence, Peking University "Pazhou Laboratory (Huangpu), Guangzhou, China. Correspon- dence to: Haoxuan Li <hxli@ wtt..pku.cduucm.... Zhouchen Lin <zlin .e.....u.cn>..
Proceedings of the 42nd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Unbiased Recommender Learning from Implicit Feedback via Weakly Supervised Learning
weighting, a methodology rooted in causal inference (Li et al., 2023b), which offers unbiased estimates provided accurate propensity score estimation (Li et al., 2023c;a). However, identifying accurate propensity scores remains an elusive goal in ImplicitRec.. Furthermore, the derived risk estimators do not necessarily maintain non-negativity (Zhu et al., 2020; Saito, 2020; Saito et al., 2020), posing the risk of non-convergence. Consequently, the MNF issue remains an open challenge in ImplicitRec that warrants investigation.
definition of ImplicitRec. Subsequently, we offer concise overview of optimal transport, mathematical framework that is intrinsically linked to the proposed PPT method for class prior estimation.
2.1. Implicit Feedback Recommendation
Let u and I denote the sets of users and items, respectively, and X' = U x I represent the set of all possible user-item interactions, where each sample is denoted by x = (u, i). We introduce Y as the binary indicator variable for positive feedback and designate Pdata as the underlying distribution goveming all interactions. The objective to construct a recommendation model, symbolized as g. that is proficient in discriminating between samples associated with positive and negative feedback. To quantify the performance of the model, we employ an error measure {() y), where y signi- fies the ground truth feedback and ý represents the model's estimated feedback. In line with existing literature (Saito et al., 2020; Luo et al., 2021), the ideal risk function is formulated as:
This work introduces Weakly Supervised Recommendation (WeaklyRec), a model-agnostic framework based on learn- ing from weak supervision paradigm (Sugiyama et al., 2022), to develop rigorous formulations and practical so- lutions for handling MNF without reliance on vulnerable propensity scores. Specifically, we first reformulate the ideal risk and construct an empirical estimator that obviates the need for negative feedback. We show that this estimator is unbiased, consistent with the ideal risk, and amenable to optimization via stochastic gradient descent, with unique at- tainable optimum for representative recommender families. However, the estimator's performance is contingent upon a predefined class prior, which requires expert knowledge for empirical selection. To address this, we develop Pro- gressive Proximal Transport (PPT) method which achieves automatic class prior estimation by minimizing the proximal transport cost between the positive and unlabeled samples. Extensive evaluations on three publicly available real-world datasets substantiate the efficacy of the proposed method.
Rideal = (x.aaa [[((()).)) v)l
(1)
The samples with negative feedback, i.c., Pdata (xly = = 1), is unavailable in ImplicitRec due to the MNF issue, render- ing the ideal risk in (1) incalculable. Current approaches typically assume that unobserved samples are all negative, resorting to negative sampling on these samples gener- ate negative feedback. However, this assumption does not hold in real-world practice as the negatively sampled data inevitably contain a proportion of positive feedback. Assign- ing negative labels to them can lead to an unstable training process and compromise overall performance (Rendle et al., 2009; Saito, 2020).
The main contributions of this paper are summarized below:
We propose WeaklyRec, model-agnostic framework that addresses the ImplicitRee problem using weakly super- vised learning. Its core component is risk estimator that eliminates the need for negative samples and is theoreti- cally unbiased with respect to the ideal risk, provided that accurate class prior estimation is available.
2.2. Optimal Transport
Optimal transport (OT) is a mathematical framework de- signed to measure the discrepancy between two distribu- tions by calculating the minimum transport cost, which has been used in many fundamental fields due to its empirical flexibility and theoretical rigor: computer vision (Courty et al., 2017; Xu et al., 2021; Shen et al., 2018). Initially, Monge (1781) first formulated it as finding an optimal map- ping between two continuous distributions. However, this formulation had limitations in terms of the existence and uniqueness of solutions. A more practical formulation was later proposed by Kantorovich (2006), with an entropic regularizer for enabling fast computation via Sinkhorn algo- rithm (Altschuler et al., 2017). We provide its specification for comparing two empirical distributions in Definition 1. Definition 1. For empirical distributions a and 8i.m with n and m samples, respectively, the Kantorovich problem aims to find an optimal plan π E T Rm that minimizes the
We introduce PPT, an optimal transport-based strategy for accurate class prior estimation, which is a necessary factor to ensure the unbiasedness of WeaklyRec.
We validate the efficacy of WeaklyRec on three real-world datasets, where the recommendation performance is im- proved and the class prior is accurately estimated.
2. Preliminary
In this section, we establish technical preliminaries essential for comprehending the proposed methods. Initially, we de- lineate the fundamental symbols and formalize the problem
'The term propensity means the likelihood that the sample would be observed. It is quite difficult to be estimated in the realm of ImplicitRec due to MNF, since the samples that are observed but disliked is unknown.
2

---

Unbiased Recommender Learning from Implicit Feedback via Weakly Supervised Learning
transport cost (a, 8) I between a and B. Formally, the problem is defined
samples are inaccessible due to MNF, workaround in- volves leveraging the unlabeled data distribution, which is mixture of positive and negative data distributions:
W(a, B) = min ==II(
(C,
(2)
(u(2) = kipP((2) (x) + K.Pu (2)).
11(a, B) = {IT E R=x=; - = = =b}
On the basis of this, the negative risk Rg(g) can be ex- pressed in terms of the unlabeled and positive samples:
where C E R2xm denotes the sample-wise distance be- wween a and B. 1n and 1m are column vectors filled with ones. The set II defines feusible transport plans. and b specify the mass of units in a and B.
Kin Rn (g) = Kin Knn (e)())) -1)]
/ [(g(z), - -1)p
Kn
Recent research in OT primarily follows two trajectories. The first aims to reduce the computational complexity of solving OT problems. While exact solutions can be obtained through linear programming algorithms, these come with cubic complexity in relation to the number of samples (Bon- neel et al., 2011). To address this, various approximate al- gorithms for acceleration have been developed, such as the Sinkhorn and sliced OT algorithm (Altschuler et al., 2017) with quadratic and linear complexity, respectively. The sec- ond line of research focuses on modifying the transport problem to suit specific applications (Wang et al., 2025a;b). Examples include the Schrödinger bridge problem in gen- erative modeling (Marino and Gerolin, 2020), the Gromov problem in graph matching (Xu et al., 2019), and the weak transport problem in causal inference (Wang et al., 2023).
/ (l((z)) - (z)dz
(4)
Kip / /Uolo)) - (x)d
-Ep_((((t)) -1)] Kip Mo ((((t))) -1]]-
Introducing {(g(x)) = {(g(2)) +1) {(g(z), -1), we can
eliminate the expectation over the negative samples in (1)
to derive surrogate of the ideal risk:
Rweak =
=k0) Ep [[((())) + E,, ([((x))-1)] (5)
where subtracting the risk of misclassifying positive sam- ples as negative compensates for the bias introduced by treating all unlabeled samples as negative. Consequently, the WeaklyRec risk excludes unavailable negative feedback, enabling empirical estimation using positive and unobserved samples:
3. Methodology
This section presents proposed WeaklyRec framework. We first construct a surrogate estimator of the ideal risk (1) utilizing only positive and unlabeled samples. A thor- ough theoretical analysis is provided to expound upon its advantageous attributes. We then formulate the Progres- sive Proximal Transport (PPT) method to achieve automatic estimation of the class prior. The overall workflow of Weak- IyRec is finally encapsulated for implementation.
Tlp
Rwealk (g)
Kip
{(g(x)))
Eg(xx)) 1). (6)
7p
Theoretical Justification. We demonstrate that the Weak- lyRec estimator is unbiased relative the ideal risk en- abling training and evaluation without reliance on negative samples (Theorem 1). Moreover, adding more unlabeled samples consistently reduces estimation error (Theorem 3), suggesting an opportunity to improve ImplicitRec perfor- mance. Thanks to the convexity of Rwwaa(g), the conver- gence of WeaklyRec is guaranteed with bounded errors in mild conditions (Lemma 3). Detailed proofs are provided in Appendix B.
3.1. Weakly Supervised Recommendation
Assuming that positive and negative samples arise from marginal distributions Pp(x)) (alta( = 1) and Pa( P = -1), respectively, the data distri- bution can be expressed as:
Pdata (xpp(()) (x))
where Kp = ply = +1) and Kin = ply = 1) are the class priors, satisfying Kip + Kn = 1. Decomposing the ideal risk in (1) yields:
3.2. Progressive Proximal Transport
The estimator in (6) necessitates a predefined positive class prior Kp. a.k.a., the proportion of positive samples con- cealed within unlabeled data. Although canonical positive- unlabeled learning scenarios often assume Kp to be known, such an assumption untenable in ImplicitRec where the class prior is costly and difficult to obtain, thereby necessi- tating class prior estimation. However, the task is difficult
Rideal (g) = Kyp TE, ((g(x), +1)] ) Ki, Ep,. (((g(x), -1]
= Rr Rp (9) Ki, Rn(g),
where Rp(g) and Rg) denote the expected error on pos- itive and negative samples, respectively. Since negative
3

---

Unbiased Recommender Learning from Implicit Feedback via Weakly Supervised Learning
in ImplicitRec due MNF. Existing methods are mainly based on partial alignment (du Plessis et al., 2015a; 2017) which minimizes the f-divergence between labeled and unla- beled distributions. However, these approaches suffer from an inherent positive bias (Sugiyama et al., 2022) and are ill suited for data based on sparse ids,
PL
T7
11
P1
B
4p
On
(a) w 1
(b) w 2
(C)
10
Figure I: Overview of the PT transport plan under varying mass weight parameters w. The dark fields in /3 demarcate the domain containing samples that are transported. Lines connecting a and B signify the transport plans π ((x, w:
Definition 2 (Proximal Transport, PT). Suppose a and Frmm are empirical distributions corresponding to samples with positive and unlabeled feedback, respectively. Given mass weight E 14,, we aim to find a transport plan T E Igax minimizing the transport cost between and 3. Formally, the problem is defined as:
The computation workflow of WeaklyRee. Input: a mini-bate of samples x, recommender model g with user and item embedding look-up table U and V. Parameter: ry: learning rate; wi-k: mass weights.
Algorithm 1
TT" (a, w) = arg min mem (C, TT),
[Ta, 3: w)= = ( E I IIm a, π In S w b, 1. mlm= = =1],
Output: updated embedding look-up tables Ul and Vi.
I: Tm Finn, < split(x)..
2: uº, v. t llook-up(U,, V v. ,x), + concat (uf, v!). i = 1: "pp
where (), represents the transport cost; C E R denotes the sample-wise distance" between a and B. In and 1 m are column vectors fillers with ones. The set 11 defines the feasible transport plans. a and b specify the mass of units in a and 3, respectively, and are implemented as 1n/n I and Im/ m under the assumption of uniform mass distribution.
3: uy. v2 t -look-up(U, V az)) 2 t
concat v.). j = nu.
4: Cjj t 2 2/ C (ii
5: t t arg mig minn KY iws) (C, k = 1:
6: 2 + (C, + KL(TTT Inp.
np: In k=
To address this gap, we introduce the Progressive Proxi- mal Transport (PPT) method, specifically designed for class prior estimation in the context of our WeaklyRee frame- work. The core concept of PPT the proximal transport (PT) problem, as outlined in Definition 2. Unlike the tradi- tional Kantorovich problem, our approach adjusts the mass of 3 by weight factor w 2 1 and modifies the mass preser- vation equality to allow for non-matched samples in 3. Fig- uree illustrates how the PT transport strategy evolves with different values of w. When w = 1, PT matches all sam- ples in B, effectively reducing to the standard Kantorovich problem. As increases, PT allows each sample in to be matched with multiple samples in a. As a result, the positively labeled samples a will be matched towards the nearest sub-distribution within 3, driven by minimizing the transport cost in (7).
7: k- + min Wk=IK,, 2+11/w8
8: Rweak <- 22 E {g (n(,,, vP),+)))+ +
mp
DE l (g (u., (g(")) v?) -1).
U + U - 1 Vu V + - 1) Vy Rweak.
negative samples are heterogeneous. The class prior is then estimated as R. = 1/w", justified by the fact that only ap- proximately 1/w" of the samples in /3 are matched by PT as illustrated in Figure 1.
The PT developed in Definition 2 is inherently a linear programming problem and thus can be solved with well- established convex optimizers. Interestingly. PT can be viewed as an augmented Kantorovich problem (Xu et al., 2020), which enables the utilization of specialized optimal transport solvers, known for superior convergence rates and computational efficiency. A comprehensive analysis of this reformulation is provided in Appendix B.2.
The proposed PPT approach leverages this property of PT to estimate the class prior Kp by progressively adjusting the weight w. Starting with large initial value of w, the matched samples in 99 are primarily positive, they are ex- pected to be closely aligned with a. As w decreases, more samples in 33 are considered for matching, without signif- icantly increasing the transport cost due to the substantial overlap between the positive samples labeled and unlabeled. Upon reaching a threshold w", the inclusion of negative samples becomes unavoidable, leading to a sharp increase in the transport cost, since the distributions of positive and
3.3. Model Implementation
In this section, we delincate the implementation of Weak- lyRec employing matrix factorization as exemplar rec- ommender model. The computational workflow for round of parameter updating is outlined in Algorithm 1.
Initially, we sample a mini-batch of samples, denoted as x. This batch is subsequently partitioned into two sub- sets: positively- samples x. zi,,, and unlabeled sam-
Here, calculate the sample-wic wise distance with the squared Fuclidean metric in line with Altschuler et al. (2017).

---

Unbiased Recommender Learning from Implicit Feedback via Weakly Supervised Learning
Table 1: Recommendation performances in terms of NDCG@K on Yahoo! R3, Coat and KuaiRec.
Dataset
Yahoo! R3
Coat
KuaiRec
Metries
NDCG@1
NDCG@3
NDC@@5
NDCG@1 1
NDCG@3
NDCG@5
NDCG@1 1
NDCG@3
NDCG@5
RMF
0.632+t0.003
0.666=a.003
0.706
0.374
0.403
0.442+t0.021
0.21 000
+0.000
0.212-
CRMF
0.003
0.667 0.03
0.706.10.003
0.377 ::00000000 00.022
0.401 10.018
0.445
0.0023
0.212 005
0.218
0.222.20.002
WMF
00.07
0.737
.033
0.772+0.002
0.389 2089900.023 025
0.425=0.023
0.466_n.002
0.226=0.00 009
0.24441.000
0.261=0.000 000008
UBPR
0.794 0.755+0.0uu
001
0.826+0.00m
0,476+4.u2
0.509
0.541 +0.007
0.248
0.265
0.281
#0.006
CUBPR
0.755
0.794=
0.826=0.0002
0.478
0.509=4.013
0.542+000007
0.2.54
0.268:+0.007
0.284
UPL
0.759 0.798
.022
0.829 100003
0,512
0.508
0.539 0.0012
0,250 006
0.265
0.28 = 005
VAE
0.650.0.002
0.68320.004 .044
0.719=0.00
0.465=20.021
0.499=0.017
0.545
0.422220.00 0o6
0.434
0,448
DCF
0.796.4..02 0.762+6.00%
0.826
00544
0.538
0,566
0.448
+0457
0,466-
ORACLE
0.652
0.732
0,443
0.470_g..012
0.507
0.204 0014
0.225
0.246_.0.008
CDR
0.669 0.695 003
0.727 0.0004
0.398 0.005
0.420
0.457 0.0014
0,491 1 012
0.425
1.0005
0,412 0.004
SDR
0,688 0.666 001
0.719=0.000
0.452
0.467:0.002
0.50220.018
0,493 2093 006
0,400=0.010 0.41910:010
UIDR
0.679
0.705
0.737 00.038
0.382+#.015
0.400=a.011
0,436
0.493=u.085
0,446++0.00
0,435=a.050
WeaklyRee
0.784]0.003
0.814 0.843
0.552$a.031
0.555 0.58910.015
0.498=0.008 0.486 0.488"aa.04
Note: The results are reported mean--ad with runs. The best and second best metries are bolded and underlined, respectively Tha
marks the metrics that WeaklyRee surpasses the best haseline with p-value < 0.05 over paired samples t-test.
plcs zY.n, (step 1). Following this partitioning, we look up the embedding tables to generate user and item embeddings (steps 2-3). The sample-wise distances between these em- beddings are then computed (step 4). For each candidate mass weight wk. we calculate the transport strategy by solv- ing the PT problem in (7) and calculate the associated trans- port cost (steps 5-6). Notably, the cost is augmented with a KL divergence term to facilitate comprehensive matching of the positive samples within the unlabeled set.
itRec studies (Saito et al., 2020; Saito, 2020; Ren et al., 2023). In Yahoo! R3 and Coat, user-item pairs with ratings above are labeled as positive, and the rest as negative, In KuaiRec, records with counts below two are considered negative, while the others are positive. Each dataset is chronologically split into training, validation, and testing sets in ratio of 0.8:0.1:0.1.
Baselines. In line with Ren et al. (2023), the collection of baselines encompasses models as follows.
Upon identifying the optimal mass weight wi- that min- imizes the transport cost, we estimate the class prior as = 1/wx- (step 7). Subsequently, we employ & cal- culate the surrogate unbiased risk (5). Notably, while the surrogate risk (5) is inherently non-negative, its empirical approximation in (6) does not preserve this non-negativity. If é is not upper-bounded, the empirical risk (6) may lack a lower bound, thereby risking model overfitting (Kiryo et al., 2017: Chaudhari and Shevade, 2012). This challenge is not exclusive to our framework; it has also been observed but not adequately addressed in prior methodologies within the ImplicitRec field (Saito et al., 2020; Saito, 2020). Therefore, we nullify the negative values of 7 by setting them to zero, so that negative terms are excluded from gradient calculation (step 8). Finally, the embedding tables are updated using stochastic gradient descent (step 9).
RecVAE (Shenbin et al., 2020) and DCF (He et al., 2024) are representative heuristic ImplicitRec models. WMF (Hu et al., 2008), RMF (Saito et al., 2020) and the clipped CRMF (Saito et al., 2020) are unbiased Implic- itRec methods with posnt-wise error measures.
UBPR (Saito, 2020), CUBPR (Saito, 2020) and UPL (Ren et al., 2023) are unbiased ImplicitRec methods with pair-wise error measures. UPL (Ren et al., 2023) integrates the point-wise and pair-wise error measures, which to our knowledge is currently the state-of-the-at method tailored for ImplicitRec.
Oracle is the recommender model that is trained using the ideal loss, but suffers from data sparsity of available explicit feedback (Wang et al., 2022) and selection bias. SDR (Li et al., 2023c), CDR (Song et al., 2023) and UIDR (Li et al., 2024) correct this bias via propensity weighting, where propensities are heuristically estimated following Saito et al. (2020).
4. Experiments
4.1. Experimental Setup
Dataset.
We utilize three real-world datasets: Yahoo! R3, Coat, and KuaiRec. These datasets are selected since they uniquely provide negative feedback for evaluating model performance and include unbiased test sets that simulate production environments, aligning with established Implic-
Training Protocol. We use matrix factorization as the primary recommendation model, following existing stud- ies (Ren et al., 2023; Saito et al., 2020; Saito, 2020; Li et al., 2023c). Experiments are conducted with two Intel
5

---

Unbiased Recommender Learning from Implicit Feedback via Weakly Supervised Learning
Table 2: Recommendation performances in terms of Recall @k on Yahoo! R3, Coat and KuaiRec.
Dataset
Yahoo! R3
Coat
KuaiRec
Metries
Recall @1 1
@ 3
@
Recall @1 1
@
@
Recall @
Recall @
Recall 5
RMF
0.042
0.132-#0.003
0.226
0.054
0.170
00.222
0,008 000
0.022+to.002
0.037.40.001
CRMF
0.132 0.04210.002
0.004
0.226
0.055 0..014
0.168 10.018
0.289
0.009 002 0.029 10.003
0.047:10.00
WMF
0.09420.002
0.203 .03
0.288
0.061=0.011
0.19220.025
0.311=0.090
0.015
003
0.058
0.104_10.00s
UBPR
0.265 0.025
0.344
0.113 +0. ous
0.270+0.0u0
0.375
0.023
0.069
+0.00
0.115 +0.005
CUBPR
0.126:40.003
0.265
0..44-00.002
0.1 13 009
0.270+0.0009
0.378
0.024
002
+0.008
0. 090000
UPL
0.129:10.002
0.268 0002
0.346:1..00
0.118 a 009
0.369 0.256
010
0,022 003
0.070
0.117 0.008
VAE
0. 0.149=0.00 0.054
0.2.3420.000
0.096_20.012
00267
0.389
0.110=0.001
0.183
0.214=10.000 .022
DCF
0.262 0..00u
0.339 00.04
0.134
0.272++0.24
0,386
0.119 #0.10
0.231 +0.005
0.288
ORACLE
0.005
0.162
004
0.258
0,085 20. 009
0.350 0.2333+0.009
0.011
0.052 00.001
0.101
CDR
0.066
0.157 0 003
0.235
0.000
0.064 012
0.185 1:0.018
0.297 017
0.140
0.191
0.207 0.0002
0033
SDR
0.149 0.001
003
0.223 2223320.002
0,085
0.337 0.224
0.138
002
0.191 0.184
UIDR
0.075
0.168
0,245 #0.077
0.060
0.163
0.
0.268
0.133=0.007
0.206
0.235+0.008
WeaklyRee 0.14620.002 0.2802a.o08 0.3542...002
0.131 #0.010 0.3021x.019 0.425
0.137 0.137=00.00 20.002 0.245 0.29640.004
Note: The results are reported in mean with runs. The best and second best metrics are bolded and underlined, respectively, meat marks the metrics that WeaklyRee surpasses the best haseline with p-value < 0.05 over paired samples t-test.
Xeon Platinum 8383C CPUs (2.70 GHz) and eight NVIDIA GeForce RTX 4090 GPU. We tune the learning rate in
MnU
Ous
0.32
niFU
Our.
{0.005, 0.01, 0.053. batch size in (256, 512, 1024, 2048), and embedding size in {8. 16, 32). All experiments are im- plemented in PyTorch using the Adam optimizer (Kingma and Ba, 2015) with early stopping (patience = 5). The candi- date mass weights (wL) are set to [5, 10, 20, 50, 100) which balances searching spectrum and efficiency.
NDCG@K
Recall
0.24
0.16
0.00
8
3
8
Figure 2: Varying risk estimator results on KuaiRec.
Evaluation Protocol. In the evaluation phase, the nega- tive feedback is employed for performance evaluation. Two commonly used metrics are adopted for evaluation: Top-k recall (Recall @ k)) and Top-k normalized discounted cumu- lative gain (NDCG@ k). We report results on Recall @ [1, 3, 5} and NDCG@ (1, 5). To perform a significance test, all experiments are repeated for 5 times.
logistc
sigreed squared
Ingidiic
SQuared
0.32
NDCG@K
RecaillaK
0.24
0.16
0.08
0.00
3
4.2. Overall Performance
Figure 3: Varying error measure (f) results on KuaiRec.
The evaluation of our proposed methods and the baselines is summarized using the NDCG in Table 2 and Recall metrics in Table 5. We outline the key findings as follows:
UPL (Ren et al., 2023) approach, which combines point-
wise and pair-wise strategies, is notably effective.
Conventional point-wise baselines exhibit practical per- formance but have inherent limitations: the sensitivity to small propensity scores leads to large estimation vari- ances. Techniques like CRMF address this by capping propensities below a certain threshold, which introduces a bias and compromises performance.
Our WeaklyRee approach outperforms baselines across most datasets and metrics. This superiority may come from its unbiasedness and consistency without theoretical guarantees. Crucially, WeaklyRec operates independently of propensity score identification, avoiding issues related to small propensities and inaccurate propensity estimation in baselines (Saito, 2020; Saito et al., 2020).
The pair-wise baselines such UBPR and CUBPR also rely on propensity scores and hence share the limitations of point-wise baselines but generally outperform them. This improvement may come from the pair-wise loss, which penalize lower ratings for unlabeled samples rather than fixing them as negatives. Among the baselines, the
4.3. Discussion on the WeaklyRee estimator
In this section, we investigate the performance of various error measures and weakly supervised learning estimators in the WeaklyRee framework.
6

---

Unbiased Recommender Learning from Implicit Feedback via Weakly Supervised Learning
2
po R
Transport cost
o m to aaa
0.05
0.3
0.1
0.5
ou ago
2000 2eduee3ssaaae www.seeesssnnn.....
20
2
1
0
2
(a) w 1, Kp 0.5.
(b)
1.5, Kp 0.5.
(c)
w 2, Kp 0.5.
(d) Transport cost with w.
Figure 4: PT transport plan (a-c) and cost (d) with different w. Circular (rectangular) markers indicate positive (negative) samples. Blue (red) markers indicate labeled (unlabeled) samples.
Table 3: Varying fixed class prior results.
Ap
W
AW
NDCG@3
ANDCG@3
NDCG@5
ANDCG@5
Recall@:
ARecall
Recall
ARecall@5
0.2
6.1143
0,4022
0,4232
0,2102
0.2732
0.005
5.5727
8.85%.
0.4696
16.75%*
0.4717
11.46%
0.2293
9.08%
0.2857
4.57%
0.01
5.5727
8.85%
0,4731
17.62%
0.4781
12.97%
0.2326
10/65%
0.2930
7.24%
0.02
5.5716
8.87%
0.4847
20.51%
0.4867
15.00%
0,2419
15.08%/
0.2954
8.12%
0.05
5,5620
9.03%
0.4847
20.51%f
0.4871
15.09%
0.2455
16.79%
0.2979
9.04%
5,5951
8.49%
0.4427
10.06%f
0.4564
7.84%
0.2296
9.22%
0.2886
5.63%
Note: A denotes the relative deviation from the results with Rp = 0.2.
Varying risk estimator results. We incorporate three widely-used weakly supervised learning estimators into the ReaklyRec framework: uPU (du Plessis et al., 2014), nnPU (Kiryo et al., 2017), and distPU (Zhao et al., 2022). Our empirical results in Figure 2 indicate that all three es- timators deliver commendable performance. Specifically, uPU, a seminal work in this area, offers theoretically unbi- ased risk estimates. However, it suffers from consequent overfitting due to the absence of lower bound on the loss function. Subsequent nnPU and distPU address this issue by introducing non-negative negaaiiit constraints, result- ing in improved performance. However, they impose non-negativity constraints in an expectation sense, over- looking the role of individual non-negative terms within the expectation and thereby compromising sample ef- ficiency. In contrast, our implementation advocates to adding non-negative constraint for each term individu- ally, which enhances sample efficiency and marginally improves performance across six out of eight metrics.
4.4. Discussion on the PPT Approach
Owing to missing negative feedback, the actual class prior k in unavailable. To showcase the efficacy of PPT in class prior estimation, we make two efforts as follows.
4.4.1. NUMERICAL VERIFICATION
We conduct a case study using a simulated dataset with a pre- defined class prior Kp. Figure 4 (a-c) illustrates the dataset and transport strategies for different class prior estimates The key observations are summarized follows:
Different values of w result in distinct matching strategies म. At w = 1, PT reduces to the standard Kantorovich problem, matching all unlabeled samples. With 1.5, PT matches positively labeled samples with concealed positives but also matches some negatives due to mass preservation, leading to high transport costs. Increasing w to 2 allows partial matching of only concealed posi- tive samples, rectifying the transport plan and minimizing transport cost. Further increases w, as shown in Fig- ure (d), do not rectify the transport plan and instead increase transport costs.
Varying error measure results. According to Theorem 4, the surrogate risk function (5) with logistic error mea- sure is convex, making it amenable optimization via stochastic gradient methods. This property also extends to the squared loss error measure, which explains its com- parable performance relative to the logistic measure in Figure 3. On the other hand, the sigmoid error measure renders the surrogate risk non-convex, which complicates optimization and causes suboptimal performance.
The behavior of w proves suitable for estimating class priors. Figure 4 (d) shows that for a true class prior Kip 0.3, transport cost decreases sharply towards about 0.2 at w = 0.4. then gradually increases. According to Section 4.4, PPT estimates the class prior to be between 0.25 and 0.33, closely matching the predefined Kp = 0.3. For other class priors of 0.05, 0.1, and 0.5, the minimum
7

---

Unbiased Recommender Learning from Implicit Feedback via Weakly Supervised Learning
0.85
0.55
NDCG@K
0.50
Recallik
0.30
1
NDCG@KK
0.60
0.45
0.45
0.25
30
0.55
0
0.35
8
0.20
0.50
O
0.40
0.25
0.15
0.45
0.20
0.35
0.10
0.40
0:15
0.05
5 10 20 30 50 60 70 80 90100
5 20 50 80 90100
10
30 40 50 60 80 90100
5 10 20 30 40 60 80 90100 Involved unlabeled data (%)
Involved unlabeled data (%)
Involved unlabeled data (%)
Involved unlabeled data (%)
(a) Results with KuaiRec.
(b) Results with Coat.
Figure 5: Results with varying ratios of unlabeled data for model training. The colored lines and shadowed areas represent mean values and 90% confidence intervals, respectively.
transport costs correspond to w values of 20, 10, and 2 respectively, supporting the relationship Kp = 1/w.
samples. For instance, using just 5% of unlabeled sam- ples achieves an NDCG@ of approximately 0.34 on the KuaiRec dataset, with performance peaking at around 0.5 when all unlabeled samples are included. These trends, which hold across other metries and datasets, validate Weak- lyRec's theoretical consistency. Additionally, the rate of performance improvement tends to plateau beyond a cer- tain threshold, indicating that while it is possible to use all unlabeled data, a subset may be sufficient to balance performance and training cost.
4.4.2. IN-LOOP VERIFICATION
A reasonable hypothesis is that the class prior yielding the optimal performance likely corresponds to the ground truth, aligning well with both theoretical properties and practical requirements. Therefore, we firstly modify WeaklyRee to use fixed class prior estimates, R, and evaluate the resulting performance. Afterwards, we examine whether PPT's esti- mated class prior approximates the fixed prior that achieves the best performance. This analysis is conducted using the large-scale KuaiRec dataset, with key results summarized in Table and analyzed below.
5. Related Works
5.1. Learning from Weak Supervision
Weakly supervised learning refers to type of machine learn- ing paradígm where the training data is partially labeled or noisily labeled (Sugiyama et al., 2022). Unlike supervised learning wherein each data point has a corresponding accu- rate label, weak supervised learning operates on data where the labels may be imprecise, incomplete, or altogether miss- ing for some instances. This approach is particularly useful when obtaining fully labeled data is expensive difficult.
Accurate class prior estimation is critical for WeaklyRec's 's performance. In Table 3, setting Rp 0.2 results in NDCG@5 of approximately 0.423. Performance consis- tently improves as & decreases, reaching the peak of around 0.487 at &p = 0.05. Further increasing &p to 0.1 causes an immediate drop. Thus, the optimal class prior estimate is approximately Rp = 0.05.
(PU) learning an important instance of weakly supervised learning, focusing on problems where the training set consists of labeled positive examples and unlabeled examples that could either be positive or negative, with applications in fundamental fields such as computer vision (Loghmani et al., 2020; Chapel et al., 2020) and text processing (Le et al., 2020; S. et al., 2022). The core challenge arises from the absence of negative labels, which impedes the application of traditional supervised learning models. There are two primary paradigms to tackle this chal- lenge: re-weighting and pseudo-labeling. The re-weighting paradigm adjusts sample weights to achieve unbiased es- timation of the ideal risk, exemplified by uPU (du Plessis et al., 2014). Subsequent modifications make uPU con- vex (du Plessis et al., 2015b) and non-negative (Kiryo et al., 2017; Chaudhari and Shevade, 2012; Zhang and Zuo, 2009), to accommodate with stochastic gradient optimizers. Re- cently, further refinements have been proposed to adapt
PPT effectively identifies the ground-truth class prior. Specifically, the PT cost is minimized at Ap 0.05 at 5.5620, which corresponds to the best overall performance in Table 3. Therefore, the PPT approach successfully iden- tifies the class prior that optimizes performance.
4.5. Scalability Analysis
According to Theorem 3, involving more unlabeled sam- ples reduces the variance to approximate the ideal risk and improves performance. In this section, investigate the performance of WeaklyRec given varying proportions of unlabeled samples for training, to substantiate the utility of unlabeled samples.
The results in Figure 5 show a consistent improvement in performance with increasing proportions of unlabeled
8

---

Unbiased Recommender Learning from Implicit Feedback via Weakly Supervised Learning
handle data imbalance (Su al., 2021) and sample selection bias (Hammoudeh and Lowd, 2020). On the other hand, the pseudo-labeling paradigm identifies reliable negative and positive samples from the unlabeled data and then turns the problem into a supervised learning task. Methods differ in the techniques of pseudo labels assignment, such as graph neural network (Chaudhari and Shevade, 2012; Zhang and Zuo, 2009), generative models (Hou al., 2018). confidence score (Northcutt al., 2017), reinforcement learning (Luo et al., 2021) and clustering analysis (Gong et al., 2019).
the training process; and (iii) the non-negativity constraint, essential for model convergence, is often violated for the sake of theoretical unbiasedness (Saito et al., 2020; Saito, 2020). These limitations are not only theoretically complex but also empirically restrict the application of IPW-based approaches in ImplicitRec.
Recently, PU learning has also been incorporated for en- hancing ImplicitRec (Togashi et al., 2021; Zhou al., 2021). For example, Zhou al. (2021) develop adversarial ap- proach using PU loss for discriminator training, Togashi et al. (2021) introduce a munsity-ratio-basee method that leverages PU learning for density-ratio estimation. Our con- tributions are distinct from them: we are the first to integrate PU learning to enhance the binary classification objective for ImplicitRec and propose a class prior estimation method based optimal transport, significant contribution given the complexity of this issue in PU learning.
5.2. Recommendation with Implicit Feedback
ImplicitRec aims infer user preferences from user behav- iors such as clicks or views rather than explicit comments.
Compared with explicit feedback, implicit feedback is char- acterised by missing negative feedback (MNF). Specifically, the data involves some positive feedback and unlabeled data. The absence of negative feedback makes it difficult to dif- ferentiate between items that are disliked and those simply unexposed to the user. Current approaches treating unla- beled data as negative can lead to biased and inaccurate recommendations (Zhou et al., 2022; Kang and McAuley, 2018; Wang et al., 2022; Li et al., 2023c). An exemplar approach to handle this issue is the weighted matrix factor- ization (Hu et al., 2008), which heuristically downweights the unobserved user-item samples versus the observed ones. EXpoMF (Liang et al., 2016) further extends it to genera- tive formulation. However, these methods are often heuristic and lack rigorous theoretical underpinning, indicating need for further exploration of this line of works.
6. Conclusions
In conclusion, the challenge of missing negative samples has long been a significant obstacle in ImplicitRec. To handle this issue, we develop WeaklyRec, model-agnostic framework that adapts principles from weakly supervised learning to the domain of ImplicitRec. A cornerstone of our approach is the PPT approach designed to estimate the class prior by minimizing the proximal transport cost between positive and unlabeled samples. Extensive evaluations on real-world datasets confirm the efficacy of our approach and support the theoretical analysis empirically.
Recent advancement further considers sample selection bias in the positively labeled data, Specifically, the observed positive feedback is not random sample of user's true preferences; rather, is influenced by various factors such as item popularity and visibility (Lim et al., 2015; Yang et al., 2018). This non-random sampling can distort the true underlying preference structure, resulting in inaccurate recommendations (Zhou et al., 2021; Gupta et al., 2023; Togashi et al., 2021). Current efforts mainly employ in- verse propensity weighting (IPW) to correct such selection bias. It re-weights each observed sample using propensity score, which theoretically ensures an unbiased modeling of user-item preferences (Wang et al., 2022; Li et al., 2023c;a). In practice, methods mainly differ in terms of the ways to estimate propensity, represented by employing the item popularity (Saito et al., 2020; Saito, 2020) and constructing a propensity estimator (Zhu al., 2020; Lee et al., 2022: Xiao et al., 2024). Despite its theoretical merits, IPW faces several practical challenges: (i) the propensity estimation is often inaccurate (Li et al., 2023c), exacerbated by the ab- sence of negative feedback; (ii) the estimators are sensitive to small propensity scores (Li et al., 2023a), complicating
Limitation & Future works This study adheres to the set- ting of ID>based recommendation research and does not con- sider user profiles. Future work could extend this framework to more content-rich scenarios. Additionally, the scalabil- ity of WeaklyRee highlights the potential of using implicit feedback to enhance recommenders. Therefore, how to har- monize explicit and implicit feedback presents a promising direction for further research.
A Acknowledgements
This work is supported by National Key R&D Program of China (20222D0160300) and the NSF China (No. 62276004, 623B2002).
Impact Statement
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.

---

Unbiased Recommender Learning from Implicit Feedback via Weakly Supervised Learning
References
Hicham Janati, Alain Rakotomamonjy, levgen Redko, Antoine Rolet, Antony Schutz, Vivien Seguy, Danica J. Sutherland, Romain Tavenard, Alexander Tong, and Titouan Vayer. 2021. POT: Python Optimal Transport. J. Mach. Learn. Res. 22 (2021), 78:1-78:88
Jason M. Altschuler, Jonathan Weed, and Philippe Rigol- let. 2017. Near-linear time approximation algorithms for optimal transport via Sinkhorn iteration. In NeurlPS. 1974.
Chongming Gao, Shijun Li, Wenqiang Lei, Jiawei Chen, Biao Li, Peng Jiang, Xiangnan He, Jiaxin Mao, and Tat- Seng Chua. 2022a. KuaiRec: A Fully-observed Dataset and Insights for Evaluating Recommender Systems. In CIKM. ACM, 540-550.
Jean-David Benamou, Guillaume Carlier, Marco Cuturi, Luca Nenna, and Gabriel Peyré. 2015. Iterative Bregman projections for regularized transportation problems. SIAM J. SCI. COMPUT. 37, 2 (2015), A1111-A1138.
Nicolas Bonneel, Michiel van de Panne, Sylvain Paris, and Wolfgang Heidrich. 2011. Displacement interpolation using Lagrangian mass transport. ACM Trans. Graph. 30, 6 (2011), 158.
Chongming Gao, Shijun Li, Yuan Zhang, Jiawei Chen, Biao Li, Wenqiang Lei, Peng Jiang, and Xiangnan He. 2022b. KuaiRand: An Unbiased Sequential Recommendation Dataset with Randomly Exposed Videos. In CIKM. ACM, 3953-3957.
Laetitia Chapel, Mokhtar 7- Alaya, and Gilles Gasso. 2020. Partial Optimal Tranport with applications on Positive- Unlabeled Learning. In NeurIPS.
Carlos A. Gomez-Uribe and Neil Hunt. 2016. The Netflix Recommender System: Algorithms, Business Value, and Innovation. ACM Trans. Manage. Inf. Syst. 6, 4 (2016), 19 pages.
Sneha Chaudhari and Shirish K. Shevade. 2012. Learning from Positive and Unlabelled Examples Using Maximum Margin Clustering. In NeurlPS, Vol. 7665. 465-473.
Chen Gong, Hong Shi, Tongliang Liu, Chuang Zhang, Jian Yang, and Dacheng Tao. 2019. Loss decomposition and centroid estimation for positive and unlabeled learning. IEEE Trans. Pattern Anal. Mach. Intell 43, 3 (2019), 918- 932.
Nicolas Courty, Rémi Flamary, Devis Tuia, and Alain Rako- tomamonjy. 2017, Optimal Transport for Domain Adap- tation. IEEE Trans. Pattern Anal. Mach. Intell. 39, 9 (2017), 1853-1865.
Shashank Gupta, Harrie Oosterhuis, and Maarten de Rijke. 2023. A Deep Generative Recommendation Method for Unbiased Learning from Implicit Feedback. In ICTIR. ACM, 87-93.
Marthinus Christoffel du Plessis, Gang Niu, and Masashi Sugiyama. 2014. Analysis of Learning from Positive and Unlabeled Data. In NeurlPS. 703-711.
Marthinus Christoffel du Plessis, Gang Niu, and Masashi Sugiyama. 2015a. Class-prior Estimation for Learning from Positive and Unlabeled Data. In ACML, Vol. 45. 221-236.
LLC Gurobi Optimization. 2020. Gurobi optimizer refer- ence manual (2020).
Zayd Hammoudeh and Daniel Lowd. 2020. Learning from Positive and Unlabeled Data with Arbitrary Positive Shift. In NewrIPS.
Marthinus Christoffel du Plessis, Gang Niu, and Masashi Sugiyama. 2015b. Convex Formulation for Learning from Positive and Unlabeled Data. In ICML, Vol. 37. 1386-1394.
William E Hart, Jean-Paul Watson, and David L Woodruff. 2011. Pyomo: modeling and solving mathematical pro- grams in Python. Mathematical Programming Computa- tion 3 (2011), 219-260.
Marthinus Christoffel du Plessis, Gang Niu, and Masashi Sugiyama. 2017. Class-prior estimation positive and unlabeled data. Machine Learning 106, 4 (2017), 463-492.
estimatiion for learning from
Zhuangzhuang He, Yifan Wang. Yonghui Yang, Peijie Sun, Le Wu, Haoyue Bai, Jinqi Gong, Richang Hong, and Min Zhang. 2024. Double Correction Framework for Denois- ing Recommendation. In KDD. ACM, 1062-1072
Pavel E. Dvurechensky, Alexander V. Gasnikov, and Alexey Kroshnin. 2018 Computational Optimal Transport: Com- plexity by Accelerated Gradient Descent Is Better Than by Sinkhorn's Algorithm. In ICML, Vol. 80. 1366-1375.
Ming Hou, Brahim Chaib-Draa, Chao Li, and Qibin Zhao. 2018. Generative adversarial positive-umlabeled learning. In IJCAI. 2255-2261.
Rémi Flamary, Nicolas Courty, Alexandre Gramfort, Mokhtar Z. Alaya, Aurélie Boisbunon, Stanislas Cham- bon, Laetitia Chapel, Adrien Corenflos, Kilian Fatras, Nemo Fournier, Léo Gautheron, Nathalie H. Gayraud,
Yifan Hu, Yehuda Koren, and Chris Volinsky. 2008. Col- laborative Filtering for Implicit Feedback Datasets. In ICDM. IEEE Computer Society.
10

---

Unbiased Recommender Learning from Implicit Feedback via Weakly Supervised Learning
Wang-Cheng Kang and Julian J. McAuley. 2018. Self- Attentive Sequential Recommendation. In ICDM. 197- 206.
Chuan Luo, Pu Zhao, Chen Chen, Bo Qiao, Chao Du, Hongyu Zhang, Wei Wu, Shaowei Cai, Bing He, Sar- avanakumar Rajmohan, and Qingwei Lin. 2021. PULNS: Desitive-Unlabeled Learning with Effective Negative Sample Selector. In AAAI 8784-8792.
Leonid V Kantorovich. 2006. On the translocation of masses. J. Math. Sci. 133, 4 (2006), 1381-1382.
Simone Di Marino and Augusto Gerolin. 2020. An Optimal Transport Approach for the Schrödinger Bridge Problem and Convergence of Sinkhorn Algorithm. J. Sci. Comput. 85, 2 (2020). 27.
Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In ICLR.
Ryuichi Kiryo, Gang Niu, Marthinus Christoffel du Plessis, and Masashi Sugiyama. 2017. Positive-nnlabeled Learn- ing with Non-Negative Risk Estimator. In NeurlPS. 1675- 1685.
Gaspard Monge. 1781. Mémoire sur la théoric des déblais et des remblais. Mem. Math. Phys. Acad. Royale Sci. (1781), 666-704.
Triet Huynh Minh Le. David Hin, Roland Croft, and Muhammad Ali Babar. 2020. PUMiner: Mining Security Posts from Developer Question and Answer Websites with PU Learning. In MSR. ACM, 350-361.
Gang Niu, Marthinus Christoffel Du Plessis, Tomoya Sakai, Yao Ma, and Masashi Sugiyama. 2016. Theoretical com- parisons of positive-unlabeldd learning against positive- negative learning. NeurlPS 29 (2016).
Jae-woong Lee, Seongmin Park, Joonseok Lee, and Jong- wuk Lee. 2022. Bilateral Self-unbiased Learning from Biased Implicit Feedback. In SIGIR. ACM, 29-39.
Curtis G. Northcutt, Tailin Wu, and Isaac L.. Chuang. 2017. Learning with Confident Examples: Rank Pruning for Ro- bust Classification with Noisy Labels. In UAI. 10 pages.
Haoxuan Li, Yan Lyu, Chumyuan Zheng, and Peng Wu. 2023a. TDR-CL: Targeted Doubly Robust Collaborative Learning for Debiased Recommendations. In ICLR.
Ofir Pele and Michael Werman. 2009. Fast and robust Earth Mover's Distances. In ICCV. 460-467.
Haoxuan Li, Yanghao Xiao, Chunyuan Zheng, Peng Wu, and Peng Cui. 2023b. Propensity Matters: Measuring and Enhancing Balancing for Recommendation. In ICMI., Vol. 202. PMLR, 201 2019944.
Ren, Hongyan Tang, Jiangpeng Rong, and Siwen Zhu. 2023, Unbiased Pairwise Learning from Implicit Feed- back for Recommender Systems without Biased Variance Control. In SIGIR. ACM, 2461-2465.
Haoxuan Li, Chunyuan Zheng. Shuyi Wang, Kunhan Wu, Hao Wang, Peng Wu, Zhi Geng, Xu Chen, and Xiao-Hua Zhou. 2024. Relaxing the Accurate Imputation Assump- tion in Doubly Robust Learning for Debiased Collabora- tive Filtering. In ICML.
Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian Person- alized Ranking from Implicit Feedback. In UAI. AUAI Press, 452-461.
Haoxuan Li, Chunyuan Zheng, and Peng Wu. 2023c. Sta- bleDR: Stabilized Doubly Robust Learning for Recom- mendation on Data Missing Not at Random. In ICLR.
Vinay M. S., Shuhan Yuan, and Xintao Wu. 2022. Fraud Detection via Contrastive Positive Unlabeled Learning. In IEFE Big Data. IEEE, 1475-1484.
Dawen Liang, Laurent Charlin, James McInerney, and David M. Blei. 2016. Modeling User Exposure in Rec- ommendation. In WWW. ACM, 951-961.
Yuta Saito. 2020. Unbiased Pairwise Learning from Biased Implicit Feedback. In ICTIR. ACM,
Yuta Saito, Suguru Yaginuma, Yuta Nishino, Hayato Sakata, and Kazuhide Nakata. 2020. Unbiased Recommender learning from Wissing-Not-At-Be back. In WSDM ACM, 501-509.
Daryl Lim, Julian J. McAuley, and Gert R. G. Lanckriet. 2015. Top-N Recommendation with Missing Implicit Feedback. In RecSys. ACM, 309-312.
Implicit Feed-
Haibin Ling and Kazunori Okada. 2007. An Efficient Earth Mover's Distance Algorithm for Robust Histogram Com- parison. IEEE Trans. Pattern Anal. Mach. Intell. 29, 5 (2007), 840-853.
Jian Shen, Yanru Qu, Weinan Zhang, and Yong Yu. 2018. Wasserstein Distance Guided Representation Learning for Domain Adaptation. In AAAI. 4058-4065.
Mohammad Reza Loghmani, Markus Vineze, and Tatiana Tommasi. 2020. Positive-unlabed learning for open set domain adaptation. Pattern Recognit. Lett. 136 (2020), 198-204.
Ilya Shenbin, Anton Alekseev, Elena Tutubalina, Valentin Malykh, and Sergey I. Nikolenko. 2020. RecVAE: A New Variational Autoencoder for Top-N Recommendations with Implicit Feedback. In WSDM. ACM, 528-536.

---

Unbiased Recommender Learning from Implicit Feedback via Weakly Supervised Learning
Brent Smith and Greg Linden. 2017. Two Decades of Rec- ommender Systems Amazon.com. IEEE Internet Com- puting 21, 3 (2017), 12-18.
Jingjing Xu, Hao Zhou, Chun Gan, Zaixiang Zheng, and Lei Li. 2021. Vocabulary Learning via Optimal Transport for Neural Machine Translation. In ACL/JCCEPP (I). ACL, 7361-7373.
Zijie Song, Jiawei Chen, Sheng Zhou, Qihao Shi, Yan Feng,
Chun Chen, and Can Wang. 2023. CDR: Conservative doubly robust learning for debiased recommendation. In CIKM. 2321-2330
Renjun Xu, Pelen Liu, Yin Zhang, Fang Cai, Jindong Wang. Shuoying Liang, Heting Ying, and Jianwei Yin. 2020. Joint Partial Optimal Transport for Open Set Domain Adaptation. In I/CAI. 2540-2546
Guangxin Su, Weitong Chen, and Miao Xu. 2021. Positive- Unlabeled Learning from Imbalanced Data. In IJCAI. 2995-3001.
Longgi Yang, Yin Cui, Yuan Xuan, Chenyang Wang, Serge J. Belongie, and Deborah Estrin. 2018. Unbiased offiine rec- ommender evaluation for missing-not-aa tott-----tt-t--ddooo implicit feedback. In RecSys. ACM, 279-287.
Masashi Sugiyama, Han Bao, Takashi Ishida, Nan Lu, and Tomoya Sakai. 2022. Machine learning from weak super- vision: An empirical risk minimization approach. MIT Press.
Bangzuo Zhang and Wanli Zuo. 2009. Reliable Negative Extracting Based on kNN for Learning from Positive and Unlabeled Examples. Journal of Computers 4. (2009), 94-101.
Riku Togashi, Masahiro Kato, Mayu Otani, and Shin iccii Satoh. 2021. Density-Ratio Based Personalised Ranking from Implicit Feedback. In WWW. ACM, 3221-3233.
Shuqiang Zhang, Yuchao Zhang, Jinkun Chen, and Haochen Sui. 2025, Addressing Correlated Latent Exogenous Vari- ables in Debiased Recommender Systems. In SIGKDD.
Hao Wang, Tei-Wei Chang, Tianqiao Liu, Jianmin Huang, Zhichao Chen, Chao Yu, Ruopeng Li, and Wei Chu. 2022. ESCM2: Entire Space Counterfactual Multi-Tass Model for Post-Click Conversion Rate Estimation. In SIGIR. 363-372.
Yunrui Zhao, Qianqian Xu, Yangbangyan Jiang, Peisong Wen, and Qingming Huang. 2022. Dist-PU: Positive- Unlabeled Learning from a Label Distribution Perspec- tive. In CVPR. IEEE, 14441-14450
Hao Wang. Zhichao Chen, Zhaoran Liu, Xu Chen, Haoxuan Li, and Zhouchen Lin. 2025a. Proximity Matters: Lo- cal Proximity Enhanced Balancing for Treatment Effect Estimation. SIGKDD.
Chunyuan Zheng, Hang Pan, Yang Zhang, and Haoxuan Li. 2025. Adaptive Structure Learning with Partial Parameter Sharing for Post-Click Conversion Rate Prediction. In SIGIR.
Hao Wang, Zhichao Chen, Honglei Zhang, Zhengnan Li, Licheng Pan, Haoxuan Li, and Mingming Gong. 2025b. Debiased Recommendation via Wasserstein Causal Bal- ancing. ACM Trans. Inf. Syst. (2025).
Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018. Deep interest network for click-through rate prediction. In SIGKDD. 1059-1068.
Hao Wang, Jiajun Fan, Zhichao Chen, Haoxuan Li, Weim- ing Liu, Tianqiao Liu, Quanyu Dai, Yichao Wang, Zhen- hua Dong, and Ruiming Tang. 2023. Optimal transport for treatment effect estimation. NewrIPS 36 (2023), 5404 5418.
Kun Zhou, Hui Yu, Wayne Xin Zhao, and Ji-Rong Wen. 2022. Filter-enhanced MLP is All You Need for Sequen- tial Recommendation. In WWW. 2388-2399.
Yao Zhou, Jianpeng Xu, Jun Wu, Zeinab Taghavi Nasrabadi, Evren Körpeoglu, Kannan Achan, and Jingrui He. 2021. PURE: Unllbele-Ulllbeled Recommendation with Gen- erative Adversarial Network. In SIGKDD, ACM, 2409- 2419.
Wenjie Wang, Fuli Feng, Xiangnan He, Liqiang Nie, and Tat-Seng Chua. 2021. Denoising implicit feedback for recommendation. In WSDM. 373-381.
Yanghao Xiao, Haoxuan Li, Yongqiang Tang, and Wen- sheng Zhang. 2024. Addressing Hidden Confounding with Heterogeneous Observational Datasets for Recom- mendation. In NeurIPS.
Ziwei Zhu, Yun He, Yin Zhang, and James Caverlee. 2020. Unbiased Implicit Recommendation and Propensity Es- timation via Combinational Joint Learning. In RecSys. ACM, 551-556.
Hongteng Xu, Dixin Luo, Hongyuan Zha, and Lawrence Carin. 2019. Matching and Node Embedding. In ICML, Vol. 97. PMLR, 6932-6941.
Learning for Graph
12

---

Unbiased Recommender Learning from Implicit Feedback via Weakly Supervised Learning
10"
10
Time (s)
10"2
Time
10"
-2
W
Rl
5
64
256
2
20
128
10
128
256
512
1024
2048
128
256
512
1024
2048
Ba
Figure 6: Running time of solving the PT problem. By default we set np = 128 and w 10.
A. Additional Experimental Results
A.1. Dataset Description
In this section, we present a brief introduction to the datasets involved this study.
Yahoo! R3 contains 31 111,704 biased ratings for training, which involve 15, 400 users and 1, 000 items. Additionally, 5400 users rate 10 randomly selected items, yielding 54, 000 unbiased ratings for evaluation. The user-item pairs with rating greater than 4 are seen as positive, and others are viewed as negative.
Coat a public dataset which consists of 290 users and 300 items; each user subjectively selects 24 items to rate based on their preference, yielding 6,960 biased ratings the training set. Additionally, each user rates 16 items that are randomly selected, yielding 4,640 unbiased ratings for model evaluation. The user-item pairs with rating greater than 4 are seen as positive, and others are viewed as negative.
KuaiRec is a public large-scale industrial dataset, which consists of 4,676,570 video watching ratio records from 1,411 users for 3,327 videos. The user-item pairs with ratings less than two are viewed as negative, and otherwise are viewed as positive. The records less than are viewed as negative feedback, and otherwise are viewed negative feedback.
A.2. Discussion on Complexity
Concerns may arise regarding the computational overhead introduced by WeaklyRec, given that it incorporates línear programming procedure- solving of the PT problem-in each iteration model training. These concerns are particularly pertinent in the context of large batch sizes commonly employed in training recommender models, as this could render the PT problem large-scale and computationally intensive. To address these concerns, we empirically evaluate the time required for solving the PT problem under various settings, as depicted in Figure 6. We summarize two key observations as follows.
The computational complexity of solving the PT problem is not prohibitive in our specific application. For instance, even when matching as many 4096 unlabeled samples, the problem can be solved within 0.1 seconds. This relatively low computational burden is primarily due to the scarcity of positively labeled samples. In contrast to standard optimal transport (OT) scenarios where two distributions have a similar number of samples, the PT problem in our case involves matching n, unlabeled samples with a significantly smaller number of np positive samples. Given that ny << Pap the computational cost substantially lower than in traditional OT applications.
The computational complexity is also influenced by the value of the mass weight w. Specifically, as w decreases, more unlabeled samples are compelled to engage in matching due to the mass-preserving constraint, thereby increasing the computational cost. When w = 1, the PT problem reduces to the canonical Kantorovich problem. However, given that the class prior is generally small in ImplicitRec applications, candidate values for w can be set to relatively large values, which further reduces the complexity of solving PT in WeaklyRec.
A.3. Hyperparameter Sensitivity
We perform a comprehensive sensitivity analysis to examine the influence of four critical hyperparameters, namely batch size, learning rate, and embedding dimension, on the performance of the WeaklyRec model using the coat and Kuairec datasets. The results of these experiments are illustrated Figure and 7, respectively.
13

---

Unbiased Recommender Learning from Implicit Feedback via Weakly Supervised Learning
0:55
0.65
0.60
20 52
0.55
0.59
0.45
612 1004 2048
1994
0035 00
016
0.565
0.51 Learingrae
00G
Batch siz
Butco stas
Learning nia
Erbodsing sice
(a) Varying batch size results.
(b) Varying learning rate results.
(c) Varying embedding dimension results.
Figure 7: Parameter sensitivity studies of the WeaklyRec framework on Coat. Different colors indicate ranking metrics
given different numbers of top candidates (K).
ND00gK
0.50
0.35
NO00BR
0.50
0.35
0.00
0440
0.39
02
0.20
0.00
6.29
0:49
0:15
0:15
8:10
844
6.15
0/10
290
512
1004
2045
<086
512
1924
2000
4000
0.001
0.05
60
3:00
0.001
0.005
0.51
0.5
0
32
Balth size
Butch
sAze
Learning 19lo
Loaniig 10000
Briboddingsss 225
E Eribodding size
(a) Varying batch size results.
(b) Varying learning rate results.
(c) Varying embedding dimension results.
Figure 8: Parameter sensitivity studies of the WeaklyRee framework on Kuairec. Different colors indicate ranking metrics given different numbers of top candidates (K).
Initially, we examine the effects of varying batch sizes on two distinct datasets. Interestingly, as the batch size increases, metrics such as NDCG@K and Recall@K K generally show a decreasing trend. However, the KuaiRec dataset exhibits an initial increase followed by a decrease in these performance measures. This fluctuation can potentially be attributed to the interplay between the inherent characteristics of the PPT algorithm, which is integral to our model architecture, and the properties of the datasets in question, as well as their batch sizes. Specifically, smaller batch sizes may lead the PPT algorithm to experience generalization issues, adversely impacting the model's overall performance. Conversely, larger batch sizes could introduce an over-smoothing problem, thereby reducing performance. These observations are further corroborated by data from the Coat and KuaiRec datasets in the batch size intervals of [256, 4096] and [2048, 4096) respectively. As for the interval [256, 2048] in the KuaiRec dataset, this behavior can be explained by the dataset's size and inherent properties. Given the large size of the KuaiRec dataset, which primarily focuses on video clips, there is likely a significant level of noise. Furthermore, the dataset's 's user base, representing a variety of socio-economic strata, tends to rapidly dismiss recommended items. Consequently, smaller batch sizes may make the model more susceptible noise.
Subsequently, we tuned the learning rate and observe divergent trends in performance across the Coat and KuaiRec datasets. Specifically, an increase in the learning rate leads to an improvement in performance metrics on the Coat dataset, while causing a decline on the KuaiRec dataset. This differential behavior can largely be attributed to the contrasting sizes of the two datasets. The KuaiRec dataset is considerably larger than the Coat dataset, and this disparity likely influences the performance outcomes. In larger datasets like KuaiRec, the presence of noise becomes more prevalent, which may adversely affect model performance when higher learning rates are employed.
Additionally, investigate the influence of varying the embedding dimension, which ranges from 2 to 64. We observe that the model performance initially improves, only to subsequently decline. This fluctuating pattern can be linked to the inherent properties of the PPT algorithm integrated into our model. PPT computations are executed in the embedding space, and an overly restricted dimensionality may lack the capacity to accurately capture the data's complex manifold, leading to a degradation in model performance. On the other hand, an excessively large embedding dimension risks introducing redundancy and noise, thereby undermining the effectiveness of the optimal transport problem solution, which in turn may not reflect the true data manifold. For the KuaiRec dataset, a monotonically decreasing performance trend is evident. This observation lends further credence to our assertion that the KuaiRec dataset likely contains significant noise. Such noise interferes with the accurate representation of the true data manifold within the PPT algorithm, thereby adversely affecting the model performance.
In a broad range of hyperparameter settings, the WeaklyRee model consistently outperforms a majority of baseline models, further substantiating its superior performance characteristics. Specifically, even under varying conditions such as different learning rates, batch sizes, and embedding dimensions, the WeaklyRec model still excels or at least matches other commonly-
14

---

Unbiased Recommender Learning from Implicit Feedback via Weakly Supervised Learning
Table 4: Recommendation performances in terms of NDCG@K on Yahoo! R3, Coat and KuaiRec.
Dataset
Yahoo! R3
Coat
KuaiRee
Metrics
NDGC@1
NDGC@3
NDGC@5
NDGC@1 1
NDGC@3
NDGC@5
NDGC@
NDGC@3
@DGC@5
RMF
0.66720.00s 0.632
20.022
0.370
0.394
0.433
0.209
0.211
0.2154400005
CRMF
0.603
0.667=0.002
0.706g0.003
20.019
0.400
0.439
0221120.00
000
0.217
0.222=:0.005
WMF
0.67920.030
0.710 20002
20.05
0,402220.22z 025
0,465 0.427
0.24620.049
0.256
0.270
UBPR
0..57
0.795
0.826+
0,459 +
0.493
0.530#0.014
0.255=0.011
0.272+0.011
0.290
CUBPR
0.795 0.756
ouz
0.826+
0,460 oau
40.020
0.53
0.257 + 007
0.273
+0.09
0.292
UPL
40.00
0.798+4.uu02
0.829 + 0002
0.508 155088+..0z +0,028
0.511 +0.012
0.540
0.281 +0.034
0.297
+0.044
0.315
ORACLE
0.693 0.654
003
0.734 17.4400000 003
0,425 =0 026
0,496 10.19
017
0.212 011 0.231 +0.008 000
0.251 ::0 008
WeaklyRec
0.787
0.817=20000
0.845 10.000
0.579 035
0.567
10:019
0.597
0.489 20.013 0.480
0,483 007
Note: The results are reported in meangetd with runs. The best and second best metrics are bolded and underlined, respectively. "g*
marks the metrics that WeaklyRec surpasses the best baseline with p-value < 0.05 over paired samples t-test.
used recommendation algorithms multiple evaluation metrics. This further confirms the robustness and versatility of the WeaklyRec model in practical recommendation system applications. Moreover, by conducting more fine-grained hyperparameter tuning, the WeaklyRec model has the potential to achieve even better performance than reported in the main text. This implies that, after more precise hyperparameter search and validation, the WeaklyRec model can only maintain its advantage over baseline models but also possibly reach higher performance levels in specific application scenarios or datasets. This further highlights the superior and adaptive nature of the WeaklyRec model, making it worth further exploration in real-world applications.
A.4. Supplementary Numerical Examples
To validate the efficacy of PPT in class prior estimation, we include case study using numerically simulated dataset with predefined (and known) class priors. In the main text, we only present the matching strategy on the simulated case wherein Kp = 0.5. Nevertheless, it is essential to consider other simulation conditions to back up the utility of the proposed approach.
To achieve this, we extend our numerical verification by considering a range of class priors Kp 0.05, 0.1, 0.3, 0.5. We visualize the resulting transport strategies for different mass weights w. The key insights consistent with those presented in the main text. The transport strategies are optimized when w 1/Kpp at which point the transport cost reaches its minimum, as illustrated in Figure 4. These additional numerical scenarios further corroborate the effectiveness of the Progressive Proximal Transport (PPT) method in accurately estimating the class prior, thereby reinforcing the validity of our WeaklyRec framework.
A.5. Performance with different backbone
In the main text, we implement the recommendation model as MF, since it is a widely used model to evaluate the efficacy of learning objectives in recommendation system (Li et al., 2023c;a), especially in recent ImplicitRec researches (Ren et al., 2023; Lee et al., 2022; Zhu et al., 2020; Saito et al., 2020). Nevertheless, we agree that it interesting investigate the performance of different ImplicitRec estimators in different backbones to test their generality. Therefore, we select NCF as our backbone and summarize the experimental results in Table and Table 5, which showease trends similar the results on MF.
B. Theoretical Justification
B.I. Formal Analysis on the Surrogate Risk
Theorem 1. Given that the ideal risk of the recommender g defined over all samples is Rideal(), the surrogate risk Rweate(g) as defined in (6) is an unbiased estimator of Rideal(g):
Ex
(Rocate (9) = Rident (g)-
Proof. Let X' be the empirical distribution of samples for training, with positive samples X, and unlabeled samples X,
15

---

Unbiased Recommender Learning from Implicit Feedback via Weakly Supervised Learning
2
2
esog
8 o dade
Mode Borke pue
des ae e cole
ah do oog be dee e
be d sat be g
ao e od
apptea g
1
2
1
2
(a) W = Kp = 0.05
(b) 10, = 0.05
(c) W = 20, Kp 0.05
(d) 30, = 0.05
2
2
00000 o aae aa
2
0
2 00000
0000 e & oo me m
war co oz & ao oe pn
us to n
ooo go A o to onn ms
2
0
(c) Rp 0.1
(f) 5, Kp 0.1
(g) 10, Ap 0.1
(h) 20, Kp 0.1
2
Lg 0 e
e dsto sa gon
w ea e eete
1-
eg goou
a
(i) w 1, Kp 0.3
(i) W 3, Kpp = 0.3
(k) 4, Kp = 0.3
(l) w = 10, Kp 0.3
2
2
% 2
2
m go P
2
00 so
0 ag
Bet Maa
so mar
2
1
2
(m) w = 1, = 0.5
(n) N 1.5, 0.5
(0) 2, Kp = 0.5
(p) w 3, 0.5
Figure 9: Overview of the PT transport plan given different weights of mass w and ground truth positive priors Kp. The ground-truth positive and negative feedbacks are denoted by circular and rectangular markers, respectively. The labeled and unlabeled intersections are indicated in red and blue, respectively. The best transport strategies, which minimize transport costs for different values Kp, are illustrated in subfigures (c. g, k, o), characterized by 1/w R3 Kpp
16

---

Unbiased Recommender Learning from Implicit Feedback via Weakly Supervised Learning
Table 5: Recommendation performances in terms of Recall @k on Yahoo! R3, Coat and KuaiRec.
Dataset
Yahoo! R3
Coat
KuaiRee
Metrics
@ecall
@
Recall
@
@ecall@
Recall
Recall
@e
RMF
0.04220.002
0.133 0004
0.226
0.054
0.270 0..582
0.009
0.025
0.042::.006
CRMF
0.04220.002
0.132=20.00 0003
0.226g0.004
0.167 0.056
10.016
0.277=0.022
0,00920.00 002
0.030+0.004
0.049
WMF
0.074
0.175
0.260
0.066_g..03
0.192
0.306
0.023=0.02
0.066
0.108440.044
UBPR
0.126+0.0u2
0.265
0.344
0.100
0.373 4..04
0.026
0.078+0.o0.
0.128
CUBPR
0.265 0.026
ouz
0.343 000
0.100
0.374 4..54400.020
0.026 003
0.078
0.130+
UPL
0.129 0.129 +0.004
0.267
0.345 + 0022
0.117 +0.012
0.261 to.011
0.367
40.00
0.033 +0.012
0.096
+0.22
0.153++a.u39
ORACLE
0.163 0.006
003
0.260 0.3
0.078:0.00
0.221 10.019
0,341 =0.021 021
0,014 004
0.054 006
0.1022::000 00s
WeaklyRec
0.148
0.282 00.004 0.356 0.00
0.149 023
0.30810.015
0.423.2.016
0.13320.000
0.245.0.00 0.296 003
Note: The results are reported in meangetd with runs. The best and second best metrics are bolded and underlined, respectively. "ga
marks the metrics that WeaklyRec surpasses the best baseline with p-value < 0.05 over paired samples t-test.
According the formulation of Weakly Rec in (6), the expectations regarding different training dataset can be formulated as follow:
Pp
Ex [weekk (0)
Exx
Kip
((g(2!))
np
i=1
(8)
((g(x)), -1)
+ Ex,
nu
For the first term, we have
np
np
Exp
Kp
{({(T\))
Kip
Ex, [(((())))
np
n2p
i=1
Kip
Σ Enp ((((xT)))
npp 1=1
(9)
Kp
Epp [((g(t))
1pp
i=1
[(((())))
Kip
For the second term, similarly we have
nu
Ex.
[g((x)) -1)
=Ep, [[(g(x),-1) - -1)]
(10)
Combining (8)-(10), we have
Ex (mwetk(0)
K6p Ep, [((((=)) + Ep, [e()()), -1)]
=kip Ep, (eg(x); +1)]
(11)
+ Kp Ep. [e (g(z), -1)]
(g).
Lemma 1. Let & be a family of recommender models. For any 0, with probability at least Niu et al. (2016)
demonstrate:
suPgeg (eweakl)) - R(g)) S 4kp,, R. + R. 2kpp
In(4/A)
In(4/8)
2np
2n.
17

---

Unbiased Recommender Learning from Implicit Feedback via Weakly Supervised Learning
where Rp and Ry are the Rademacher complexities of 9 given np positive samples and n. unlabeled samples, respectively. Le is the Lipschitz constant of with respect 9-
Theorem (Variance reduction). The variance of Bweak is smaller than that of the ideal loss Rideal given Kip S 0.5.
Proof. The variance of the ideal loss can be expressed as
Var (ardeeiea)(9)) =
P
Var( (ar(((((""),+1))) +
(
1lp
2.2
2
"P ppp
Pp
Ppp
where of Var(([g(=)),+1)), of <Var(((g<>"), -1)). The variance of Rweak (g) is:
K (ar(((o()))) +
(1 Var(((((=")--1
Var Mar(Rweak(9)) (g)) =
Pp
nu
1 Var((((()), +1)) + Var((((g(z) -1)))
z
Up
Var(e(g(x*)) -11)
Bu
x2 D
1
(ap
+
2)
nu
"p
Thu
where the covariance between the two terms is zero, which immediately holds if positive and unlabeled samples are independently drawn. To show that Var (gwe & Var (ar(ea)a))g)) it equivalent to show that
x² + o
(o"
Pp
2
op
2
n.
np
7u
7p
n2n
That is:
Kp
Ki
Ppp
Since nu >> np, Da. 1/nu -> 0. Moreover, since np/no nu = Kp/ Kus the above inequality holds
Rp Ku:
which holds if xp < 0.5, condition naturally holds in most recommendation scenarios where the positive samples sparse and much smaller than positive samples.
Theorem 3. Let be a family of recommender models; g" arg mingeg (ideatg) be the ground-truth optimal rec- ommender acquired by minimizing the ideal risk (1), g! = arg mingeg Riveak (g) be the recommender acquired by minimizing (6). For any 8 O, with probability least 5, we have
8 8ln(4/))
2 2l
R ( - R(')) < SKpLi + RL + Koy
+
nhp
where Rp and Ri are the Rademacher complexities of & given up positive samples and nu unlabeled samples, respectively. Te the Lipschitz constant of with respect g-
18

---

Unbiased Recommender Learning from Implicit Feedback via Weakly Supervised Learning
Proof. Based Lemma
have
R(g)) R((g)) (Rwoak(g') Reweak(e"))
R(gg)) Roweak(')) (9")) + (Rweak(g")) -R('))
R(g²))
(R(g") Reneate Rwnak((')) + + (Rreaaklo") R(g"))
v 0 + 2 2sup | (g) - RR())
s Lp 1h,9, + +Kpp k 81m(4/8)
2ln(4/6)
7k
Lemma 2. For any E(g. v) that convex D. (().9) v) E(D. -y) convex in 9 if and only E(ú. y) ((D--)) is linear y.
Proof. The lemma comes from the Theorem 4.1 by Sugiyama et al. (2022). We provide a straight-forward proof here. Since E(g. y) is convex, have:
f(y1 + 92- y) $ (()1. y) ((2. v)
To make V( E(ů. v) convex, the term -(( -)) should be convex. Therefore, have:
(iyi + 122 v) ((i1,)) v) E(02. v)
Combining the two inequalities, we conclude that to make E(g. y) { y) convex, the sufficient and necessary condition is: Elýi + V2, v) ((jn,)) y) ((j2,)), which indicates that ((j, v) {(ú, y) linear in y.
Theorem 4. Consider a reconmender g where the output score is transformed by a sigmoid function such that: g(x) sigmoid(s(z)).. Consider the logistic error measure, the surrogate risk Rweek (g) convex with respect the score s.
Proof. The surrogate risk Rweak (g) consists of two terms: E(g(x)) and ((g()), +1). Under the logistic error measure, we have:
{(g(x), +1) = -I(((x)))
{(g(x), -1) = -Inn1 ((x)).
Since g(x) sigmoid(ss). the error measure can be rewritten as:
{(8,y) = In( 1 w-v
First, we examine the convexity of the term {(s, 1).
The second derivative is:
d (((ss+1))
ex
>
(12)
ds2
(e 1)2
which means that E(s, +1) is convex with respect to
Next, we analyze the term ((s), which can be expressed as:
1
[(s,+1) - -(s,-1)) -1)
In
1 + In
1t 4
e + 8
(13)
In i+e-s 8.
In(
Since {(s) is linear with respect to s, it is also convex according to Lemma 2. Finally, as the sum of two convex functions remains convex, Rweak(g) (g) convex with respect to 8.
19

---

Unbiased Recommender Learning from Implicit Feedback via Weakly Supervised Learning
Then, consider a matrix factorization model with U2 and V₂ being the embeddings and item embeddings indexed by d, respectively. The model is expressed as:
s(x) U V 2. (a) sigmoid(s(z)).
(14)
The score function s is convex in model parameters (U, x and V.,, which immediately follows from
d
0,
dU²
(15)
d's
0.
dV
Therefore, Rweak is convex in U, and V., which immediately follows from the composition of convex functions is convex, Rweak convex in and is convex in model parameters.
Lemma 3. Suppose is recommender model based on matrix factorization, with the output score transformed by sigmoid function, and the surrogate risk (g) is convex with respect to the score The surrogate risk Rweak (g) is minimized by the recommender g. Then, the minimization of the surrogate risk Reweak (g) converges within a finite number of iterations.
Proof. According Theorem 4, the surrogate risk (wwak is convex with respect to the score s. Given the recommender g as matrix factorization, Rwwak (g) is convex in the model parameters, which immediately follows from the composition of convex functions. Therefore, the minimization for the convex (weak(g) converges within a finite number of iterations.
B.2. Solution to Proximal Transport Problem
The PT problem in (7) is linear programming problem and can be solved with traditional linear programming solvers (Gurobi Optimization, 2020; Hart et al., 2011). Nevertheless, such solutions often overlook the unique structure of optimal transport problems, thereby suffering from relatively large cost. In this section, we demonstrate a reformulation that aligns PT with the well-established Kantorovich problem, which permits solution to PT problem using specialized solvers for the Kantorovich problem, such as iterative Bregman projections (Benamou et al., 2015), Sinkhorn iterations (Altschuler al., 2017; Dvurechensky al., 2018), and the network-simplex method (Flamary al., 2021), with superior convergence rate and efficiency (Bonneel al., 2011; Ling and Okada, 2007; Pele and Werman, 2009).
To reformulate the PT problem (7) as Kantorovich problem, we introduce slack variable s E IR, and rephrase the inequality constraint π 1, S w b in (7) as an equality constraint:
. T 1n S w * b + T In + = w b,
(16)
which yields the equality constraints b as follow:
{
1. 11m + 1m (w bb) b)TTT 1m
(17)
1 T 1m = w |b]l
Combining (17) and (7), we reformulate the PT problem (7) as:
n' (a, 33 wr))= = arg mellae (C, TT) [T(; 3: =
Raxm TIm a, 1. 1 1, s E Rm π 11, + w b, s 1m
|[b]]]
To substantiate the structural identity between the constraint set II B; w) in PT and that the Kantorovich problem (Xu al., 2020), introduce augmented matrices and vectors follows:
0nx1
and i. = w b 01x1
ā
01x1
|
20

---

Unbiased Recommender Learning from Implicit Feedback via Weakly Supervised Learning
On the basis, we reformulate (x, w:w) as
iā 3: w) == {
Rixi
#1m = ā,
1, =b}
(18)
which precisely mirrors the constraint set in the Kantorovich problem. Furthermore, extend the transport cost matrix C to C to align with this formulation:
C E 5lm
(19)
2 A
which incorporates bounded scalar ç and a constant A max(Ci))) Xu al. (2020) shows that the transport cost Kantorovich problem can be rewritten C, Ā Therefore, the PT problem can be formulated
(ā, B; w) = arg min #, (C, Φ
(20)
In summary, through these augmentations and reformulations, encapsulated (20), we seamlessly recast the PT problem Kantorovich problem. This equivalence enables to leverage existing optimal transport solvers for solving the PT problem, thereby offering computational advantages and methodological coherence.
21


Reviews:
[{"review_id": "NOl05tjhai", "reviewer": "Reviewer_bC89", "summary": "This paper focuses on addressing the lack of negative feedback in recommendations with implicit feedback and points out the limitations of recent unbiased estimator-based methods in identifying propensity scores and non-negative estimates. It proposes a novel positive-unlabeled recommender learning (PURL) framework, in which the core idea is to introduce a new estimator and use progressive proximal transport (PPT) to estimate the required class priors of unobserved samples. Extensive experiments are conducted on three public datasets to verify the effectiveness of PURL.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**We sincerely appreciate the reviewer’s great efforts and insightful comments to improve our manuscript. In below, we address the raised concerns point by point and try our best to clarify any confusions.**\n\n#### **[W1] The connection between mass weight $w$ to class prior estimation is not explicit.**\n**Response.**  We agree that the connection needs clarification. \n- **Connection clarification.** We estimate the class prior as the inverse of the mass weight, i.e., $\\hat{\\kappa}_p=w$. \n- **Rationale of the connection.** The PT problem is designed to match the positive samples ($\\alpha$) with a selective set including $1/w$ of the unlabeled samples ($\\beta$). If there exist negative samples in the selective set, the PT discrepancy will be large. By finding the $w^*$ with minimum discrepancy, we can estimate the class prior $\\hat{\\kappa}_p = 1/w^*$. This process is visualized in Fig. 1. \n- **Toy example.** We provide simulated results in Fig.4. The ground-truth class prior $\\kappa=0.5$. When we set $w=2$, all positive samples are exactly matched with the proximal positive samples within the unlabeled population, producing the minimal transport cost.\n#### **[W2] The claim concerning BPR requires elaboration.**\n**Response.** Thank you for your insightful comment. We elaborate the claim as follows, and promise to add it in revision. \n- The BPR loss randomly samples unlabeled samples as negatives. However, **not all unlabeled samples are true negatives.** Some may simply be items the user hasn't encountered yet. This misclassification makes BPR biased.\n- In contrast, **our proposed PURL loss is unbiased** given accurate class prior estimation. This is supported by Theorem 1 in our paper, which provides a theoretical support for the advantage of the proposed PURL.\n\n#### **[W3] Computational time and memory requirement.**\n**Response.** Thank you for your insightful question. We add the analysis on computational time and memory consumption below.\n\n- The complexity of solving the proposed PPT problem is mainly determined by the number of samples. We conducted an empirical study on its running time, revealing that, although the theoretical complexity remains not straightforward to derive, the **running time scales linearly with the number of samples on a log-log scale.** In the **largest test with batch size of 2,048**—exceeding typical batch sizes in most practices—the running time was **under 0.1 seconds**, rendering it negligible compared to model training.\n- The memory consumption is also affected by the number of samples. **We add experiments where we used the decorator profile in the memory_profiler package to test the memory consumption** in solving the PT problem. According to the results below, **in the largest setting of 2048 samples, the memory consumption is 52.9 MB**, which is acceptable and even negligible compared to the memory cost of training typical recommendation system.\n\n| Batch size | 128 | 256 | 512 | 1024 | 2048 |\n|------------|-----|-----|-----|------|------|\n| Memory consumption | 1.2Mb | 4Mb | 10.5Mb | 20.7Mb | 52.9Mb |\n\n- We promise to adding these discussions in the revised manuscript.\n\n#### **[W4] The principle of dataset selection should be explained.**\n**Response.** Thank you for your kind reminder. We select the datasets based on the following principles:\n- Firstly, we opt for the datasets that contain unbiased test set with explicit feedback. This is crucial for evaluating the performance of our method, where the explicit feedback is used to calculate the metrics (NDCG, Recall, etc.), and the unbiased set mirrors the online serving scenario.\n- Secondly, we follow the practice of previous works in the ImplicitRec field. Specifically, the employed Yahoo and Coat datasets were used in CUMF (WSDM'20), UBPR (SIGIR'20), UPL (SIGIR'23), CDR (ICLR'23) and SDR (ICLR'23). \n- Moreover, motivated by UIDR (ICLR'24), we add an additional dataset--Kuairec--to provide comprehensive evaluation of the proposed method on large-scale industrial dataset.\n\n#### **[W5] The derivation of Eq.7.**\n**Response.** Eq. 7 defines a proximal transport problem, which is a generalization of the canonical OT problem in Eq. 2. \n- To derive Eq. 7, we replace the marginal constraint $\\boldsymbol{\\pi}^\\top \\boldsymbol{1}_n=\\mathbf{b}$ with $\\boldsymbol{\\pi}^\\top \\boldsymbol{1}_n \\leq w \\mathbf{b}$ with $w\\geq 1$, which acquires a semi-relaxed OT. \n- On the basis, we add the constraint $ \\boldsymbol{1}_m\\boldsymbol{\\pi}^\\top \\boldsymbol{1}_n = 1$, which forces the full mass to be transported to be 1. \n- Then, given the assumption that the mass in $\\alpha$ and $\\beta$ is uniformly distributed and sums to 1, we have $\\mathbf{a}=\\boldsymbol{1}_n$ and $\\mathbf{b}=\\boldsymbol{1}_m$. The derivation of Eq. 7 is completed.\n\n#### **[Other comments] Consistency of terms.**\n**Response.** Once again, thank you for your valuable comment. They refer to the same thing and we will use a unified term \"user-item pair\" in revision.", "claims_and_evidence": "The claims made in the submission are supported by clear and convincing evidence; the claimed problem (MNF) is a common problem in recommendation systems with implicit feedback.", "methods_and_evaluation_criteria": "The proposed methods and/or evaluation criteria (e.g., benchmark datasets) make sense.", "theoretical_claims": "I have checked the correctness of any proofs for theoretical claims. Some obstacles are presented in the questions below.", "experimental_designs_or_analyses": "I have checked the soundness/validity of any experimental designs or analyses. The experiments validate the overall performance of the proposed method.", "relation_to_broader_scientific_literature": "This paper focuses on the problem of missing negative feedback, an issue seen in the literature of implicit feedback recommendation. The proposed PURL is connected to the method from the weakly supervised learning field. The proposed PPT is an optimal transport approach with modification for achieving class prior estimation.", "essential_references_not_discussed": "The literature discussion in the main text is a little brief. \nIt is better to move the discussion of related work from the appendix to the main text.", "other_strengths_and_weaknesses": "Strengths:\n1. This paper targets missing negative feedback, an important issue in the field of implicit feedback recommendation. It also widely exists in real-world applications.\n2. The proposed method is promising to tackle the issue. It contains a creative fusing of OT and weakly supervised learning and an important application in recommendation.\n3. Diagrams are well prepared to facilitate understanding of important concepts and showcase efficacy. \n\nWeaknesses:\n1. The mass weight's connection to class prior estimation is not explicit, but it is a critical aspect to understand the subsequent contents.\n2. The claim that Bayesian Personalized Ranking (BPR) lacks a solid theoretical foundation requires elaboration. \n3. This paper utilized PPT, an OT-based approach for class prior estimation. However, OT is often costly in terms of both computation time and memory, as well as poor scalability w.r.t. the number of decision variables, which is quadratic to the batch size. Regrettably, in real-world recommendations, large batch size is often used in training recommendation systems for daily updates.  In this case, PPT might require longer training time and larger memory requirement.\n4. The principle of dataset selection should be explained. In this paper, the selected datasets are Yahoo, Coat, and Kuairec. Why not select some standard datasets such as ML-10M?\n5. Some steps in the derivation are difficult to follow. e.g., the derivation in equation 7. Necessary conditions and explanations are expected to facilitate readers to follow and check the derivation.", "other_comments_or_suggestions": "The consistency of the terms used needs improvement. For example, Are the user-item pair, user-item intersection, and user-item interaction the same thing?", "questions_for_authors": "See the above two parts.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "21vujp6u8J", "reviewer": "Reviewer_t2jt", "summary": "The paper addresses the challenge of missing negative feedback (MNF) in implicit feedback-based recommender systems. Traditional approaches often rely on negative sampling, which risks misclassifying positive samples as negative, leading to bias and performance degradation. The authors propose PURL (Positive-Unlabeled Recommender Learning), a framework that treats implicit recommendation as a weakly supervised learning task, thereby eliminating the need for negative samples.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Thank you very much for your kind and sincere words. It is impressive to meet a review that candidly express limitations while responsibly and analyzing the claims and details of the paper. We truly appreciate it!**\n\n\nWe have noticed the question about the dataset generation process and are happy to provide a detailed response.\n- Firstly, we chose the Yahoo and Coat datasets, which are commonly used in prior ImplicitRec studies such as CUMF [1], UBPR [2], and UPL [3].\n- In the training set, we retained only user-item pairs with positive feedback and treated those with negative feedback as unlabeled. This reflects real-world implicit feedback scenarios where only positive interactions are observed, and the rest are unlabeled. In the test set, we preserve the explicit feedback which is necessary to calculate the metrics (NDCG, Recall, etc.), mirroring the online serving scenario of implicit feedback recommendation.\n- This process aligns with common practices in ImplicitRec research [1-3].\n\n**Reference.**\n\n[1] Yuta Saito, Suguru Yaginuma, Yuta Nishino, Hayato Sakata, and Kazuhide Nakata. 2020. Unbiased Recommender Learning from Missing-Not-At-Random Implicit Feed- back. In WSDM. ACM, 501–509.\n\n[2] Yuta Saito. 2020. Unbiased Pairwise Learning from Biased Implicit Feedback. In ICTIR. ACM, 5–12.\n\n[3] Yi Ren, Hongyan Tang, Jiangpeng Rong, and Siwen Zhu. 2023. Unbiased Pairwise Learning from Implicit Feed- back for Recommender Systems without Biased Variance Control. In SIGIR. ACM, 2461–2465.", "claims_and_evidence": "- PURL estimator is unbiased relative to the ideal risk\nevidence: C.1 Theorem 1.\n- Progressive Proximal Transport (PPT) accurately estimates class priors\nevidence: Empirical results\n-  PURL outperforms existing implicit feedback methods", "methods_and_evaluation_criteria": "The paper introduces PURL (Positive-Unlabeled Recommender Learning) and Progressive Proximal Transport (PPT) as core components to address missing negative feedback (MNF) in implicit recommendation. I think the proposed method is theoretically and empirically sound.\n\nThe evaluation metric include NDCG@1 NDCG@3 NDCG@5, I think make sense. recall is also measured.", "theoretical_claims": "I think the theretical claims are well proved. I'm not an expert for theory, therefore please also refer to other reviewer's suggesitons.", "experimental_designs_or_analyses": "I think the experiment is comprehensive: 13 baselines, covering pointwise, pairwise, and unbiased methods, which are commonly used in implicit feedback recommendation research.\nThe dataset is also diverse: Yahoo! R3 and Coat are explicit feedback datasets converted to implicit feedback KuaiRec is a fully implicit feedback dataset.", "relation_to_broader_scientific_literature": "I believe the proposed method contributes to the literature as a model-agnostic framework that reframes implicit feedback recommendation as a weakly supervised learning task, eliminating the need for negative samples.", "essential_references_not_discussed": "I'm not familiar enough with the literature, so I don't see essential reference not discussed.", "other_strengths_and_weaknesses": "Overall, I think this is a good paper and all the claim are solid and sound for me. But I'm not working on the field for a while, so please also refer to other reviwer's suggestions.", "other_comments_or_suggestions": "See Above", "questions_for_authors": "A small question, will using Yahoo! R3 and Coat that transfer rating to implicit feedback influence the evaluation?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "um489YdaT3", "reviewer": "Reviewer_pJHQ", "summary": "- The study reframes implicit feedback recommendation as a weakly supervised learning problem, and introduces a model-agnostic framework, termed PURL, to handle the missing negative feedback problem. \n- Central to this framework is the incorporation of the PU-learning method, which ensures unbiasedness given positive and unlabeled samples given accurate prior estimation. \n- To achieve accurate prior estimation, authors proposed Progressive Proximal Transport (PPT), which estimates class priors by minimizing the proximal transport cost between positive and unlabeled data samples. \n- To validate the efficacy of the proposed approach, extensive experiments were carried out across multiple datasets, yielding results that underscore its practical utility and effectiveness.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "#### **[W1] Ablation study matters.**\n**Response:** Thank you for your kind comment! In this work, we devise two components: (1) the PURL loss; (2) the PPT strategy. **We have conducted experiments to discern their individual contributions**.\n- To discern the contribution of PURL that leverages unlabeled samples, we have conducted ablative experiments by changing the proportion of unlabeled samples involved in Fig.2. The results show that on both datasets, the performance increases with the proportion of involved unlabeled samples, which demonstrates the effectiveness of PURL in leveraging unlabeled samples.\n- To discern the contribution of PPT that estimates class priors, we have conducted ablative experiments in Table 3. The results show that the class prior estimate of PPT, with the smallest $\\mathbb{W}$, corresponds to the best recommendation performance.\n\n\n#### **[W2, Q1] Challenge of propensity estimation.**\n**Response:** Thank you for your insightful comment. We agree that the propensity score estimation is a general challenge, but we demonstrate that this challenge is exacerbated in ImplicitRec, for the following reasons:\n\n- In recommendation, the propensity score is typically the probability of a specific treatment given a user-item pair.\n- In ImplicitRec, this probability is hard to estimate due to the absence of negative feedback. For example, in CTR prediction, the propensity score is the exposure probability of a user-item pair. However, ImplicitRec only provides positive (exposure & click) and unlabeled samples (non-exposure), lacking negative samples (exposure & non-click). This makes estimating exposure probability infeasible.\n- This challenge is unique to ImplicitRec and significantly hampers the effectiveness of existing propensity-based methods in achieving unbiased recommendations in ImplicitRec.\n  \n#### **[W2, Q2] Why does ImplicitRec model outperform some explicit models?**\n**Response:** Thank you for your insightful comments. The phenomenon is noticable and we justify it from two aspects.\n- **Theoretical aspect.** The proposed PURL loss $R_\\mathrm{purl}$ **reduces the variance** of the ideal binary loss $R_\\mathrm{ideal}$ that uses explicit feedback, **given $\\kappa_p\\leq 0.5$ (a naturally holding condition) and large number of unlabeled samples.** The variance reduction property is justified in Theorem 2, which leads to better generalization and performance.\n- **Empirical aspect.** Figure 5 shows consistent performance improvement with more unlabeled samples, validating the theoretical aspect above. Thus, more unlabeled samples decrease PURL variance and improve performance over explicit feedback methods.\n\n#### **[W2, Q3] The connection with noisy label recommendation.**\n**Response.** Yes, it is feasible to employ learning-from-noisy-label techniques to address the MNF issue within ImplicitRec. However, the weakly supervised learning framework used in this work is more suitable for two main reasons:\n- **Theoretical supports**. Most noisy label methods are heuristic or rely on strong assumptions about the noise mechanisms. In contrast, the weakly supervised learning framework provides theoretical guarantees without such harsh assumptions.\n- **Empirical supports**. One exemplar work approaching ImplcitRec as learning from noisy labels is T-CE [1]. Our experiments below show that PURL outperforms T-CE, demonstrating the effectiveness of using weakly supervised learning for ImplicitRec.\n\n| Dataset | Model  | NDCG@1 | NDCG@3 | NDCG@5 | Recall@1 | Recall@3 | Recall@5 |\n|--|--|--|--|---|---|--|----|\n| Yahoo   | RecVAE | 0.65   | 0.683  | 0.719  | 0.054 | 0.149 | 0.234 |\n|  | T-CE   | 0.749  | 0.785  | 0.817  | 0.125 | 0.252 | 0.333 |\n|  | PURL   | 0.784  | 0.814  | 0.843  | 0.146 | 0.28 | 0.351 |\n| Coat    | RecVAE | 0.465  | 0.499  | 0.545 | 0.096 | 0.267 | 0.389    |\n|  | T-CE   | 0.448  | 0.47   | 0.509  | 0.081 | 0.228 | 0.347 |\n| | PURL   | 0.552  | 0.555  | 0.589  | 0.131 | 0.302 | 0.425 |\n| Kuairec | RecVAE | 0.422  | 0.434  | 0.448 | 0.11 | 0.183 | 0.214 |\n|  | T-CE   | 0.418  | 0.429  | 0.435  | 0.108 | 0.214    | 0.264    |\n|   | PURL   | 0.498  | 0.486  | 0.488  | 0.137  | 0.245    | 0.296    |\n\n#### **[W3] Originality of PPT.**\n**Response:** Thank you for the reminder. The PPT generalizes the canonical OT problem by relaxing the equality constraint for $\\beta$ and changes its total mass from 1 to $w$. This formulation is specifically designed for ImplicitRec and, to our knowledge, has not been previously studied.\n\n#### **[W4] Project website.**\n**Response:** Thank you for your kind reminder. The repository has now been restored. Additionally, we offer a Docker file to facilitate quick reproduction. \n\n\n#### **[Reference and Other comments].**\n**Response.** Thank you for your meticulous comment. We will fix the typos, remove the invalid reference, adjust the related works and unify the reference formats in revision. \n\n[1] Denoising implicit feedback for recommendation. WSDM. 2021.", "claims_and_evidence": "The proposed method provides a PU learning approach, which is feasible to handle this problem. Theoretical and empirical evidence is provided.", "methods_and_evaluation_criteria": "The proposed methods make sense and evaluation criteria follow the standard practice.", "theoretical_claims": "Theories are clear to follow and the sources from PU learning literature are cited.", "experimental_designs_or_analyses": "The empirical investigation of this study is comprehensive, and the analysis makes sense.", "relation_to_broader_scientific_literature": "In terms of methodology, this paper is related to the field of PU learning and optimal transport. In terms of application, it is related to the field of recommendation systems with implicit feedback.", "essential_references_not_discussed": "The section 'related works' discussed recent progress of PU learning and implicit feedback recommendation. Section 2.2 introduces optimal transport. Despite the primary references are covered, you can consider expanding the discussion to include broader applications of optimal transport in recommendation systems. This would provide a more comprehensive context for understanding the role and relevance of optimal transport within the field.", "other_strengths_and_weaknesses": "Strengths\n- This paper effectively highlights the significance of unlabeled data in recommendation systems.  An estimator that harness the unlabeled data for model training while ensuring unbiasedness is proposed.\n- The theoretical properties of the proposed estimator is investigated. The method using OT for class prior estimation is fresh for me.\n- According to Table 1-2, the proposed method exhibits superior empirical performance. Comprehensive empirical studies are conducted to validate the proposed method.\n\nWeaknesses\n- This paper has two main components. In this situation, an ablation study is critical to assess the contribution of each component to the overall performance. However, this study seems lacking.\n- Some statements in this paper need further clarification. Please see the questions below.\n- The originality of the proposed optimal transport problem--PPT--is not clarified. Authors should clarify the originality of PPT, or provide a concrete reference.\n- The code link is provided, but the repository seems to be expired.", "other_comments_or_suggestions": "- Reference format should be checked. For example, you used \"ACM Trans. Manage. Inf. Syst.\", ISO abbreviation for journals. However, you used \"NeurIPS\" and \"CIKM\" for conferences, which are not ISO abbreviations.\n\n- In section 4.4, you wrote \"According to Section 4.4\". Is there any reason to refer to the same section? Otherwise, it should be deleted.\n\n- In line 75, you wrote \"missing-not-at-random (MNF) issue\". It should be \"missing negative feedback (MNF) issue\", right?", "questions_for_authors": "- In line 40, you wrote \"identifying accurate propensity scores remains an elusive goal in ImplicitRec\". However, propensity score estimation is challenging in general, not only in implicit feedback recommendation. It does not interfere the success of propensity-based methods. \n- In table 1-2, it seems that some methods using negative feedback are included. Ideally they should outperform implicit feedback methods, since the accurate negative feedback is available, is it right?\n- I am curious about the connection between PURL and noisy label recommendation. Since the unlabeled samples are available, naively treating them as negative could lead to false labeling, which seems to be a noisy label problem. What is the advantage of formulating implicit feedback recommendation as a weak supervised learning problem over formulating it as anoisy label problem?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UIn3ZNnSBx", "reviewer": "Reviewer_Z2t6", "summary": "This paper proposes a novel approach named positive-unlabeled recommender learning, to handling the challenge of missing negative feedback in implicit feedback recommendation systems. The key contribution is Progressive Proximal Transport (PPT), an optimal transport-based method that estimates the class prior by minimizing the transport cost between positive and unlabeled samples. The paper presents theoretical justifications for the proposed method and validates it through extensive experiments on three real-world datasets, demonstrating superior performance compared to existing methods.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "#### **[W1] More discussion on scalability.**\n**Response.** We express our sincere gratitude for this valuable comment. **We add a scalability analysis on the additional ML-1M dataset that is larger than the datasets involved in this paper.** \nIn the below table, we summarize the performance of PURL given varying ratios of unlabeled samples (denoted as P). \n\n| P| 0.1 | 0.2 | 0.3 | 0.4 | 0.5 | 0.6 | 0.7 | 0.8 | 0.9 | 1 |\n|------|--|--|--|--|--|--|--|--|--|---|\n| NDCG\\@1| 0.81572 | 0.81875 | 0.81841 | 0.82143 | 0.81320| 0.82093 | 0.81455 | 0.82244 | **0.82463** | 0.82059|\n| NDCG\\@3| 0.80695 | 0.80828 | 0.81152 | 0.81185 | 0.80832 | 0.81150| 0.8101 0 | 0.81088 | 0.81037 | **0.81316**|\n| NDCG\\@5| 0.80176 | 0.80649 | 0.80738 | 0.80680| 0.80483 | 0.80667 | 0.80835 | 0.80744 | 0.80868 | **0.80978**|\n\n\nAs P increases, the performance of PURL generally increases with some fluctuations. The best performance is achieved when P>0.9, which is **consistent with the results reported in the main text and showcases scalability.** Compared to the results on smaller datasets, the performance is less sensitive to P. This can be attributed to the fact that even a small P in a large dataset corresponds to a large number of unlabeled samples, which suffices to meet the demand of unlabeled samples in Theorem 2 to reduce variance and improve overall performance.\n\n#### **[W2] The direct impact of PPT on performance.**\n**Response:** ! In this work, we introduce two components: (1) the PURL loss, which effectively leverages unlabeled samples; and (2) the PPT strategy, which accurately estimates class priors for calculating the PURL loss. To quantify the individual contributions of these components, we conducted a series of ablative experiments.\n- To discern the contribution of PURL, we varied the proportion of unlabeled samples (denoted as P) used in the PURL loss calculation, as illustrated in Figure 2. On both datasets,  the performance consistently improves as P increases, which demonstrates the effectiveness of PURL.\n- **To discern the contribution of PPT**, we compared the performance of PURL with PPT-estimated class prior and other values of class priors, in Table 3. The results show that the PPT-estimated class prior consistently led to the best recommendation performance. **This indicates that the PPT class prior is effective in estimating the class prior and thus improving the ImplicitRec performance.**\n\n#### **[Q1] Could PPT be extended to multi-class implicit feedback scenarios?**\n**Response.** We agree that **PPT can be extended to multi-class** scenarios. A plain approach would be treating the multi-value class labels as multiple binary classes. For example, if we have two positive classes (1, 2) and a negative class (0), we can treat them as two binary classes (0 v.s. 1 and 0 v.s. 2). The PPT can be applied to each binary class separately.\n\n#### **[Q2] Sensitivity to hyperparameter tuning.**\n\n**Response.**  Once again, we sincerely appreciate the reviewer's meticulous and constructive comments. We discuss the sensitivity to hyperparameter tuning in the following two aspects:\n- **Firstly, for the recommendation performance**, we conducted comprehensive experiments to investigate the sensitivity of key hyperparameters, including the ratio of unlabeled samples (Fig. 5), batch size (Fig. 7-a), learning rate (Fig. 7-b), and embedding dimension (Fig. 7-c). Different datasets exhibit varying sensitivity to these hyperparameters, which is largely dependent on the specific characteristics of the data. The detailed analysis is provided in Section 4.5 and Appendix B.3.\n- **Secondly, for the class prior estimation**, the PPT approach is generally isolated from these hyperparameters. To showcase this claim, we investigate the logs of sensitivity analysis that record the class prior estimation results $\\hat{\\kappa}_p$. Below are the results with different batch sizes, averaged over 5 random seeds:\n\n| Batch size | 256 | 512 | 1024 | 2048 | \n|--|--|--|--|--|\n| $\\hat{\\kappa}_p$| $0.0499_{\\pm0.018}$ | $0.0541_{\\pm0.0194}$ | $0.0607_{\\pm0.0212}$ | $0.0499_{\\pm0.0133}$ |\n\n Across a wide range of batch sizes, the estimated class prior $\\hat{\\kappa}_p$  remains stable around 0.05 (i.e., the class prior producing the best recommendation performance in Table 3). It indicates that PPT is robust to variations in hyperparameters.\n\n\n#### **[References] MNF in sequential recommendation should be discussed.**\n**Response.** Thank you very much for your insightful comment. We agree that the MNF problem also matters in sequential recommendation. In the revised version, we will include discussions on key works in this area, including but not limited to [1,2,3].\n\n[1] Modeling dynamic missingness of implicit feedback for recommendation. NeurIPS. 2018.\n\n[2] Modeling dynamic missingness of implicit feedback for sequential recommendation. IEEE TKDE. 2020.\n\n[3] Sequential learning over implicit feedback for robust large-scale recommender systems. PKDD. 2019.", "claims_and_evidence": "Yes. PURL provides an unbiased and consistent risk estimator for implicit feedback recommendation. PPT effectively estimates the class prior without requiring heuristic assumptions or propensity scores. Empirical results show that PURL outperforms state-of-the-art baseline methods.", "methods_and_evaluation_criteria": "Yes, the chosen datasets are sufficient and the NDCG is a reasonable metric for recommendation tasks.", "theoretical_claims": "I've checked the derivation of the PURL risk estimator and its unbiasedness, finding the claims are sound.", "experimental_designs_or_analyses": "I've checked the experimental setting and their results & analysis.", "relation_to_broader_scientific_literature": "The paper introduced progressive proximal transport to address the non-random missing problem, which is widely existing issue for broader research community. So the work has a potential of big impacts.", "essential_references_not_discussed": "The references discussed some classical research directions of implicit feedback recommendation and MNF, but overlooked one direction: MNF in sequential recommendation. This is a very practical direction since people interact with recommender systems in a ordinal manner. Prior work like below discussed this direction which is not discussed in this paper.\n- Wang, M., Gong, M., Zheng, X., & Zhang, K. (2018). Modeling dynamic missingness of implicit feedback for recommendation. Advances in neural information processing systems, 31.", "other_strengths_and_weaknesses": "Strengths:\nThe use of optimal transport for class prior estimation is innovative. Unbiasedness and consistency of the estimator are formally proven, and the empirical results on real-world datasets are promising.\n\nWeaknesses:\nMore discussion on scalability for large datasets is needed. In reality the data sparsity will reach 99% in recommender systems. How would proposed PPT perform is still unclear. The direct impact of PPT on performance is not fully isolated.", "other_comments_or_suggestions": "1. Could PPT be extended to handle multi-class implicit feedback scenarios instead of binary PU learning?\n2. How sensitive is PURL’s performance to hyperparameter tuning, particularly in estimating the class prior?", "questions_for_authors": "n/a", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper presents positive-unlabeled recommender learning (PURL), aiming to address the ambiguity of the missing entries (0's) in implicit feedback data (a 0 can mean either negative feedback or not aware of the item). Overall the paper received positive scores. All the reviewers acknowledge the importance of the studied problem and the proposed solution appears theoretically sound, and supported by empirical evidence. There were some shared concerns regarding the scalability of the optimal transport component of the method, which are largely resolved during the author-reviewer discussion phase. 

However, after reading all the reviews, I am not sure if some of the reviewers' expertise perfectly aligns with this paper. Therefore, I read the paper very carefully myself (plus getting some external opinions from people whose research is on this very topic). 

Let me start with the strengths: this paper tackles the unbiased recommender systems learning problem from an alternative angle as a weakly supervised learning problem (prior work often frames the task from a causal inference perspective and replies on some kind of propensity adjustment which can often be problematic), and consequently introduces an objective function that doesn't involve any propensity weighting, instead depends on the positive class prior $p(y=+1)$. The paper then revolves around estimating this positive class prior using an optimal-transport based approach. The empirical results demonstrate that the proposed method outperforms IPS-based baselines on three standard benchmark datasets for this kind of setting (Yahoo music, coat, and KuaiRec).

Regarding the weaknesses: I think the biggest one would be the assumption made in Section 3.1 where the unlabeled data distribution $p_u(x)$ follows the same distribution as the underlying data distribution $p_{data}(x)$. I am not convinced that this assumption is entirely correct. Intuitively this implies the unlabeled data has the same ratio of positive and negative as the (idealized) fully-observed data. However, we know the negative remains unchanged between the unlabeled data and the fully-observed data, but positive from the fully-observed data gets "distorted" by us observing some of the positive interactions. So in some sense, the more positive interactions we observe, the less correct this assumptions will be -- consider the extreme case of observing all the positive, then unlabeled data would consist of only negative which clearly does not follow the same distribution as $p_{data}(x)$. Fortunately typical RecSys dataset is highly-sparse hence this assumption is probably approximately correct. However, I wonder if there is any existing work validating this assumption, and if not, I feel that more emphasis should be put on addressing this. 

Another weakness as pointed out by many other reviewers is the scalability. The authors did provide some evidence that the algorithm is capable of running on small-sized datasets with simple matrix factorization model. How would it scale to large-scale datasets/more complex models which are commonplace these days for recommender systems research?

Finally, a comment regarding the experimentation setup: it seems that all the algorithms are optimized with SGD. However, I think it has been observed that if a closed-form alternating least-square-style solution is available, it can often outperform SGD by a rather big margin. And I believe for some of the baselines, it is possible to run ALS-style optimization. How would the proposed method compare in that case? 

In summary, I believe this paper certainly has enough strengths but I don't think it is as solid as its current scores indicate. Therefore, I am voting for a weak accept and hope the authors can take the feedback into account. 

- One minor comment: I noticed in the experimental section, WMF (Hu et al. 2008) is listed as "unbiased" point-wise method. I don't think that is correct since it downweights 0's uniformly.
Decision: Accept (poster)