Inverse Learning with Extremely Sparse Feedback for Recommendation
Authors: Guanyu Lin
Yu Zheng
Chen Gao
Jianxin Chang
Yanan Niu
Yang Song
Zhiheng Li
Depeng Jin
Yong Li
Venue: Submitted to ICLR 2023 (2023)
Keywords: Recommender System
Unlabeled Data
Denoising Training

Abstract: Negative sampling is widely used in modern recommender systems, where negative data is randomly sampled from the whole item pool. However, such a strategy often introduces false-positive noises. Existing approaches about de-noising recommendation mainly focus on positive instances while ignoring the noise in the large amount of sampled negative feedback. In this paper, we propose a meta learning method to annotate the unlabeled data from loss and gradient perspectives, which considers the noises on both positive and negative instances. Specifically, we first propose $\textit{inverse dual loss}$ (IDL) to boost the true label learning and prevent the false label learning, based on the loss of unlabeled data towards true and false labels during the training process. To achieve more robust sampling on hard instances, we further propose $\textit{inverse gradient}$ (IG) to explore the correct updating gradient and adjust the updating based on meta learning. We conduct extensive experiments on a benchmark and an industrially collected dataset where our proposed method can significantly improve AUC by $9.25\%$ against state-of-the-art methods. Further analysis verifies the proposed inverse learning is model-agnostic and can well annotate the labels combined with different recommendation backbones. The source code along with the best hyper-parameter settings is available at this link: https://anonymous.4open.science/r/InverseLearning-4F4F.


Under review as a conference paper at ICLR 2023
INVERSE LEARNING WITH EXTREMELY SPARSE FEED- BACK FOR RECOMMENDATION
Anonymous authors Paper under double-blind review
ABSTRACT
Negative sampling is widely used in modern recommender systems, where nega- tive data is randomly sampled from the whole item pool. However, such strategy often introduces false-negative noises. Existing approaches about de-noising rec- ommendation mainly focus on positive instances while ignoring the noise in the large amount of sampled negative feedback. In this paper, we propose a meta learning method to annotate the unlabeled data from loss and gradient perspec- tives, which considers the noises on both positive and negative instances. Specif- ically, we first propose inverse dual loss (IDL) to boost the true label learning and prevent the false label learning, based on the loss of unlabeled data towards true and false labels during the training process. To achieve more robust sam- pling on hard instances, we further propose inverse gradient (IG) to explore the correct updating gradient and adjust the updating based on meta learning. We con- duct extensive experiments on a benchmark and an industrially collected dataset where our proposed method can significantly improve AUC by 9.25% against state-of- srce-offthe ff-the-art art methods. Further analysis verifies the proposed inverse learning is model-agnostic and can well annotate the labels combined with different rec- ommendation backbones. The source code along with the best hyper- parameter settings is available at this link: https: / anooymooss 4open. science/ r/ e/Inerrseeearnigg 4F 4F.
INTRODUCTION
As one of the most successful machine learning applications in industry, recommender systems are essential to promote user experience and improve user engagement (Ricci et al., 2011: 1; Xue al., 2017: Liu et al., 2010b), which are widely adopted in online services such as E-commerce and Mirco-video platforms. Aiming to capture users' preference towards items based on their histori- cal behaviors, existing recommenders generally focus on explicit or implicit feedback. Specifically, explicit feedback (Liang et al., 2021) refers to rating data that represents the user preference ex explic- itly. However, collecting sufficient explicit data for recommendations is difficult because it requires users to actively provide ratings (Jannach et al., 2018). In contrast, implicit feedback, such as user clicks, purchases, and views (Liang et al., 2016), is much richer (Liu al., 2010b) and frequently used in modern recommender systems (Chen et al., 2020). Particularly, in feed recommendation for online platforms such as Micro-video, users are passive receive the recommended items without any active clicking or rating action. That is to say, we have a large number of unlabeled feedback, with extremely sparse labeled data, which becomes a key challenge for recommendation.
To tackle the unlabeled feedback, most works (He et al., 2017; Chen et al., 2019) randomly sample unlabeled data and treat it as negative feedback, resulting in unavoidable noise. Specifically, the collected user click data is often treated as positive feedback, and the unclicked data is sampled as negative feedback (He et al., 2017: Chen et al., 2019). However, there may be some positive unlabeled data sampled by the negative sampling strategy, which means that these instances will be false-ne There are also some works about hard negative sampling which will decrease the false-positive but increase the false-negative instances (Zhang et al., 2013; Ding et al., 2019; 2020). These hard negative methods tend to perform poorly when tested on both true positive and negative data instead of true positive but sampled negative data (as shown in Appendix 5). A recent work DenoisingRec (Wang et al., 2021) denoises positive feedback by truncating or reweighing the loss of false-positive instances without further consideration of the noisy negative feedback. In general,

---

Under review as a conference paper at ICLR 2023
existing works either focus only on the positive or the negative perspectives. To resolve the data- noise problem from both positive and negative perspectives at the same time, we propose a novel Inverse Learning approach that automatically labels the unlabeled data via inverse dual loss and inverse gradient, assuming there are both positive and negative feedbacks in the unlabeled data. Firstly, based the property that the loss of false positive/negative instance will be larger than that of a true positiveeneegatve instance (Wang et al., 2021), we assign both positive and negative labels to the unlabeled instance with different weights calculated by inverse dual loss. Specifically, we assign larger (smaller) weight to the label with smaller (larger) loss value. In this way, we take full advantage of true positive/negative instances, and eliminate the noise of false positive/negative instances. To achieve more robust sampling on hard instances, we further propose an inverse gradient method. Here we build a meta-learning process (Finn et al., 2017; Li et al., 2017) and split the training data into training-train and training-test data. We first exploit training-train data to pre-train the model. Then we further use training-test data to validate the correctness of classification on the sampled instance. Specifically, we calculate the gradient for the inverse dual loss of sampled instances as well as the additive inverse the gradient. The model is optimized by either the direct gradient or the additive inverse of gradient, determined by the split training- test data. Experimental results illustrate that inverse gradient can truly improve the inverse dual loss. In summary, the main contributions of this paper are as follows:
To the best of our knowledge, we are the first to sample both positive and negative data in recom- mendation, which is far more challenging while existing works only sample negative data.
We propose inverse dual loss to learn the label for sampled instances and further exploit inverse gradient to adjust the false label for hard instances.
We experiment on two real-world datasets, verifying the superiority of our method compared with Mate-of-theeard approaches. Further studies sustain the effectiveness of our proposed method in label annotation and gradient descent.
2 PROBLEM FORMULATION AND OUR APPROACH
In this section, will firstly formulate the problem and perform in-depth analysis of existing lutions and their limitations. Then we will propose inverse dual loss to address the limitations of existing works for easy samples. Finally, we further propose inverse gradient address the limita- tion of inverse dual loss and make it capable for not only easy samples but also hard samples.
2.1 PROBLEM DEFINITION
The recommendation task aims to model relevance score in: = fu, (l)) of user u towards item i under parameters 0. The LogLoss function (Zhou et al., 2018; 2019) function to learn ideal param- eters 0* is as:
(D-(0)
8 ( Vin) (L..D,,)ED
Σ
(1)
[
where É (G Vi.s) = - log (ihn) + (1 - yoee log (1 I.)). via E {0,1) is the feedback of user u towards item i. D {(u, i, y E e I the reliable interaction data between all user-item pairs. Indeed, due to the limited collected feedback, the model training is truly formalized as: arg ming CD + LD (0). where D ~ D* is the collected labeled data, and 2 {uu, i, Mus) | u E, E I} is the sampled unlabeled data where T.. = 0 is often assumed in existing recommenders for negative sampling. However, such a strategy will inevitably introduce noise because there are some positive unlabeled instances in the sampled data. As consequence, model (i.c., 0) trained with noisy data tends to exhibit suboptimal performance. Thus, our goal is to construct denoising recommender approximating to the ideal recommender 0* as:
arg min Lp: (0) + 0
Adenoise L
where C (0) indicates the loss on unlabeled data with all samples annotated correctly, iee de- noising sampling.
2

---

Under review as a conference paper at ICLR 2023
Unlabeled data
Positive data
Negalive data
(a) Unlabeled
(b) Negative sampling
(c) Reweight loss
(d) Truncated loss
(e) Easy Negatives
(f) True-negative label
(g) Easy Positives
(h) True- Prue-positive label
Hard Negatives
(j) Palse-positii label
(k) Hard Positives
(I) False-negative label
Figure 1: Illustrations of existing solutions, and our inverse dual loss's effectiveness and limitation.
(a)-(d) are the illustrations of existing solutions: (a) illustrates there are lot of unlabeled data; (b) illustrates the traditional negative sampling approach; (c) illustrates the reweighted loss of Denois- ingRec; (d) illustrates the truncated loss of DenoisingRec adapted on the false-negative instance. (e)-(f) are the illustrations of our inverse dual loss's ss effectiveness with easy sampling: (e) illus- trates the easy negative instances are sampled; (f) illustrates labeling the sampled instances as true negative; (g) illustrates the easy positive instances are sampled: (h) illustrates labeling the sampled instances as true positive and approximate to ground-truth. (i)-() are the illustrations of our inverse dual loss's limitation with hard sampling: (i) illustrates the hard negative instances sampled; (i) illustrates labeling part of the sampled instances as false positive; (k) illustrates the hard positive instances are sampled: (1) illustrates labeling part of the sampled instances as false negative.
2.2 INVERSE DUAL LOSs
In this section, we first analyze the characteristics of existing solutions the sampled unlabeled data. Then we introduce the proposed inverse dual loss solution to denoise sampled data.
2.2.1 ANALYSIS OF EXISTING APPROACH
We first explain the data sparsity problem in recommender systems from the perspective of classifi- cation boundary, based on which we will introduce the existing solutions. As shown in Figure (a), in recommender systems, labeled data tends to be extremely sparse compared with large number of unlabeled data. A recommendation model is prone to overfitting if it is only trained based on the sparse labeled data, compared with the ground-trutt -trth Figure (h).
In practice, existing recommenders often sample from unlabeled data and treat all the sampled data as negative feedback. Such an approach introduces false negative, which fails to retrieve items that users may be interested in, as shown by the classification boundary in Figure 1 (b). That is, there exists noise in the sampled negative data. However, existing denoising approaches mainly focus on the noise in positive samples (false positive). For example, DenoisingRec (Wang et al., 2021) attempts to achieve denoising for false positive instances as:
0' arg min L denoise
(0) + (D)
where Dmoise = ((,,i,1) u E U, E Vis = 0} is the noisy false positive data they introduce in experiments. For example, R-CE (Reweight Cross-Entropy) of DenoisingRec assigns lower weight on false-positive instances with large loss (Figure (c)), and T-CE (Truncated Cross-Entropy of DenoisingRec discards those false positive instances with large loss (Figure 1 (d)). Though achiev- ing denoising for false-positive instances, they ignore false-negative instances and fail to address the noise brought by the negative sampling.
3

---

Under review as a conference paper at ICLR 2023
In fact, the collected labeled data is much cleaner than sampled unlabeled data, and the number of false-pose posseposiiive instances limited in real-world recommender systems. On the contrary, the noise brought by negative sampling is far more harmful. In other words, the noise level of positive un- labeled data incorrectly sampled as negative feedback is much higher than that of negative samples wrongly regarded positive feedback.
To sum up, existing solutions either introduce noise or perform incomplete denoising, which moti- vates us propose denoising solution for unlabeled data.
2.2.2 LABELING WITH INVERSE DUAL LOsS
As an existing attempt DenoisingRec, we have discovered that the false positive instances are with greater loss. It is also apparent phenomenon machine learning. For example, if we have a positive instance and a well-trained model, the loss of classifying it as negative will be greater than that of classifying it as positive. Otherwise, if we have negative instance and well-trained model, the loss of classifying it as positive will be greater. Hence, we can assume the sampled unlaheled instances both possibly positive and negative and then exploit this inherent characteristic to automatically weigh more on the true positive or negative instances while weighing less on the false ones.
Definition (Inverse Dual Loss) The inverse dual loss is defined as denoising loss to automatically classify the unlabeled data
Edencise D=
()
D
(duat(O)
(0)
Σ
w'l (Dlss 1) + 2 (jla 0)
(4)
iDu
1e 10(82.001
18(12..1)
where w spectively. Zu
w
1e (02,0.0) are the weights for positive loss and negative loss, re- the normalization parameter. Du is the sampled unlabeled
10 112...11
Zu
le( 1
[(G
data and the model parameters be learned.
The pros inverse dual loss are as shown in (c)-(h) Figure I: (c) when the easy negative in- stances sampled, the loss of classifying them positive will be greater than that of classifying them as negative, and thus inverse dual loss will assign more weights on the negative loss; (f) grad- ually assigning more and more weights the negative loss, the negative unlabeled instances will eventually be classified as negative; (g) likewise, when the easy positive instances are sampled, the inverse dual loss will assign more weights the positive loss; (h) the positive unlabeled instances will eventually be classified as positive and approximate to the ground-truth
2.2.3 LIMITATION OF INVERSE DUAL. LOSS
When sampling the easy positive negative instances, inverse dual loss can boost the learning via correctly labeling the sampled instances. However, it may be an obstacle when there are some hard positive negative instances. As shown in (i)-(I) of Figure 1, given some hard positive or negative instances, the classification boundary will be prevented from the ground-truth: (i) the hard negative instances are sampled: (j) half of the negative unlabeled instances will be classified as pos- itive and become noise; (k) the hard positive instances are sampled; (I) half of the positive unlabeled instances will be classified as negative and become noise. That is to say, our inverse dual loss relies heavily on the current training classification boundary and the difficulty of sampled data, requiring us further improve its robustness.
2.3 INVERSE GRADIENT
To improve the robustness of inverse dual loss on the hard sampled data towards the current training model, in this section, further propose inverse gradient adjust the gradient of hard sampled data, inspired by the meta-learning framework (Finn et al., 2017; Li et al., 2017). Then we analyze the convergence of the proposed inverse gradient.
4

---

Under review as a conference paper at ICLR 2023
2.3.1 LEARNING TO LABEL WITH INVERSE GRADIENT
In this part, we introduce our solution for tackling the hard sampled data of inverse dual loss.
Definition 2 (Inverse Gradient) We define the gradient and additive inverse of gradient calculated by equation 4 w.r.t. v L (0) and -V -VL L (0) as direct gradient and inverse gradient, respec- lively, of the loss for unlabeled data D
Theorem Given learning rate a E R, a + O, assume the temporal model parameters updated by the direct gradient and inverse gradient. respectively, are as 8
a V. (0) and O V V (0). Then, the relationship between the loss of them and the model with parameter on data D will be either L (0*) (0) LD (0') or L (0') CDi L (04).
The proof of Theorem can be found in Appendix A.I. Based this theorem, we can have the following gradient updating strategies. Generally, we will first split the training data into training- train data and training-test data, where pre-train the model on the training-train data. Then will calculate the direct gradient of inverse dual loss on the sampled unlabeled data, which can further results in the following three cases:
When the sampled data is easy, we can exploit the direct gradient to update the model, and it will gain a smaller test loss on the training-test data, as shown in Figure 2 (a);
When the sampled data is hard, exploiting the inverse gradient update the model will gain smaller test loss on the training-test data, and thus we exploit inverse gradient here as shown in Figure 2 (b);
When the model is approximately optimal, either direct gradient inverse gradient will prevent it from ground-truth, and we discard this batch unlabeled data as shown in Figure 2 (c);
Training model
model
Optimal model
Direct gradient
Inverse gradient
Test gradient
2
2
(a) Update with direct gradient
(b) Update with inverse gradient
2
(c) Pass this batch of unlabeled data
(d) Convergence analysis
Figure Illustrations inverse gradient adaption.
Specifically, we present the procedure of exploring these three cases as Algorithm in the Ap- pendix A.2. The algorithm first pre-trains the model using the split training-train data. Then the model will update with direct gradient or inverse gradient or even do not update, determined by the validation the split training- data.
2.3.2 CONVERGENCE ANALYSIS
As shown in Figure 2 (d), the first case illustrates when the unlabeled data is ideally sampled, the updating with direct gradient will lead to smaller loss and better convergence on the training- test data, while the third case with poorly sampled data supposes to update with inverse gradient. However, when the model approximates convergence on the training-test data, updating with either direct gradient or inverse gradient may be poorer than updating shown in the stage 2 of Figure (d).
To avoid the gradient ascent problem for the second case, dual loss to be smaller than the test loss, ie., a < y in Algorithm 1. this way, the scale of updating by the gradient for inverse dual loss will be within the scale of updating by the gradient for
set the learning rate for the inverse
5

---

Under review as a conference paper at ICLR 2023
test loss. That is to say. the gradient ascent problem is less likely the inverse dual loss for unlabeled data than the loss for labeled data.
EXPERIMENTS
In this section, we perform experiments on two real-world datasets, targeting three research ques- tions (RQ): (1) How does the proposed method perform compared with the state-of-ther offtthearrt denoising recommenders? (2) How does our proposed inverse dual loss identify the unlabeled data? (3) What is the effect of the inverse gradient on convergence?
Datasets To practice and verify the effectiveness of our proposed method, we conduct experiments on an industrial Micro-Video dataset and public benchmark MLIM dataset which is widely used existing work for recommender systems (Lian al., 2018; Cheng et al., 2020). Micro- Video is the extremely sparse dataset where users are passive to receive the feed videos and have rare active feedback. We introduce the details of these datasets, including the pre-processing steps, in Appendix A.3.
Baselines and Backbones To demonstrate the effectiveness of our proposed inverse learning on unlabeled data, compared the performance of recommenders trained by our inverse gradient (IG) with recommenders trained by inverse dual loss (IDL) and normal training by standard loss or negative sampling (NS) (Rendle al., 2012; He et al., 2017; Chen et al., 2019). Besides, also compare our inverse learning method with the state-of-the-art methods for denoising recommender systems. Specifically, we also compare two adaptive denoising training strategies, T-CE and R-CE, of DenoisingRec (Wang et al., 2021).
Traditional strategies: standard loss without sampling and with negative sampling (NS) (Rendle et al., 2012; He et al., 2017; Chen et al., 2019).
Denoising strategies: T-CE (Wang et al., 2021) DenoisingRec that truncates the loss for false- positive instances and R-CE (Wang et al., 2021) of DenoisingRec weighs less the loss for false-positive instances; Our inverse dual loss that weighs more on the label with smaller loss.
Following DenoisingRec (Wang et al., 2021), we select GMF and NeuMF (He et al., 2017) as back- bones, which are neural Collaborative Filtering models.
GMF (He et al., 0017): A variant of matrix factorization with the element-wise product and a linear neural layer as the interaction function instead of the inner product.
NeuMF (He et al., 2017): A combination of GMF and Multi-Layer Perceptron (MLP).
Evaluation Metries We adopt widely-used AUC and GAUC as accuracy metrics (Gunawardana & Shani, 2015). Besides, two widely-used ranking metrics (Chang et al., 2021), MRR and NDCG @ 10, are also adopted for evaluation.
dlyper-paramede Settings For the two denoising strategies (Wang et al., 2021), followed their default settings and verified the effectiveness our methods under the same conditions. The em- bedding size and batch size of all models set as 32 and 1,024, respectively. Besides, adopt Adam (Kingma& & Ba, 2015) to optimize all the model parameters with the learning rate ry initialized as 0.0001 and 0.00001 for labeled data ML1M and Micro-Video datasets, respectively, while the learning rate for sampled data is set as a = 0.1y. As for the inverse gradient, we split 90% of the training data training-train data, and the left is used as training-test data. The sampling rate is as 1. The provided code has included the best hyper-parameters.
3.1 OVERALL PERFORMANCE
The performance comparison is shown in Table 1, from which we have the following observations.
Our inverse gradient performs best. Our inverse gradient (IG) method achieves the best perfor- mance compared with four baselines and our inverse dual loss (IDL) for three metrics. Specifi- cally, our IG improves the backbone sharply, which shows the ability of proposed method to well classify the unlabeled data and achieve effective data augmentation to resolve the data spar- sity problem of existing recommenders. Besides, IG outperforms the existing negative sampling

---

Under review as a conference paper at ICLR 2023
Table 1: Performance comparisons with GMF and NeuMF backbones datasets. Bold and
underline refer to the best and second best result, respectively. Here IG includes the IDL method.
ML1M
Micro- Video
Model
Method
NDCG
AUC
GAUC
NDCG
AUC
GAUC
None
0.9285
0.7671
0.6919
0.7365
0.8024
0.7558
NS
0.9400
0.7639
0.7200
0.7049
0.7802
0.7247
T-CE
0.9349
0.7612
0.7163
0.6994
0.6486
0.6956
GMF
R-CE
0.9391
0.7632
0.7192
0.7069
0.7820
0.7260
IDL
0.8996
0.7304
0.6400
0.7272
0.7858
0.7335
IG
0.9521
0.8318
0.7642
0.7773
0.8033
0.7593
None
0.9214
0.7524
0.6856
0.6649
0.7504
0.6916
0.9298
0.7495
0.7088
0.6350
0.7191
0.6689
T-CE
0.9351
0.7587
0.7158
0.6725
0.7469
0.6911
NeuMF
R-CE
0.9349
0.7521
0.7117
0.6288
0.7038
0.6568
IDL 0.9212
0.7962
0.6984
0.6466
0.7174
0.6647
IG
0.9449
0.8253
0.7569
0.7809
0.8198
0.7689
(NS) method, which means there is truly a large number of positive unlabeled data, and directly treating them all negative feedback will confuse the model. Finally, IG also outperforms exist- ing ofate-of-art denoising methods, T-CE and R-CE, showing the importance of tackling the noise from both positive and negative feedback.
Inverse gradient can make inverse dual loss more robust. Inverse dual loss (IDL) only outper- forms the NeuMF on MLIM dataset for AUC and GAUC, which shows the inferior robustness of IDL since depends heavily on the training model and sampled data. Besides, it is even out- performed by GMF on ML.M and NeuMF on Micro-Video, which means the poor sampling of unlabeled data will have negative impact on the model training. These results show the signifi- cance of improving the robustness of IDL and confirm that is necessary to exploit IG to adjust IDL.
3.2
IDENTIFICATION ON UNLABELED DATA
GMF
GMF
NeuMF
NeuMF
15.00
15.00
20.00
20.00
Densily
Density
10.00
Density
10.00
10.00
5.00
5.00
0.00
0.35
0.60
0.00
0.10
0:45
0.00
0.50
0.75
1.00
0.00
0.00
0.25
0.50
Postive Weight
Negative Weight
Postive Weight
Negalive Weight
GMF
NeuMF
NeuMF
15.00
7.50
7.50
Density
Density
Density
10.00
Density
5,00
5.00
2.50
2.50
5.00
0.00
0.00
0.00
0.00
Positive Weight
Negative Weight
Positive Weight
Negative Weight
Figure 3: Positive and negative weight distributions for dual loss on MLIM at first (up) and final (bottom) epochs.
We visualize the distribution of weights for positive loss and negative loss MLIM dataset in Figure 3. In specific, the upper part represents the weights at the first epoch after pre- pre-training on the training- train data, and the bottom part represents the weights at the convergence epoch. From the figure, we can observe that: (1) at the beginning, GMF model fails to well classify the unlabeled data but NeuME model has well captured the pattern of unlabeled data, which shows the generalization

---

Under review as a conference paper at ICLR 2023
GMF
GMF
NeuMF
NouMF
Density
Density
Density
Density
2,00
200.00
200,00
1.00
0.00
0.40230465000000 465004665
(15220.555005577
Postive Weight
Negative Weight
Postive Weight
Negative Weight
GMF
GMF
NeuMF
NeuMF
3.00
3,00
3,00
Density
2,00
Density
2,00
Density
2.00
Density
2,00
1.00
1.00
1.00
0.00
0.00
0.00
0.00
Positive Weight
Negative Weight
Positive Weight
Negative Weight
Figure 4:
Positive and negative weight distributions for dual loss on Micro-Vides first (up) and final (bottom) epochs.
ability of deep learning model (Cheng et al., 2016); (2) after convergence, both GMF and NeuMF well classify the unlabeled data and capture the similar pattern where most sampled instances are labeled as positive with more positive weights.
Besides, Figure 4 demonstrates the distribution of weights for positive loss and negative loss on Micro-Video dataset, which shows similar results as that on ML1M dataset despite there being more balanced positive and negative labels the unlabeled data.
3.3
IMPACT OF INVERSE GRADIENT ON CONVERGENCE
GMF on ML Dataset
NeuMF on 1L Dataset
0.60
direct
inverse
direct
pass
Inverse
0.40
Loss
Loss
0.40
0.30
500
1000
200
400
600
800
Training Iteration
Training Iteration
GMF on Micro Video Dataset
NeuMF on Micro Video Dataset
0.60
direct
pass
inverse
direct
pass
inverse
0.60
Loss
Loss
0.50
0.40
0.40
2000
4000
6000
8000
0
2000
4000
6000
8000
Training Iteration
Training lteration
Figure 5: The loss on training-test data with adapted gradient from the dual loss of unlabeled data. Each point of the loss curve is marked by its updated gradient direction.
To investigate the convergence our proposed inverse gradient, we also plot the loss curve on the training-tes test data shown in Figure 5. Based the results MLIM dataset, that: (1) the pre-tree vre-training on training-train data can promote the model learning effectively at the early stage for both GMF and NeuMF models, while NeuMF will come into fast convergence in
discover
8

---

Under review as a conference paper at ICLR 2023
the first epoch; (2) GMF model is updated with direct gradient, then with hybrid of direct and inverse gradients, and finally with inverse gradient while in NeuMF model we can discover more hybrid of gradients in the valley of loss curve, this is because the test gradient is more likely to ascent at the valley and thus pushes the gradient for dual loss to adjust its direction. Besides, based on the results Micro-Vides dataset, gradient. This is because the learning rate is relatively low here (as analyzed in the Appendix A.5), leading almost no gradient ascent problem; (2) the pre-training on the training-train data will conftict with the test data, which means deep learning based model is prone to be overfittigg (Cheng al., 2016) in the sparse data. Most importantly, we can discover that there is no case of passing gradient in the descent procedures, which confirms our analysis at Section 2.3.2 that setting lower learning rate a can avoid the gradient ascent problem for inverse dual loss.
discover that: (1) the models are always updated with direct
4 RELATED WORK
Implicit Feedback with Negative Sampling. Existing recommenders are generally based on im- plicit feedback data, where the collected data is often treated as positive feedback, and negative sampling (Rendle et al., 2012; He et al., 2017: Chen et al., 2019) exploited to balance the lack of negative instances. However, the negative sampling strategy will introduce noise because there are some positive unlabeled (Elkan & Noto, 2008; Bekker et al., 2019; Saito et al., 2020) data in the sampled instances. To improve existing implicit feedback recommendation, the identification of negative experiences (Fox et al., 2005; Kim et al., 2014) has grabbed the researchers' attention. However, these methods collect either the various user feedback (e.g., dwell time (Kim et al., 2014) and skip (Wen et al., 2019)) or the item characteristics (Lu et al., 2018), requiring additional feed- back and manual labeling, e.g., users are supposed actively provide their satisfaction. Besides, the evaluation of items relies heavily on manual labeling and professional knowledge (Lu et al., 2018). Thus in practice, these methods are too expensive to implement in real-world recommenders. In addition, hard negative sampling is adopted to improve the negative sampling (Zhang et al., 2013; Ding et al., 2019; 2020). However, with less false positive samples, the hard negative instances also brings more false-negative samples. Our meta-learning method elegantly annotates the unlabeled instances based on the sparse labeled instances.
Denoising Recommender Systems. One intuitive approach to reduce noise is to directly include more accurate feedback (Liu et al., 2010a; Yang et al., 2012), such as dwell time (Yi et al., 2014) and skip (Wen et al., 2019)). However, forcefully requiring additional feedback from users may harm the user experiences. To address this problem, DenoisingRec (Wang et al., 2021) achieves denoising recommendation for implicit feedback without any additional data. More specifically, they perform denoising on the false positive instances via truncating or reweighting the samples with larger loss. However, they only consider the positive feedback without further addressing the noise brought by negative sampling, Our work considers the sampled instances as possibly positive and negative, then achieve denoising data augmentation from both positive and negative perspectives.
5 CONCLUSIONS AND FUTURE WORK
In this paper, we proposed novel method that learns the sampled data with both positive and negative feedback. Such exploration not only tackled the unavoidable noise brought by widely used negative sampling but also improved the current denoising recommenders. Specifically, we proposed inverse learning from both loss and gradient perspectives. The first one was the inverse dual loss that assumed the sampled data be possibly positive or negative. the positive loss was greater than the negative loss (difficult to label the data as positive), the inverse dual loss would inversely assign more weights to the negative loss and vice versa. Since the inverse dual loss depended heavily on the training model and sampled data, we further proposed inverse gradient which makes inverse dual loss more robust by adjusting the gradient. We designed a meta-learning method with the training data split into training- train data and training-test data. The model was first pre-trained on the training-tme trring-trannnn data. Then the pre- pre-trained model would explore to update with the gradient or the additive inverse of gradient, even do not update, determined by the training- test data.
As for future work, we plan to apply our inverse learning with more recommendation models as the backbones to further verify the generalization of our proposed methods.
9

---

Under review as a conference paper at ICLR 2023
REFERENCES
Jessa Bekker, Pieter Robberechts, and Jesse Davis. Beyond the selected completely at random assumption for learning from positive and unlabeled data. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 71-85. Springer, 2019.
Jianxin Chang, Chen Gao, Yu Zheng, Yiqun Hui, Yanan NIu, Yang Song, Depeng Jin, and Yong Li. Sequential recommendation with graph neural networks. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 378-387, 2021.
Chong Chen, Min Zhang, Yiqun Liu, and Shaoping Ma. Social attentional memory network: Mod- eling aspect-and friend-level differences in recommendation. In Proceedings of the twelfth ACM international conference on web search and data mining, pp. 177-185, 2019.
Chong Chen, Min Zhang, Weizhi Ma, Yiqun Liu, and Shaoping Ma. Efficient non non- con-samplnng factor- ization machines for optimal context-aware recommendation. Proceedings of The Web Confer- ence 2020, pp. 2400-2410, 2020.
Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, et al. Wide & deep learning for recom- mender systems. Proceedings of the workshop on deep learning for recommender systems, pp. 7-10, 2016.
Weiyu Cheng, Yanyan Shen, and Linpeng Huang. Adaptive factorization network: Learning adaptive-order feature interactions. In Proceedingy of the AAAI Conference on Artificial Intel- ligence, volume 34, pp. 3609-3616, 2020.
Jingtao Ding, Yuhan Quan, Xiangnan He, Yong Li, and Depeng Jin. Reinforced negative sampling recommendation with exposure data. In Proceedings of the Twenty-Fighth International Joint Conference on Artificial Intelligence. IJCAI-19, pp. 2230-2236 International Joint Conferences on Artificial Intelligence Organization, 2019. doi: 10.24 10.24963/jjcai.21199300 URL https:
//doi. wr0/10...0.. .20...29999
Jingtao Ding, Yuhan Quan. Quamming Yao, Yong Li, and Depeng Jin. and robustify negative sampling for implicit collaborative filtering. In Advances in Neural Information Processing Systems, volume 33, pp. 1094-1105. Curran Asso- ciates, Inc., 2020. URL http::::///pooeeddngg.... heweeedddinns.uuuips.........e.s wwtrrisscc/paaaee cc/ppaper/222//11 6a5bd0 5bd- -
Simplify
Charles Elkan and Keith Noto. Learning classifiers from only positive and unlabeled data. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 213-220, 2008
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In International conference on machine learning, pp. 1126-1135. PMLR, 2017.
Steve Fox, Kuldeep Karnawat, Mark Mydland, Susan Dumais, and Thomas White. Evaluating implicit measures improve web search. ACM Transactions on Information Systems (TOIS), 23 (2):147-168, 2005.
Asela Gunawardana and Guy Shani. Evaluating recommender systems. In Francesco Ricci, Lior Rokach, and Bracha Shapira (eds.), Recommender Systems Handbook, pp. 265-308. Springer US, 2015. ISBN 978 1-489997637-6 doi: 1--0099778--4999-7666---- 8. URL https://doi. 4899 1188999777 7637-6_8
org/ /0/ 007 978 1
Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. Neural col- laborative filtering. In Proceedings of the 26th international conference on world wide web, pp- 173-182, 2017.
David W Hosmer Jr, Stanley Lemeshow, and Rodney X Sturdivant. Applied logistic regression, volume 398. John Wiley & Sons, 2013.
10

---

Under review as a conference paper at ICLR 2023
Dietmar Jannach, Lukas Lerche, and Markus Zanker. Recommending based implicit feedback. In Social Information Access, pp. 510-569. Springer, 2018
Youngho Kim, Ahmed Hassan, Ryen w White, and Imed Zitouni. Modeling dwell time to predict click-level satisfaction. Proceedings of the 7th ACM international conference on Web search and data mining, pp- 293-202,200 2014.
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2015.
Zhenguo Li, Fengwei Zhou, Fei Chen, and Hang Li. Meta-sgd: Learning to learn quickly for few- shot learning. arXiv preprint arXiv: 2017.
Jianxun Lian, Xiaohuan Zhou, Fuzheng Zhang, Zhongxia Chen, Xing Xie, and Guangzhong Sun. xdeepfm: Combining explicit and implicit feature interactions for recommender systems. In Proceedings of the 24th ACM SIGKDD international conference knowledge discovery & data mining, pp. 1754-1763, 2018.
Dawen Liang, Laurent Charlin, James McInerney, and David M Blei. Modeling exposure in recommendation. In Proceedings of the 25th international conference on World Wide Web, pp- 951-961, 2016.
Feng Liang, Weike Pan, and Zhong Ming. Fedrec++: Lossless federated recommendation with explicit feedback. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pp. 4224-4231, 2021.
Chao Liu, Ryen W White, and Susan Dumais. Understanding web browsing behaviors through weibull analysis of dwell time. In Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval, pp. 379-386, 2010a.
Jiahui Liu, Peter Dolan, and Elin Rønby Pedersen. Personalized news recommendation based on click behavior. In Proceedings of the 15th international conference on Intelligent user interfaces, pp. 31-40, 2010b.
Hongyu Lu, Min Zhang, and Shaoping Ma. Between clicks and satisfaction: Study on multi-phase user preferences and satisfaction for online news reading. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, pp. 435 444, 2018
Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. Bpr: Bayesian personalized ranking from implicit feedback. arXiv preprint arXiv:/205.2611 2012.
Francesco Ricci, Lior Rokach, and Bracha Shapira. Introduction to recommender systems hand- book. In Recommender systems handbook, pp. 1-35. Springer, 2011.
Yuta Saito, Suguru Yaginuma, Yuta Nishino, Hayato Sakata, and Kazuhide Nakata. Unbiased rec- ommender learning from missing-not-at- -oaa----o implicit feedback. pp. 501-509, 2020.
Wenjie Wang, Fuli Feng, Xiangnan He, Liqiang Nie, and Tat-Seng Chua. Denoising implicit Proceedings of the 14th ACM International Conference Web Search and Data Mining. ACM, mar 2021. doi: 10.1445/443793333441900. URL https: http:///oooo.gog/ wtt..//../.............. wet.11112133399555 01179963.3448880 3441800.
feedback for recommendation.
Hongyi Wen, Longqi Yang, and Deborah Estrin. Leveraging post-click feedback for content recom- mendations. Proceedings of the 13th ACM Conference on Recommender Systems, pp. 278-286, 2019.
Hong-Jian Xue, Xinyu Dai, Jianbing Zhang, Shujian Huang, and Jiajun Chen. Deep matrix fac- torization models for recommender systems. In IJCAI, volume 17, pp. 3203-3209 Melbourne, Australia, 2017.
Byoungju Yang, Sangkeun Lee, Sungchan Park, and Sang-go goo Lee. Exploiting various implicit feedback for collaborative filtering. In Proceedings of the 21st International Conference on World Wide Web, pp. 639, 2012.
11

---

Under review as a conference paper at ICLR 2023
Xing Yi, Liangjie Hong, Erheng Zhong, Nanthan Nan Liu, and Suju Rajan. Beyond clicks: dwell time for personalization. In Proceedings of the 8th ACM Conference on Recommender systems, pp. 113-120, 2014.
Weinan Zhang, Tianqi Chen, Jun Wang, and Yong Yu. Optimizing top-n collaborative filtering via dynamic negative item sampling. SIGIR '13, pp. 785-788, New York, NY, USA, 2013. Association for Computing Machinery. ISBN 9781450320344, doi: 20.1445248402824484122
URL https://doi...... 1tt.ooooo//.115524488000000 24444126.
Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma. Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. Deep interest network for click-through rate prediction. KDD, pp.
1059-1068, 2018
Guorui Zhou, Na Mou, Ying Fan, Qi Pi, Weijie Bian, Chang Zhou, Xiaoqiang Zhu, and Kun Gai. Deep interest evolution network for click-through rate prediction. In AAAI, pp. 5941-5948, 2019.
12

---

Under review as a conference paper at ICLR 2023
A APPENDIX
A.I PROOF OF THROREM 1
Theorem Given learning rate I. * O, assume the temporal model parameters updated by the direct gradient and inverse gradient, respectively, are as G = -0 - a a V V (0) and (0). Then, the relationship between the loss of them and the model with parameter 0 on data instance (u, i, Yui) E D' will be either LD (0t) > LD:(0) > L (0') or L (0¹) L (0) < LDD
0' = 0 + a o V V
Proof 1 To simplify the problem, from a stochastic perspective with one instance (u, i, Man). Our target is to satisfy:
((0") (()) L())),
(5)
or
((0") < L()) < L()'),
(6)
where [(0"), [(0), [()") are the loss functions for the models with parameters 0d, 0,0' To sim- plify the proof procedure, we define the prediction function as the hypothesis function of the well known logistic regression (Hosmer Jr et al., ;
f f(,, i 00)
Oz.,s²
where Fu is the input feature under the interaction between user and item i. Given learning rate R. + 0 with 0 0 o V VL (0) and O = 0 + V VLiual (8) as the temporal model parameters updated by the direct gradient and inverse gradient, respectively, we can have:
0 0 = Oz,,; - a a V L
(0) Lu,i
(8)
0 0'xu,i Oxn, + a o V VLD ( Du,i
(9)
Assume V Ldual (0) 0 then we have: 0'xx,,5 Ow.,i O' Review the loss function from a stochastic perspective with one instance (u, i, via):
(10)
((0) = - yuu log (b (1 Fue) log (1- - )
(11)
Suppose (u, i, Vuu) is positive instance with yor 1, then we can have:
((0) log (i
(12)
which is a decreasing function towards predicted probability Ph.
(()) will decrease towards ihe and further decrease towards Oxu,i. Base on the assumption of eqqutiqutton 10, we can have:
((0 < (()) < L(B')
(13)
which satisfies the target of Eqn.equation 6. Besides, similarly, vf = O. we can have: L(O L()) > L(O'). af o V Cduaal (.... > O, we can have similar conclusion. That is to say, whatever cases, targets of Eqn.equation 5 and Eqn.equation 6 will be satisfied.
The case a V C (0)x.. 0 will result in gradient vanishing and modern machine learning approach often randomly initialize the features to avoid such case.
A.2 ALGORITHM
The algorithm of our proposed inverse gradient is as Algorithm 1, where labeled data DD, unlabeled data D" and learning rate 7, a are required as input. The first iteration aims to pre-train the model using the split training-train data. The second iteration aims to explore the direct gradient and inverse gradient on the loss for sampled unlabeled data, where three strategies are explored here as line 13- 15 with gd 0 and f as the model parameters updated by the direct gradient, inverse gradient and without being updated by the gradient on the loss for sampled unlabeled data, respectively. Finally, the explored updated direction with minimal test loss the training-test data will be selected to update the model for this iteration.
httpps://
wwwpn//aahhhiicii ne- earnnng
13

---

Under review as a conference paper at ICLR 2023
Algorithm 1: Inverse Gradient Adaptation
input : Labeled data D', unlabeled data DD" learning rate 7,s a
output: 9
test Initialize 0, split D' into training-treee train data train (D') and training test data test (D');
while not done do T do
for t
Sample butch of truining labeled data train(D; ~ D': (iu.avi Jetrain (Dais Fur);
mmai (D) (0)
crain| (D)
0 0 7 V (
for to T' do
Sample batch of training unlabeled data train(Di) D4; Cduua
Σ (u.. w (furs 1) + w (D 0);
"caamn (2))
train (DF)
10
0 a
Eduat
"raainf (Dt)
(0):
11
0 Edeat
trrain pF) (0):
Sample batch of test labeled data test(D)) ~ D':
12
(pe) (D!) (0") reast (D!)
Σ I(uAidd) Etertt (pl e (Dat M
(p()( rest [i]]
Σ (....
(ini Vie
15
Lees (pt) (0) teee [P]]]
Σ (uris )(sen(p) (Bhes Whis):
0 arg min 0,00 (eest (0"), L(ee)(D)) (0) C Ceees (D:) (0))):
16
C sest p1) (0) m (0 (p) (0), C Cress (Dt) ( (0)}:
17
18
0 0 To V (e))s(
A.3 DATASETS AND PRE-PROCESSING
Table 2: Data statistics for processed Micro- Vidoo dataset and MLIM dataset.
Feedback
Dataset
Users
Items
Density
Positive
Negative
Total
Micro- Vidro 37,692
131,690
4,915,745
4.546,74 747
9,462.492
0.19%
ML1M
6,041
3,953
836,478
163,731
1,000,209
4.19%
The public MLIM dataset is published at? and we also have uploaded the processed dataset on the Github of the code and the supplementary material. The statistics of adopted Micro- Video dataset and ML M dataset are as Table 2. We will put the Micro-Videe dataset public to benefit the community.
Table 3: Interaction statistics for Micro Micro-Video dataset.
Dataset Like
Hate
Total Like/Total Hate/Total
Micro-Vide
382,570
6,157
8,008,965
4.78%
0.08%
Micro-Video This dataset is collected from one of the largest Micro-
Micro-Video Video platforms in China, where user behaviors such playing time, like and hate are recorded. The data is downsampled from September 11 to September 22, 2021. Users are passive to receive the recommended videos here, and there extremely limited active feedback such like and hate shown in Table 3. That is to say, we have extremely limited reliable feedback in this data which very challenging in modern industries. We take the active feedback of like and hate analyse the regular pattern of playing time and duration of each video which are included in each interaction. From Figure 6, we can discover that the users' like and hate behaviors are related to the finish rate of playing time (user's
https:: phttps://groupll htwp:/oppeees..oorttttttteeeeeee http:s///grrroog......oaaaaaaaa eliiee/eeeeeeee 1m/
14

---

Under review as a conference paper at ICLR 2023
(0033)
53000
30003
33000
Playing Time
Playing Time
30003
X00000
200000
230000
(X003)
1900E N00M 19300 acom 53066 60000
10006 S0c0n teaxe 90600 1oxe 9060)
1+5
Duration
Duration
Figure 6: Joint distribution of playing time and duration for like (left) and hate (right).
Table 4: Performance comparison of GMF on Micro- Video dataset based on feedback classification by finish rate and playing time.
metric
AUC
GAUC
Finish Rate
0.8024
0.7558
Playing Time
0.751
0.528
playing time towards certain item) to be divided by duration (item's 's total time): when the users like the video, they are more likely to finish watching it, and vice versa. Thus we treat the finish rate greater than 80% and less than 20% as positive and negative feedback, respectively. Another way to classify the positive and negative feedback is according to the playing time, and we also report the results by treating the playing time over upper quartile and under lower quartile positive and negative feedback, respectively. From Table 4, we can observe that finish rate is more suitable for pattern capturing and adopt such way as feedback processing in experimental evaluation.
MLIM
This is widely used public movie dataset in recommendation. The rating score in MLIM ranging from to 5 and we treat the rating score over and under 2 as positive and negative feedback, respectively, following DenoisingRec (Wang et al., 2021).
Besides, we splite 60%, 20% and 20% of the data training, validation and test data, for these two datasets.
A.4 IMPLEMENTATION DETAILS
All the models are implemented based Python with a Pytorch* framework based on the repository DenoisingRee*. The environment is as below.
Anaconda
python 3.7.3
pytorch 1.4.0
numpy 1.16.4
httte////gropleee.r////aatttttts 'https://pytorch.orrt
htttpp///githbbb..mmm//jeeeeeeee http
15

---

Under review as a conference paper at ICLR 2023
A.5 HYPER-PARAMERT PARAMETER STUDY
Impact of learning rate for convergence We vary the learning rate for inverse dual loss with 1, 10, 50 and 100 times as much as the learning rate for test loss, to investigate the convergence analysis at Section 2.3.2. From Figure 7, we can discover that the greater of a will truly result in gradient ascent, as there appears pass gradient when a 100y. Besides, the greater of a, the faster of convergence on training-test data but also overlitting.
GMF on Micro Video Dataset
GMF on Micro Video Dataset
direct gradient
0.60
direct gradient pass gradient inverse gradient
pass gradient
Loss
inverse gradient
Loss
0.40
0.40
0.20
2000
4000
6000
8000
2000
4000
6000
8000
Training Iteration
Training Iteration
(1) a ?
(2)
10y
GMF on Micro Video Dataset
GMF on Micro Video Dataset
0.60
direct gradient
0.60
direct gradient pass gradient inverse gradient
pass gradient
Loss
0.40
inverse gradient
Loss
0.40
0.20
0.20
0.00
0.00
1000
2000
3000
4000
1000
2000
3000
4000
Training Iteration
Training Iteration
(3) a = 50y
(4) a = 100y
Figure 7: The loss training-test data with different learning rate for adapted gradient.
A.6 COMPARISON WITH HARD NEGATIVE SAMPLING
Table 5: Performance comparisons towards hard negative sampling with GMF and NeuMF back- bones on two datasets. Bold and underline refer to the best and second best result, respectively
ML1M
Micro Video
Model
Method
NDCG
AUC
GAUC
NDCG
AUC
GAUC
None
0.9285
0.7671
0.6919
0.7365
0.8024
0.7558
NS
0.9400
0.7639
0.7200
0.7049
0.7802
0.7247
DNS
0.9178
0.6290
0.6290
0.6878
0.6310
0.6356
GMF
SRNS
0.9176
0.6253
0.6293
0.6799
0.6293
0.6302
IDL
0.8996
0.7304
0.6400
0.7272
0.7858
0.7335
IG
0.9521
0.8318
0.7642
0.7773
0.8033
0.7593
None
0.9214
0.7524
0.6856
0.6649
0.7504
0.6916
NS
0.9298
0.7495
0.7088
0.6350
0.7191
0.6689
DNS
0.9168
0.6277
0.6283
0.6203
0.6274
0.6261
NeuMF
SRNS
0.9172
0.6282
0.6284
0.6212
0.6296
0.6295
IDL
0.9212
0.7962
0.6984
0.6466
0.7174
0.6647
IG
0.9449
0.8253
0.7569
0.7809
0.8198
0.7689
The results of performance comparison with hard negative sampling are as Table 5, where we can discover that the hard negative sampling methods truly perform worse in setting where both
16

---

Under review as a conference paper at ICLR 2023
ground-truth positive and negative samples are tested. The typical and baselines of hard negative sampling are illustrated as below.
DNS (Zhang et al., 2013): Dynamic Negative Sampling (DNS) samples unlabeled instances as negatives and picks the hard instances with high predicted score.
SRNS (Ding et al., 2020): SRNS further selects true negatives with high-variance to improve DNS.
17


Reviews:
[{"review_id": "rTJ8O5YmB2", "reviewer": "Reviewer_6jCU", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper studies the de-noising recommendation problem. The authors propose a meta learning method to annotate the unlabelled data from both the loss and gradient perspective. They propose the inverse dual loss to boost the true label learning and prevent false label learning. Moreover, they also propose the inverse gradient to explore the correct updating gradient. To demonstrate the effectiveness of the proposed method, the authors perform extensive experiments on two real datasets. ", "summary_of_the_review": "Overall, this paper provides some novel ideas to solve the drawbacks of existing negative sampling strategies widely used in existing recommendation research. Some claims in this paper are not supported by experiments. Moreover, the experimental analysis is not sufficient. The authors only study the effectiveness of the proposed method on the Logloss. The quality of this work can be improved by studying the performance of the proposed method with other recommendation functions, e.g., BPR loss. In addition, the backbone models are a little weak. The authors need to consider other advanced recommendation models, e.g., LightGCN or other sequential recommendation models, to demonstrate the effectiveness of the proposed method.", "strength_and_weaknesses": "Strengths:\n1.\tThe authors propose to perform denoising from both positive and negative perspectives in recommendation. \n2.\tThe authors propose two new methods, i.e., inverse dual loss and inverse gradient, to improve the model performance.\n3.\tThe authors perform extensive experiments on two real datasets to demonstrate the effectiveness of the proposed method.\n\nWeaknesses:\n1.\tThe proposed method is only studied with the logloss. It is not clear whether the proposed method can help other ranking-based recommendation loss, e.g., Bayesian personalized ranking loss. \n2.\tThe backbone models, i.e., GMF and NeuMF, are a little weak. The authors are recommended to consider other advanced recommendation models as backbone models, e.g., LightGCN and other sequential recommendation models.\n3.\tSome claims in this paper are not supported by experimental analysis. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "dDpvl2x4g8z", "reviewer": "Reviewer_16D4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "To tackle the noisy negative sampling from the unlabeled data, the paper proposes an inverse dual loss (IDL) to assign larger weights to the more confident loss (either negative loss or positive loss) on the unlabeled data to achieve automatic labeling. To further improve learning robustness, it proposes a meta-learning procedure to determine whether a direct gradient, an inverse gradient (IG), or a pass gradient of the inverse dual loss will be applied based on the meta-test performance. Experiments on two datasets demonstrate the method's superiority over naive negative sampling and a denoising method that only focuses on denoising the labeled data. The effectiveness of IDL to classify the unlabeled data and the effectiveness of IG to assist convergence are also analyzed in the experiments.", "summary_of_the_review": "Since some important claims made in the paper are ambiguous/not well-supported and the experiments lack evaluation in an important aspect (i.e., implicit setting), I feel that the paper is marginally below the acceptance threshold.", "strength_and_weaknesses": "Strengths\n- The paper focuses on solving an important and challenging issue of noisy negative sampling in recommender systems.\n- The idea of employing the inverse dual loss to annotate the unlabeled data and applying a meta-learning framework to determine the update gradient for more robust learning is novel.\n- A processed Micro-Video dataset is released to the public, where labeling has been done based on the playing time of videos. This I believe will become a good contribution to the community.\n- The experiments to analyze the effectiveness of IDL and IG are helpful.\n\nWeaknesses (see the concerns below)\n- Some claims in the paper are not well-supported.\n- Several parts of the paper lack clarity, which affects the understanding of some important ideas.\n- Experiments can be further improved to include the implicit feedback setting.\n\nMajor Concerns\n1. In section 2.3.1, the authors claim that \"when the sampled data is hard, exploiting the inverse gradient to update the model will gain a smaller test loss on the training-test data, and thus we exploit inverse gradient here\". However, I find that the link between the hard instances and the inverse gradient is not clear, and hence the claim that the inverse gradient helps improve learning with the hard instances is not supported.\\\nBased on the descriptions in the previous sections (and illustrations in Fig. 1), hard instances are points that are close to the decision boundary (and easy instances are those that are far from the decision boundary). This means that for these hard unlabeled instances, the negative loss and positive loss values are close to each other (as the classifier has a hard time determining labels for them), which results in similar weights for the negative and positive loss in the inverse dual loss in eq. 4. Hence, for hard instances, the resulting effect on learning is just that the model will hardly change as both the positive and negative loss are optimized equally. I don't see why in this case, employing an inverse gradient of the inverse dual loss will help achieve better learning and result in \"a smaller test loss on the training-test data\" (similarly for the case of easy instances, why employing direct gradient for them is better). The authors may want to provide more explanations on this to support the claim.\n2.  The experiments in this paper analyze two scenarios: passive feed recommendations (Micro-Video) and explicit user rating (ML1M). For both cases, the authors pre-processed the datasets to have binary labeling where the labeled user-item pairs can be either positive or negative. However, for the implicit feedback setting (i.e., users provide interactions like clicks and purchases as feedback), all the labeled/interacted user-item pairs are considered positive. For this case, the test loss in the meta-learning procedure will only have positive data points. How will this affect the annotation quality of the inverse dual loss for the unlabeled/uninteracted data? Since the implicit setting is quite a prevalent scenario in the industry, I suggest the authors incorporate experiments on the implicit setting (e.g., using Yelp, Amazon datasets) to make the evaluation of the proposed method more comprehensive.\n\nMinor Concerns\n1. In Definition 1, it is not clear why the normalization parameter z_w is formulated in this way. Can the authors provide some intuitions on this?\n2. In the summary of the main contributions, the authors claim that they are \"the first to perform denoising from both positive and negative perspectives in recommendation\". According to my understanding, the paper only proposes a method to denoise the negative sampling (avoid false-negative) from the unlabeled data. Can the authors explain why they claim that their method also serves to perform denoising from a positive perspective? \n3. There is this recent work [1] that aims to tackle noises in both positive and negative implicit feedback. The authors may consider including a comparison with this work (adding it as a baseline in experiments or at least discussing it in the related work).\n\nParts that need further clarification/adjustment\n1. Based on the descriptions in \"Baselines and Backbones\" and section 3.1, it is not clear whether IG is adding on top of IDL or the two methods are applied separately.\n2. In the second paragraph of section 3.1, what does it mean by \"IDL only outperforms the NeuMF\" and \"it is even outperformed by GMF\", as both NeuMF and GMF are just the backbones but not the methods to compare? The authors may want to rephrase this part.\n3. In section 3.3, the authors state something about the effects of pre-training on the training-train data. However, experiments in this section all involve pre-training, and a set of experiments without pre-training is required to make comparisons and support such claims.\n4. In Fig. 5, for the two plots on NeuMF, the legend boxes are too large and block out some important parts of the learning curves.\n\n[1] Denoising Neural Network for News Recommendation with Positive and Negative Implicit Feedback, NAACL 2022\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "HNL39grQh8K", "reviewer": "Reviewer_a7qD", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper introduces a heuristic method to better exploit unlabeled data in recommender system in order to boost performance and also avoid too strong introduced noise. The method has components such as a re-weighted loss based on current model prediction and an explorative gradient selection logic. The performance gain demonstrates the necessarily of the gradient selection logic.  ", "summary_of_the_review": "The author should also benchmark with semi-supervised learning in recommender system since both trying to exploit (assign a label) to large volume unlabeled data.\n\nWhen model is not well trained (such as in early stage training), there will be a negative feedback loop to use such inverse loss and inverse gradient. The method is too heuristic to cope with such edge cases.\n\nFlaws:\nUsually the predicted label is between 0 and 1 and the loss in Eq one is always positive. In Eq 4, the absolute symbol doesn't make sense.\n\nThe inverse gradient is too ad hoc. For example, in the proof, we see a very much simplified case, this is not sufficient proof for a theorem. It's very hard to define easy example, hard example and guarantee the gradient triage to work.\n\nTypo:\nin abstract line 3, it should be \"often introduces false-negative noises\", instead of \"often introduces false-positive noises\"?", "strength_and_weaknesses": "Strength \nThe paper is clear to follow. The motivation and intuition is clear and valid.\n\nWeakness\nThere is no rigorous theoretical guarantee. There is no data point in a complicated experimental set up such as industrial level production A/B testing results.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject