Curriculum Disentangled Recommendation with Noisy Multi-feedback
Authors: Hong Chen
Yudong Chen
Xin Wang
Ruobing Xie
Rui Wang
Feng Xia
Wenwu Zhu
Venue: NeurIPS 2021 Poster (2021)
Keywords: Recommender system
Disentangled representation learning
Curriculum learning
Multi-feedback

Abstract: Learning disentangled representations for user intentions from multi-feedback (i.e., positive and negative feedback) can enhance the accuracy and explainability of recommendation algorithms. However, learning such disentangled representations from multi-feedback data is challenging because  i) multi-feedback is complex: there exist complex relations among different types of feedback (e.g., click, unclick, and dislike, etc) as well as various user intentions, and ii) multi-feedback is noisy: there exists noisy (useless) information both in features and labels, which may deteriorate the recommendation performance.  Existing works on disentangled representation learning only focus on positive feedback, failing to handle the complex relations and noise hidden in multi-feedback data. To solve this problem, in this work we propose a Curriculum Disentangled Recommendation (CDR) model that is capable of efficiently learning disentangled representations from complex and noisy multi-feedback for better recommendation. Concretely, we design a co-filtering dynamic routing mechanism that simultaneously captures the complex relations among different behavioral feedback and user intentions as well as denoise the representations in the feature level. We then present an adjustable self-evaluating curriculum that is able to evaluate sample difficulties for better model training and conduct denoising in the label level via disregarding useless information. Our extensive experiments on several real-world datasets demonstrate that the proposed CDR model can significantly outperform several state-of-the-art methods in terms of recommendation accuracy.

Curriculum Disentangled Recommendation with Noisy Multi-feedback
Hong Chen'; Yudong Chen Xin Wang
Ruobing Xie² Rui Wang', Feng Xia² Wenwu Zhu
TTsinguua University, WWehht Search Application Department, Tencent.
(h-chen20, cyd cyy.000y11111mm1maa1 Amaails. tsinghua. edu. fruobingxie, (rabbingxieryssawarrr.. xiaffnggxia)@teecett.co
edu. en, {xin_wwarrr wang, w edu. cn,
Abstract
Learning disentangled representations for user intentions from multi-feedback (i.e., positive and negative feedback) can enhance the accuracy and explainability of recommendation algorithms. However, learning such disentangled representations from multi-feedbaed data is challenging because i) multi-feedbacd is complex: there exist complex relations among different types feedback (e.g., click, unclick, and dislike, etc) well various intentions, and ii) nalti-feedback noisy: there exists noisy (useless) information both in features and labels, which may deteriorate the recommendation performance. Existing disentangled recommenda- tion works only focus on positive feedback, failing to handle the complex relations and noise hidden in multi-feedback data. To solve this problem, in this work we propose a Curriculum Disentangled Recommendation (CDR) model that is capa- ble of efficiently learning disentangled representations from complex and noisy multi-feedbaat for better recommendation. Concretely, we design co-filtering dynamic routing mechanism which simultaneously captures the complex relations among different behavioral feedback and user intentions as well as denoise the representations in the feature level. We then present an adjustable self-evaluating curriculum that is able to evaluate sample difficulties for better model training and conduct denoising in the label level disregarding useless information. Our extensive experiments several real-world datasets demonstrate that the proposed CDR model can significantly outperform several state-of-the-art methods terms of recommendation accuracy
Introduction
Recommenders aim to capture the user's preferences from different aspects of information for more accurate prediction 1 Learning disentangled representations that uncover and disentangle the latent explanatory factors hidden in user behavioral data has recently been shown as an effective way discover users' intentions, improving both recommendation accuracy and explainability (5/0).
Multi-feedback, normally including the positive feedback (e.g., click) and negative feedback (unclick and dislike, etc.), is of great significance depict the user's unbiased and various intentions [10. Learning disentangled representation from multi- feedbickeebbckk is able to capture various user intentions more accurately, leading to improvement of accuracy and explainability in recommendation.
However, learning such disentangled representation from multi-fcedback that can best serve recom- mendation is quite challenging due to reasons. Multi-feedback is complex: different types of
'Equal contributions
i Corresppnding Authors. 'Our code will be released at attttpss//gittu...o//ooottteeeee
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

feedback in multi-feedback and various user intentions are also complicated. For instance, a user may unclick an item because of disinclination, getting tired of seeing the same type of item previously clicked too many times, as well as lacking enough time to view the item in detail. ii) Multi-feedback is noisy: the large amount of feedback such as unclick brings lot of noisy information (useless for recommendation) that may severely deteriorate the recommendation accuracy. For example, a user may unclick an item because he truly dislikes this item or he has interests in the item but is interrupted by others before making the click decision. As such, completely incorporating the unclick feedback as historical features to extract user intention representations may run the risk of bringing feature-level noise, while directly regarding the unclick behavior as negative samples to train the model will probably introduce label-Ievel noise. Existing works disentangled representation learning merely rely on the positive user feedback to extract the user intentions, failing to handle the complex relations and noisy (useless) information hidden in multi- data.
data have complex relations with each other, and the relations between
To tackle these challenges, we propose a Curriculum Disentangled Recommendation (CDR) model that is able to accurately discover various user intentions from different kinds of user feedback. The CDR model consists of two core components, i.e., the co-filtering dynamic routing mechanism and the adjustable self-evaluating curriculum strategy, which together learn disentangled representations as well as captures the complex relational dependencies and filters out noise from neeti--feedback to achieve more accurate recommendation. More concretely, the proposed routing mechanism utilizes the dependencies among different feedback to accurately discover the user preferences, followed by the intention aggregation which further helps to learn disentangled representations for users. Our adjustable self-evaluating curriculum then guides the model towards better optima by reweighing samples according the self-evaluated difficulty. The proposed curriculum can further enhance the model performance by making the model learn from multi- neeti--feedbak data of different difficulties at different learning paces, thus preventing the model from overfitting the data with improper labels. We conduct extensive experiments on several real-world datasets to demonstrate that our proposed CDR model can significantly beat baseline methods in terms of recommendation accuracy.
To summarize, our contributions are listed as follows. (1) We propose a Curriculum Disentangled Recommendation (CDR) model to learn disentangled representations for recommendation from meetii-feedbac with complex relations and noisy information. (2) We propose a co- co-filtering dynamic routing mechanism simultaneously capture complex relations among different sorts of feedhack and various user intentions as well as filtering out feature-leve noise in an adjustable self-evaluating curriculum to guide the model towards better optima by alleviating the impact of label-level noise in more controllable and convenient way.
(3) We propose
2 Related Work
Recommendation Based on Multi-Feedback Apart from positive feedback, negative feedback data are also essential for capturing user preference [I]] Early efforts [12H14] regard all unclick data equally as negative feedback and decrease the confidence compared click data. Later works utilize additional information [1518] or reinforcement learning [19 to distinguish real negative signals
from all the unclick data. A recent method [22] also recognizes negative feedback from the clicked news by reading dwell time. However, all these methods merely select better negative training samples and ignore multi-feedback as features to learn user interests. DFN [00 simultaneously incorporate click, unclick, and dislike feedback simply by rough attention mechanism and concatenation, which is insufficient to capture the user's comprehensive and accurate interests.
Disentangled Representation Learning Disentangled representation learning aims to learn various hidden explanatory factors behind observable data in different parts of the learned vector presenta- tion 23 Many variants of variational auto- encoderr (VAE) [2] have been studied to improve the disentanglement of the learned representation 25 27 by adding regularization terms to decrease the mutual information between different parts of the learned vector. Disentangled representation
learning has also found its application in recommendation SLS 28 29 by learning disentangled user preference from user positive feedback to improve both the performance and interpretability. Different from these works, our work focuses on learning more comprehensive disentangled interests with nelti-.fee
2

---

Curriculum Learning Curriculum learning [30, 31 aims to design a dynamic sample reweighting strategy throughout the training process improve performance and training efficiency. Traditional methods mostly follow an easy-to-hard paradigm, i.e., assigning higher weights to easier samples in earlier training, where the "easiness" measurement can be both Komain-knowledge-basse
[32 34 and loss-based [35] [36]. These methods can only be effective for specific tasks, and "easy- to-hard" sometimes sub-optimal compared with "harder first" [37]. Recent works propose to automatically learn a curriculum by reinforcement learning (38/40), meta-learning [4]], etc. However, the optimization process of these curricula is time- or resource-consuming which is too costly for abundant recommendation data. Our method provides an efficient and also flexible curriculum that is beyond the easy-to-hard limit and more adjustable and interpretable than automatic methods.
3 Method
In this section, we introduce the proposed Curriculum Disentangled Recommendation (CDR) model (Figure I mine comprehensive and accurate user intentions from users' noisy multi-feedback.
3.1 Notations and Problem Formulation
Notations We denote c(a, b) as the inner product of two vectors Layer N (orm(a) and LoyerNormmb) where Layer Norm refers to Layer Normalization operation and Φ(a, b) measures the similarity between vector and b. sim(key, query) is the normalized similarity between query and key, on set Q that is composed of all the keys: sim(key,,
expl (kkey, .qury))
= Eveg explo(key, qqerry))
Multi-feedback based prediction The wtth user's historical behavior contains his or her clicked item sequence cl*) (e(") cl"), ..... ,.,,,,)) unclicked (i.e., presented but not clicked) item sequence (0)
(()) = (u(") uu2 " un (1)) and disliked (i.e., press to the dislike button or low rated) item se-
quence d(x) (d oh .... .d/") where each term in these sequences represents an item that the ((h)) user interacted with. Our goal to learn users' disentangled intentions and then accurately predict Sut, i.e., the th user's preference towards the candidate item ((()), Besides, utilize the profile (c.g., age and gender) of the th user to aid the learning and prediction process.
3.2 Co-filterng Dynamic Routing
The routing mechanism takes as input user profile, candidate item feature, and user multi-feedback history, and makes the final prediction by the following three steps. It first utilizes the relations behind different kinds of feedback to discover where the user's true interests locate. Then, the model aggregates the user's disentangled intentions from the useful behavior. Finally, it predicts the user's preference towards the candidate item based on the learned intentions.
3.2.1 Interests Mining
The user's interests could be reflected by his or her various kinds of feedback. Ignoring any kind of feedback may lead incomplete or inaccurate preference modeling. However, the noise hidden behind these data makes it infeasible to directly use multi-feedbacc to learn the user's interests. Thus, how make use of the relations of different kinds of feedback discover where the user's true interests locate is the key in this step.
First, we project each item in c(*), d(t), and (()) into the embedding space by concatenating its and category embedding. Thus, we obtain the clicked embedding sequence h." (1) = [ h h h()))
unclicked embedding sequence h(.') [1(1) h . ... h(]] and disliked embedding sequence
h
(0) = [h h (o)
(o)
" We also project the information of the user profile into the embedding
space and obtain the user profile feature F(²) = (F("), F(") F(e) where g is the number of
user profiles. We then utilize the power of transformer encoder 42 to obtain better representation for
each item with fully interactions:
(n))) (v)
22
(w)] C- FF-eodeerrh (()))) (u 7 Fun =
(v)
(v) (v) U- U-Encoder(t!" (x) ), [zan 2,2 .... -z dimensional feature vectors containing the th user's 's interests.
(v) D (-Enodder(hh") (v) where the z 2u
(w) (v) refer to d-
3

---

Interests Mining
Intention Aggregation
Prediction
prototype
Tinek Candidate
clicked
items
Enorder
Awareness
simalarity bmaskk with
Filtert
filerr
disliked items
DeFreaderr
negaltive intentiinn
Filerr
pesull
4m
Filte33
discenlanglod
unclicked Hiems
L-Fraaddr
Awertenes
Candidale
intentions
Intemaatin
Fasture
ACLP
candiditte
item
user profile
Adjustable Self-evaluating Curriculum
Co-filtering Dynamic Routing
data batch
predicted labels
weighted losses
Difficulty thre
1.0
Adjustable
sample
hyperparameters
Smoool
learning speed:
0.4 weightt
curriculum C:
evaluued difficulty
52 na
200
number of tmining batches
500
1000
[thre,
Figure 1: The framework of the proposed Curriculum Disentangled Recommendation (CDR) model
We then need deeper thinking towards the relations among multiple feedback to discover user's true interests. The first thing to consider is where the user's interests will locate. We note that not only the click behavior contains user interests, but the unclick sequence generated by the recommender algorithms could also contain rich information about user interests. Therefore, utilizing both kinds of feedback could help to learn comprehensive user interests. The second noteworthy thing is how to discover the user's true interests from the noisy data. To filter out the noise information in these historical sequences, we need the disliked item sequences that could reflect the user's strong negative intentions and have high confidence. Thus, average these strong negative features and regard it as the negative intention of the th user: n (v) = Σ'= Z/ This highly confident negative intention could be used to filter the noise in both clicked and unclicked history items. We adopt the similarity between each item and the negative intention to judge whether the item should contribute to the user intentions. Based on the intuition that higher similarity to the negative intention should contribute less to the user preference the contribution of each item to the user intentions can be formulated as follows:
(1) (0) Dal. Zei Za
(()) (c)
on (Waz + b,2).
(()) expl (z n. ((e)))
5- cepp $62.0 (1)
n (")))
n (v) (0) bui, Zuy
1(0)
+ + ( (v) + +bw)..
= Luj
((0) = =-
(()) n. ((0)) MLP(F (2)).
where o, is the ReLU activation function and is the Sigmoid activation function. de and (") represent the contribution each clicked item and unclicked item to the user intention, respectively. We obtain g(t) by calculating the similarity between each clicked item and the negative intention, and we also introduce the parameters Wa. Det, b to avoid some outliers that disobey our intuition. As for the unclicked item sequence, there exists more noise. Thus, we utilize not only the information of the user negative intention but also the user profile feature F(0) to measure the importance of each

---

unclicked item. We first concatenate user profile features and the unclicked item feature together, and then use two-layer MLP to judge whether the unclicked item is important.
Besides the noise effect, we also take the time and candidate item factors into consideration. Intuitively, more recently clicked items and items having higher similarity to the candidate item will contribute more to the user preference towards the candidate item. The formulation is as follows:
time (v) (a + (Wea) (Walzaa P.] bes), time ((r) cand
z.m to (Wes ( Pm. bot),
cm
La + (Wi Wo (1) bes). Zei mn Deb:
(e)
fei sim (time (v) time (()) + simu (cand (0) oi
((1)).
cand Zu3 + a1 wW bus), Zue (n bus, (ug sand uj Zuc 20))
We consider the impact of both time and candidate factors for the clicked items. The candidate item feature h²dh) Rd and the most recently clicked item feature Z measures the importance of each clicked item. P; is the position embedding of each item and encoded for more accurate time factor consideration. However, for the unclick sequence, we only consider the candidate factor, because the time of unclicking one item has low relations with the user's s current interests. f(()) and fuz are the importance of each clicked item and unclicked item, respectively.
3.2.2 Intention Aggregation
After considering how each kind of feedback will contribute to the user's 's intentions, we then conduct intention aggregation obtain the th user intentions under various latent categories. Assuming that the intentions of all users can be decomposed to K latent categories and each latent category has its prototype ma E IRd, k 1,2,-.. K, can predict the probability of each item belonging the h latent category by their similarities:
sim (m U3A sim mg Zuj
We then the highly confident negative intention to filter the click and unclick historical item features by residual structure. The tt intention of the th user could be aggregated from all the filtered clicked and unclicked item features, (1(0)
(v) and (1(T) Eq. GB. is the bias for the g
latent category. À < is the prior confidence of the unclicked item. inten under the Rth latent category.
(1) is the 4tth user's interest
'(0), z1 2/ MLP((E n 2 n. 2 n. ())) (v), (0), (v)
(0) n (w) 1
z1 (w) ug
= uu + +LPP(
(v)
uj n. Zus
④ n. ()
inten (e) ( plo) c (1 (V) +
(1) uy
p(v)
(0)
-x Bx).
3.2.3 Prediction
Based on the learned intentions, can predict the user's preference towards the candidate item by calculating the inner product between the candidate item and user disentangled intentions. However, there also exist some users who might have little historical behaviors. So we also manage to discover the interests of the th user from his or her profile feature and the candidate item feature, aiming to capture some common patterns, e.g., females may show high interests in fashion articles. We collect
(1) these information in Fau [F()) ..... F(1) h .... b,"),, where h 2j
R & is one field of the
candidate item embedding h." (v) Then the multi-head attention of different fields of features:
is used to capture the interactions
(5) By simultaneously considering the user's s interests learned from the historical multi-feedback and feature interaction, we can predict the user preference towards the candidate item 8ut as Eq. (6). where the first term is the preference inferred from user feedback (<> is the inner product) and the second term is from the feature interaction between the user profile and item features.
f2. fote] M MultiheaddAtteeniinnff..
inten (e) .15 > +MLP([f1; : :fotz)).
(6)
5

---

Our loss function is designed as follows, composing of the widely adopted binary cross coos-py loss and the regularization term for disentanglement aiming to minimize the similarities among different intentions. Yot is the ground truth label, and Dr is the training set.
L =
Yue log (8ve) + (1 - Vee) log(1 (80c)) + (u,))E
(m, m,)
Drl
3.3 Adjustable Self-evaluating Curriculum towards A Better Self
All the aforementioned methods could help to capture the user's comprehensive true interests from the input features. However, the noise existing in the training labels will also mislead our model to suboptimal parameters. To tackle the problem, we leverage the idea of curriculum learning to denoise [3I]. However, it is proved in (36) that different curriculum strategies (e.g., easy to hard, hard example mining, etc.) can be effective for different dataset settings, and thus we need a flexible curriculum design to adapt to complex recommendation scenarios.
To this end, we propose a novel adjustable self-evaluating curriculum for recommendation, which algorithm is shown in Algorithm 1. Our goal is to obtain the optimal parameters I for recom- mender F. When training on the batch B, we first calculate the prediction result (ss)) )) for each sample and the difference between the ground truth label yj and our prediction. Then, we obtain the importance of each sample through a Gaussian distribution, where the sample with a closer difference to preset value thre will get higher importance. Here, the hyperparameter thre E (0, 1) reflects the sample difficulty we hope the model to focus on. Concretely, if thre approaches I, the harder samples will get higher weights. While if thre approaches O, the model will lay more emphasis on the easier samples. In our experiment, we can adjust the values for thre to set the curriculum of different difficulty levels for our model to improve itself. Finally, we optimize the parameters by the reweighted loss with an existing optimization method æ. In the algorithm, μ controls the degree of concentration of the model and becoming smaller and smaller by timing т, which makes the Gaussian distribution smoother and smoother so that all the samples almost get equal weights for training in the end. This time-varying process conforms to the human learning process: after we gradually improve ourselves by scheduled curriculum, we should review all the samples to further consolidate the knowledge in our mind. Since in the early stage the model has already learned enough knowledge, it could be more robust and less likely to be influenced by the noisy samples. By adjusting the values of T, we can control the curriculum learning speed, where a larger means slower learning process.
time- tame-weight-de weight-decay factor. As the training goes on, il is
Algorithm 1 Adjustable Self-evaluating Curriculum towards A Better Self
1: input: ((ret Vox) (w.))ED * ((),(;))) < 1, counter, interval, thre,
2: initialize: f,P.
3: for B Dr do
4: for (Iz. E B do
5:
8, F(I,), dj = abs(y) - ((5)));
6:
wy = exp( x (dg thre)?);
7:
Wj w// 1 (wy):
8:
end for
0- (vo 3 1 (1 ΣK
9:
10:
counter +- counter + 1; if counter conntre/intervalll
then
12:
+ p 1
13:
end if
14:
end for
Discussion. Our proposed curriculum elegantly converts the designing process of a curriculum training strategy to controllability, and explainability of curriculum design, while goes beyond the conventional assump- tion of "easy-to-hard" [30 35 32 33 Meanwhile, compared to automatic curricula [38
search process on thre, μ and r, which improves the flexibility,
40 43 39)
our method requires almost no extra time and memory overhead for learning and applying the curriculum, which is efficient and conforms to the demand of recommendation scenarios.

---

Experiments
4.1 Experimental Setup
Table 1: Dataset statistics
Amazon-Beant Beauty
Amazon-Sports
MovieLens- IM
WeChat5D
of users
22,342
35,590
6,039
13,340
# of items
12,099
18,356
3,628
112,859
of click
176,520
277,088
836,478
749,138
# of unclick
788,008
1,179,266
2,138,040
7,766,013
# of dislike
21,847
19,203
163,515
295,504
Datasets We conduct our experiments on four real-world datasets: WeChat5D, MovieLens- 1M 44 Amazon Sports[45] and Amazon Beauty 455 The datasets statistics are shown in Table ! WeChat5D is a mobile article recommendation dataset of WeChat Top Stories and itself has different kinds of user feedback. All data are preprocessed via data masking to protect user privacy. For the Amazon and Moviel dataset, we regard the ratings that are larger than 2 points as click feedback, while the rest as the dislike feedback. Since these three datasets have no information about the items that are recommended to the users but are unclicked, we simulate simple recommendation environment. For each piece of user like or dislike interaction, we generate four pieces of unclick interactions for this user. They include three items that are sampled from the top popular items at that time and one item randomly sampled from all the items. This simulates the simplest recommendation rule, i.e., recommendation based on item popularity. The randomly sampled item simulates the scenario that the recommenders would always recommend something to explore the customer's ' potential interests [46). Note that all the user's 's unclick interactions are not in the user clicked or disliked item sets. More specifically, based on the two Amazon datasets and the MovieLens- 1M dataset, for each piece of user like or dislike interaction, include three items that are sampled from the top popular (top 3000 for Amazon datasets and top 1/2 for MovieLens- IM) items that time and one item randomly sampled from all the items. The whole dataset is chronologically divided to the train, valid, and test dataset by the ratio of 8:1:1. Note our training and testing phase follow the sequential recommendation setting. For example, if one user's his- torical behavior is sequence {1, 2, 3, ... 18, 19, 20). Then, we will generate the validation and test samples as follows: two validation samples {[1, 2, 3, 4, ... 16) [17]]. (11, 2, 4,
generate four pieces of unclick interactions for this user. They
16, 17], [18]] and two test samples {[1, 2, 3, 4.... 16, 17, 18), [19] }. [[1, 2, 3, 4... 16, 17, 18, 19). 20]]. where the first term in represents the historical information we use for prediction and the second term is the next item for prediction. We always use all the user's 's real historical behaviors as the sequential input to the models to predict the next item the user will click.
Baselines We compare our approach with the state-of-the-an (SOTA) methods. DeepFM 47 and AutoInt 48 are recommenders based on feature interaction. DIN 49) SASRec [50], DFN [10),
SDR 9 are methods based on the user's sequential historical behavior. Specifically, DFN considers different kinds of user feedback and concatenates these features together. SDR only utilizes positive click feedback to capture the user's disentangled interests. For fair comparison, we add the feature interaction module to all the baselines if the model originally does not utilize the user profile.
Implementation and hyper- parameters We implement our method in Tensorflow and use the Adagrad [5I) optimizer for mini-batch gradient descent that is suitable for sparse data, while the size of each mitc-batch is 256. All the mentioned transformer encoders are four-head and lne-layer. We cap the maximum sequential historical behavior length to 30 for all datasets. We fix μ in the curriculum to 10 and the other hyper-parament parameters are then tuned using random search. The search space listed follows. More detailed experimental settings can be found in our appendix.
The number of latent intentions K E (1.2...... ,8].
The prior confidence for the unclicked data À E (0.1, ,103.
The learning rate E {0.0001, 0.001, 0.1, 1.0).
The hidden size of each field of feature e {32, 64, 128, 256}
7

---

Table 2: Model performance
Dataset
Model
Amazon Amazon-Beauty
Amazon-Sponn Sports
MovieLens-Ihe IM
WeChat5D
AUC
Relalmpr
AUC
Relalmpr
AUC
Relalmpr
AUC
Relalmpr
DeepFM
0.6975
0.00%
0.7608
0.00%
0.8098
0.00%
0.7219
0.00%
AutoInt
0.6826
-7.57%
0.7575
-1.27%
0.7940
-5.08%
0.7263
1.96%
SASRee
0.7415
22.28%
0.7830
8.52%
0.8826
19.32%
0.7126
4.18%
DIN
0.7633
33.32%
0.7968
13.80%
0.8760
21.39%
0.7282
2.83%
DFN
0.7670
35.21%
0.7763
5.93%
0.8293
6.31%
0.7329
4.96%
SDR
0.7238
13.30%
0.7972
13.95%
0.8865
24.78%
0.7296
3.455
CDR (Ours)
0.7991
51.43%
0.8152
20.85%
0.9065
31.24%
0.7622
18.14%
4.2
Recommendation Performance
We evaluate the performance of proposed method on the classical click-through-rate (CTR) prediction task and utilize widely-used metric Area Under Curve (AUC) for evaluation. We also follow 2 to the Relalmpr to measure the relative improvements over the base model (ie.,DeepPMM in our experiment). The results are shown in Table
We observe that our approach outperforms the baselines significantly, both the dense Movielens dataset and the sparse Amazon and WeChat5D dataset. We can see that on the Moviel.ens dataset where each user has rich historical behaviors, the models that utilize the sequential historical hehavior obtain much better performance. However, DFN fails to accurately capture the user's interests compared to other sequential models in MovieLens- IM. This is likely because DFN uses three kinds of feedback and this dataset, the user has rich feedback that contains more noise (introduced by more unclick behavior), while DFN fails to filter the noise. Our method performs best datasets that have different sparsity, mainly benefiting from both our model design and the curriculum training strategy, which is discussed in Section 4.3 and Section 4.4 respectively.
4.3 Multi-feedback
We validate the effectiveness of our method on capturing the user's preference from different kinds of feedback. We compare the following four situations, our complete method (complete), our model without curriculum (w/o CL), our model that only utilizes the clicked historical feedback without curriculum (click), and our model that directly utilizes the clicked and unclicked historical feedback(i.e, the output of the C-Encoder and U-Encoder) to aggregate the user's intentions without curriculum (c&un). The result is shown in Table 3
Table 3: Effectiveness of our model components
Dataset
Model
Amazon-Beamn
Amazon-Sport
Moviel Lens-MM 1M
WeChat51)
AUC
RelaImpr
AUC
Relalmpr
AUC
RelaImpr
AUC
Relalmpr
w/o CL
0.7914
47.56%
0.8083
18.22%
0.8972
28.22%
0.7548
14.82%
click
0.7477
25.43%
0.8006
15.26%
0,8879
25.22%
0.7347
5.76%
c&cun 0.7477
25.42%
0.7969
13.86%
0.8854
24.40%
0.7340
5,42%
complete 0.7991
51.43%
0.8152
20.85%
0.9065
31.24%
0.7622
18.14%
By comparing the click and c&un results, conclude that directly utilizing the unclick historical behavior will not improve the model performance. A possible reason that noise unclick feedback will prevent the model from capturing the user's true interests. However, with our proposed co-filtering mechanism to filter the noise and locate the user's 's true preference, the model could more precisely capture the user's s intentions and bring higher performance (comparing the results of w/o CL with that of c&un). Furthermore, the results of complete and w/o CL indicate that the curriculum learning strategy takes further step to help the model lear better parameters.
4.4 Curriculum Exploration
In this part, we explore what kind of curriculum strategy would benefit our model. We respectively explore the impact of the curriculum difficulty thre and the curriculum learning speed

---

Impact of Curriculum Difficulty As the difficulty of our curriculum is controlled by the parameter thre (larger thre means curriculum of higher difficulty), we set different values for thre to train our model, while fixing the other hyper-parameters the same as the random grid search result. The performance of the model trained with different values of thre is shown in Fig [[a.. The results on Amazon Sports dataset show the typical "easy-to-hard" curriculum pattern [31]. Learning from the easier samples first will make the objective smoother, thus more easily reaching the global optimal. In contrast, the results on the Amazon Beauty dataset show a different curriculum scenario: although learning easy samples first could achieve comparably good performance, learning from the samples that have difficulty of about 0.8 could lead to better performance. It matches the spirits of "hard example mining" [37], focusing more on the harder samples that are more informative for the model helps the model to discriminate the samples better. Although the results on the two datasets show that they have different suitable curriculum, there is one phenomenon in common, the too hard samples (thre 1.0) results in bad performance. This is quite reasonable for our scenario, because there exists label-level noise. While the noisy data always cause higher prediction error than the clean data [53] focusing the noisy data will absolutely cause the learned parameters sub-optimal.
curriculum concentrating
Impact of Curriculum Speed Fixing the most suitable curriculum difficulty thre, we change the hyper-parameter parameter T to see at what pace the model should learn. Smaller T means faster adapting to all the samples considering that smaller will make the Gaussian function smooth faster. From the results in bb), we can see that too fast or too slow curriculum is not good enough, and the best paces for different datasets are different. Moreover, we observe that if 7 equals 1, the performance on both datasets will drop 4] This phenomenon is easy to understand from human learning. If we always concentrate the problems of some particular level of difficulty, when the problems of different levels of difficulty are present to us. For the Amazon Sports dataset, since we fix thre 0.0, it always concentrates on the easiest samples and hardly learns the more difficult samples, and thus suffer dramatically performance drop when test, While for the Amazon Beauty dataset, since thre 0.8, it more focuses on its errors during all the training process, and thus not suffers as much performance drop as the Amazon Sports dataset.
cannot perform well in exams
4.5 Other Studies on Explainability
Disentangled Intentions We validate the disentanglement of our learned different intentions. We calculate the similarity between each item and each of K prototype intention, and assign each item to the intention that has the highest similarity with it. Then under each intention, we calculate the number of items belonging to each item category and plot the results in Figure 4 We the learned intentions have disentangled meanings. For example, in Figure 4(d), intention means interests in "Sports", while intention 5 mainly means preference towards "Enntrtaiiment". However, the disentanglement on the MovienLens-IN IM dataset is not so promising, almost all intentions are highly related to "Comedy", "Action", and "drama". This phenomenon is probably because the long historical behavior in MovieLens- IM brings great challenges for disentanglement as mentioned in
that
Mining Interests from Unclick Sequence We also conduct an ablation study to validate our claim that there also exist user's interests in the unclick sequence and model can locate these interests from these noisy data. One example is shown in Figure 3 and we can see that this user clicked topics about "Social livelihood", "Technology" and "Human history". Our interests mining algorithm discovers the user's interests in "International News" and "Education" from the unclick feedback. When the candidate item about "International News" comes, our model can make the right prediction based on these located interests, showing that our model could mine the user's s true interests that do not hide in the clicked history from the user's unclick history, thus capturing users' comprehensive and accurate intentions.
'we don't plot the performance on Amazon Sports dataset because its performance drops to 0.5645 when equals 1.0.

---

0.815
0.815
0.810
0,810
0.805
0.805
0.800
Sports
0.800
Amazon Sports CL
0.795
Sports
0.795
Amazon Beauty
0.790
Amazon Beauty
0.790
Amazon Beauty
Amazon Beauty CL
0.0
0.2
0.4
0.6
0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
thre
T
(a) curriculum difficulty
(b) curriculum speed
Figure 2:
Ablation on Curriculum
User:
#231
Click:
Social livelihood Technology
Human history
Unlick:
ffliss
International News
Comady
Pol
Spts
Education
Interests Mining
Edu
Suc live
Pul Afftirs
o
International News
Soc live.
Comeedy
ffairr
Poi
0
Spis
Education
(0.053) (0.012) (0.018) (0.31)
(0.72)
(0:01)
(0.01)
(0.003yc (030(0000000 (0.86) (0.003)
Intention A ggregation
Right Prediction V
International News
Candidate:
Prediction
0.78
Ground Truth: 1
Figure 3: Interests mining from unclick feedback
IntentionO
0.7
0.5
Hurling RRiii
Dutcoa
intertion1
Intertiond
0
Drimm
intentiond
0.35
Sporss
intention
Intertiond
0.4
intentions
Intention2
0.5
Gercho
intereion3
0 0.3
[2 0.5
0.4
intertiona
intertio22
0.25
0.20
0.15
Socia
Dely
Intertiona
Itantioon
0.2
0.1
0.1
0.2
intertion
0.05
ntetionns
Intention
8 1012141618
0.0
8 1012141618
0.00
100
Categories
categories
categories
categories
(a) Amazon Beauty
(b) Amazon Sports
(c) MovieLens- IM
(d) WcChat5D
Figure 4: Disentangled Intentions
5 Conclusion
In this paper, we propose to learn disentangled representations for user's intentions from multi- feedback. The proposed routing mechanism models the complex relations among multi-feedbbck and various user intentions, and tackles the impact of noise brought by the multi-feedback. Experimental results show that the learned disentangled representations from multi-feedback could capture compre- hensive user intentions and consequently improve the recommendation performance. Moreover, the proposed curriculum further alleviates the impact of data with improper labels in different datasets by adjustable hyperparameters, which can serve as an efficient plug-in for recommendation models. Future work may include explicitly locating the noisy sample labels in multi-feedback data.
Acknowledgement
This work is supported by the National Key Research and Development Program of China No. 2020AAA0106300 and National Natural Science Foundation of China (No. 62050110, No. 62102222).
10

---

References
[1] Xin Wang, Wenwu Zhu, and Chenghao Liu. Social recommendation with optimal limited attention. In Ankur Teredesai, Vipia Kumar, Ying Li, Rómer Rosales, Evimaria Terzi, and George Karypis, editors. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2019, Anchorage, AK, USA, August 4-8, 2019, pages 1518-1527. ACM, 2019.
[2] Shengze Yu, Xin Wang, Wenwu Zhu, Peng Cui, and Jingdong Wang- Disparity-presevee deep cross- platform association for cross-platform video recommendation. In Sarit Kraus, editor, Proceedings of the Twenty- Eiigty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, Macao, China, August 10-16, 2019, pages 4635 4641. ijeai.org. 2019.
[3] Qiangian Xu. Jiechao Xiong, Xiaochun Cao, Qingming Huang, and Yuan Yao. From social to individuals: A parsimonious path of multi-level models for crowdsourced preference aggregation. IEEE Trans. Puttern Anal. Mach. Intell., 8144):844-856, 2019.
[4] Xin Wang. Wei La, Martin Ester, Can Wang, and Chun Chen. Social recommendation with strong and weak ties. In Snehasis Mukhopadhyay, ChengXiang Zhai, Elisa Bertino, Fabio Crestani, Javed Mostafa, Jie Tang, Luo Si, Xiaofang Zhou, Yi Chang, Yunyao Li, and Parikshit Sondhi, editors, Proceedings of the 25th ACM International Conference on Information and Knowledge Management, CIKM 2016, Indianapolis, IN. USA, October 24-28, 2016, pages 14. ACM, 2016.
[5] Jianxin Ma, Chang Zhou, Peng Cui, Hongxia Yang, and Wenwu Zhu. Learning disentangled representations for recommendation. Advances in neural information processing systems, pages 571 171225 2019.
[6] Yin Zhang. Ziwei Zhu, Yun He, and James Caverlee. Content-collaborative disentanglement representation learning for enhanced recommendation. In Fourteenth ACM Conference on Recommender Systems, pages 43-52, 2020.
[7] Linmei Hu, Siyong Xu, Chen Li, Cheng Yang. Chuan Shi, Nan Duan, Xing Xie, and Ming Zhou. Graph neural news recommendation with unsupervised preference disentanglement. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4255-4264, 2020.
[8] Xiang Wang, Hongye Jin, An Zhang, Xiangnan He, Tong Xu, and Tat- tat-Seng Chua. Disentangled graph collahorative filterin.. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1001-1010, 2020.
[9] Jianxin Ma. Chang Zhou, Hongxia Yang. Peng Cui, Xin Wang, and Wenwu Zhu. Disentangled self- supervision in sequential recommenders. Rajesh Gupta, Yan Liu, Jiliang Tang, and B. Aditya Prakash, editors, KDD 200: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, CA, USA, August 23-27, 2020, pages 483 491. ACM, 2020.
[10] Ruobing Xie. Cheng Ling, Yalong Wang. Rui Wang. Feng Xia, and Leyu Lin. Deep feedback network for recommendation. Proceedings of IUCAI-,RICA, 2020.
[11] Qian Zhao, F Maxwell Harper, Gediminas Adomavicius. and Joseph A Konstan. Explicit or implicit leedback? engagement or satisfaction? field experiment on brecine-learning-bare based recommender systems. In Proceedings of the 33rd Anmual ACM Symposium on Applied Computing, pages 1331-1340, 2018.
[12] Yifan Hu, Yehuda Koren, and Chris Volinsky. Collaborative filtering for implicit feedback datasets. In 2008 Eighth IEEE International Conference on Data Mining, pages 263-272. Ieee, 2008.
[13] Yao Wu, Christopher DuBois, Alice Zheng, and Martin Ester. Collaborative denoising antodene for top-n recommender systems. Proceedings of the Ninth ACM International Conference on Web Search and Data Mining, pages 153-162, 2016.
[14] Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. Neural collaborative filtering. In Proceedings of the 26th international conference world wide web, pages 173-182, 2017.
[15] Xiangnan He, Hanwang Zhang, Min-Yen Yen Kan, and Tat-Seng Chua. Fast matrix factorization for online recommendation with implicit feedback. In Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval, pages 349-558, 2016.
[16] Jiawei Chen, Can Wang, Sheng Zhou Qihao Shi, Yan Feng. and Chun Chen. Samwalker: Social recommendation with informative sampling strategy. In The World Wide Web Conference, pages 228-239, 2019.

---

[17] Dawen Liang, Laurent Charlin. James McInemnyy and David M Blei. Modeling user exposure in rec- ommendation. In Proceedings the 25th international conference World Wide Web, pages 951-961, 2016.
[18] Erheng Zhong. Nathan Liu, Yue Shi. and Suju Rajan. Building discriminative user profiles for large- scale content recommendation. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 2277-2286, 2015.
[19] Xiang Wang, Yaokun Xu, Xiangnan He, Yixin Cao, Meng Wang. and Tat- Sen- Chua. Reinforced negative sampling over knowledge graph for recommendation. In Proceedings of The Web Conference 2020, pages 99-109, 2020.
[20] Jingtao Ding, Yuhan Quan, Xiangnan Hc, Yong Li, and Depeng Jin. Reinforced negative sampling for recommendation with exposure data. In IJCAI, pages 2230-2236,, 2019.
[21] Xiangyu Zhao, Liang Zhang, Zhuaye Ding, Long Xia, Jiliang Tang, and Dawei Yin. Recommendations with negative feedback via pairwise deep reinforcement learning. In Proceedings the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1040-1048, 2018.
[22] Chuhan Wu, Fangzhao Wu, Yongfeng Huang, and Xing Xie. Neural news recommendation with negative feedback. CCF Transactions Pervasive Computing and Interaction, 2(3): (3):178-188, 2020.
[23] Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives. IEEE transactions pattern analysis and machine intelligence, (788))1798-8888 2013.
[24] Diederik Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv: 1312,611 2013.
[25] Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-vae: Learning basic visual concepts with constrained variational framework. ICLR, 2017.
[26] Ricky Chen, Xuechen Li. Roger B Grosse, and David K Duvenaud. Isolating sources of disentanglement in variational autoencoders. In Advances in neural information processing systems, pages 26 10-2620. 2018.
[27] Hyunjik Kim and Andriy Mnih. Disentangling by factorising, arXiv preprint
2018.
[28] Haoyang Li, Xin Wang, Ziwei Zhang, Jianxin Ma, Peng Cui, and Wenwu Zhu. Intention aware sequential recommendation with structured intent transition. IEEE Transactions Knowledge and Data Engineering,
2021.
[29] Xin Wang, Hong Chen, and Wenwu Zhu. Multimodal disentangled representation for recommendation. In 2021 IEEE International Conference on Multimedia and Expo (ICME), pages 1-6, 2021.
[30] Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. Curriculum learning. In Proceedings of the 26th annual international conference on machine learning, pages 41-48, 2009.
[31] Xin Wang, Yudong Chen, and Wenwu Zhu, A survey curriculum learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021.
[32] Radu Tudor Tonescu, Bogdan Alexe, Marius I Leordeanu, Marius Popescu, Dim P Papadopoulos, and Vittorio Ferrari. How hard can it he? estimating the difficulty of visual search in image. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2157-2166, 2016.
[33] Emmanouil Antonios Platanios, Otilia Streteu, Graham Neubig, Barnabas Paczos, and Tom M Mitchell. Competence-hased curriculum learning for neural machine translation. arXiv preprint arXiv:/900 1903,0988,, 2019.
[34] Yudong Chen, Xin Wang, Miao Fan, Jizhou Huang, Shengwen Yang, and Wenwu Zhu. Curriculum meta-learning for next POI recommendation, In Feida Zhu, Beng Chin Ooi, and Chunyan Miao, editors, KDD '21: The 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, Singapore, August 14-18, 18, 2021, pages 2692 20022 ACM, 2021.
[35] M Pawan Kumar, Benjamin Packer, and Daphne Koller. Self-paced learning for latent variable models. In Advances in neural information processing systems, pages 1 1189-1199 2010.
[36] Guy Hacohen and Daphna Weinshall. On the power of curriculum learning in training deep networks. arXiv preprint 19Xiv:1904.03622 03626936.......... 2012.
12

---

[37] Abhinav Shrivastava, Abhinav Gupta, and Ross Girshick. Training region-based object detectors with online hard example mining. In Proceedings of the IEEE conference on computer vision and puttern recogmition, pages 761-769, 2016.
[38] Yang Fan, Fei Tian, Tao Qin, Xiang- Yang Li, and Tie- Yan Liu. Learning to teach. arXiv preprint arXiv:1805.0333 803603633,, 2018.
[39] Gaurav Kumar, George Foster, Colin Cherry, and Maxim Krikun, Reinforcement learning based curriculum optimization for neural machine translation. arXiv preprint arXiv:193..0000 2019.
[40] Xinyi Wang, Hieu Pham. Paul Michel, Antonios Anastasopoulos. Jaime Carbonell, and Graham Neubig. Optimizing data usage via differentiable rewards. In International Conference on Machine Learning, pages 9983-9995. PMLR, 2020.
[41] Mengye Ren, Wenyuan Zeng. Bin Yang. and Raquel Urtasun. Learnngg to reweight examples for robust deep learning. arXiv preprint arXiv:1803.09055 2018.
[42] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett, editors, Advances in Neural Information Processing Systems 30: Annual Conference Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 5998 6008, 2017.
[43] Tianyi Zhou, Shengjie Wang, and Jeff A Bilmes. Curriculum learning by dynamic instance hardness. Advances in Neural Information Processing Systems, 33, 2020.
[44] F. Maxwell Harper and Joseph A. Konstan. The movielens datasets: History and context. ACM Trans. Interact. Intell. Syst., 1(4):19:1-19:19 2016.
[45] Ruining He and Julian J. McAuley. Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. Jacqueline Bourdeau, Jim Hendler, Roger Nkambou, Ian Horrocks, and Ben Y. Zhao, editors, Proceedings of the 25th International Conference on World Wide Web, WWW 2016, Montreal, Canada, April 11 15, 2016, pages 507-517. ACM, 2016.
[46] Ruobing Xie, Shaoliang Zhang, Rui Wang, Feng Xia, and Loyu Lin. Hicrarchical reinforcement learning for integrated recommendation. 2021.
[47] Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiugiang He. Deepfim: a factorization- based neural network for ctr prediction. arXiv preprint arviv (033.0/7033 2017.
[48] Weiping Song, Chence Shi, Zhiping Xiao, Zhijian Duan, Yewen Xu, Ming Zhang, and Jian Tang. Autoint: Automatic feature interaction learning via self- attentive neural networks. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pages 1161-1170, 2019.
[49] Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. Deep interest network for click-through rate prediction. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages (059-1068, 2018.
[50] Wang Cheng Kang and Julian J. McAuley. Self-attentive sequential recommendation. In IEEE International Conference on Data Mining, ICDM 2018, Singapore, November 17-20, 2018, pages 197-206. IEEE Computer Suciety, 2018
[51] John C. Duchi, Elad Hazan, and Yoram Singer. Adaptive subgradient methods for online learning and stochastic optimization. J. Mach. Learn. Res., 12:2121-21.9.. 201
[52] Ling Yan, Wu-Jun Li, Gui-Rong Xue, and Dingyi Han. Coupled group lasso for web-scale CTR prediction in display advertising. In Proceedings of the 31th International Conference on Machine Learning, ICML 2014, Beijing, China, 21-26 June 2014, volume 32 of JMLR Workshop and Conference Proceedings, pages 602-810. JMLR.org. 2014.
[53] Eric Arazo, Diego Ortego, Paul Albert, Noel F. O'Connor and Kevin McGuinness. Unsupervised label noise modeling and loss correction. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, volume of Proceedings of Machine Learning Research, pages 312-321, PMLR, 2019.
13


Reviews:
[{"review_id": "cU63ENpc27m", "reviewer": "Reviewer_Bacg", "summary": "This paper proposes a model to learn disentangled representations for user intentions from complex and noisy multi-feedback. The model can capture complex relations among multi-feedback and various user intentions, and filter out feature-level noise in multi-feedback. Moreover, this paper proposes an adjustable curriculum to alleviate the impact of label-level noise in a more controllable way.\n\n", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality: This paper proposes a new method to utilize multi-feedback data and a new adjustable curriculum learning method. The contributions are clear and the related work is adequate. The method combines existing components to form a reasonable solution, so the overall technical novelty is not big. \n\nQuality: The proposed method is clear and the experiment is sufficient.\n\nClarity: This paper is well written and organized. And most of the proposed method and the experiment are clear. But I have doubts on some details. In second row and fourth row of Eq. (1), why does d_{ci}^{v} use softmax and d_{ui}^{v} use sigmoid? And in fourth row of Eq. (1), should the output of MLP between 0 and 1 like sigmoid? In Eq. (4), why are z_{ci}^{v}-n_{c}^{v} and z_{ci}^{v} \\odot n_{c}^{v} also inputs to MLP?\n\nSignificance: The idea of filtering the noise in both clicked and unclicked items from disliked items is particular. The proposed curriculum may be applied to other tasks. The extensive experiments shows that the model achieve state-of-the-art performance. \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "In Section 3.2.1, you take the time and candidate item factors into consideration, how these factors affect the performance of the model? You need to add an experiment to demonstrate it.\n\nIn In Algorithm 1, \\mu changes as the training goes on. Can $thre$ also change as the training goes on? \n", "review_text": "", "extra_scores": {"ethical_concerns": "NA", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4 hours"}, "extra_text": {}}, {"review_id": "UuJo1q6Ooqc", "reviewer": "Reviewer_foyQ", "summary": "The paper introduces an approach to take into account different types of positive and negative feedback in a recommendation model, while accounting for the possible noise contained in the feedback. The proposed model relies in particular on three types of feedback (click, lack of click and dislike). The sequences of different types of feedback are first encoded with a Transformer and then aggregated through a series of MLPs and attention mechanisms into vectors supposed to capture and disentangle the intents of the user.\n\nIndependently from this module, the paper also proposes a curriculum learning approach to train the model and allocate with flexibility more weight on either the easier samples or the harder samples at the beginning of the training.\n\nThe proposed approach is evaluated on 3 public datasets and 1 proprietary dataset, and compared against several baselines.", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "### Strengths\n\n- The proposed curriculum approach seems easy to use and general enough to be applied to a wide range of recommendation approaches.\n\n### Weaknesses\n\n- Many choices in the modeling of the \"co-filtering dynamic routing\" are overly complex, unclear or lack motivations/justifications (see Section \"Model design\").\n- The results of the paper are not reproducible as the code is proprietary.\n- The experiments miss the comparison against appropriate and recent baselines (see Section \"Empirical methodology\"), and did not report statistical significance or error bars. \n- Given the high complexity of the approach and its large number of parameters, comparing the runtime of the approach against that of baselines would have been needed.\n\n### Model design\n\n- Based on Section 3.1, the similarity function seems to systematically apply LayerNorm before computing the dot product and softmax operation. What is the motivation for using LayerNorm here?\n- In Section 3.2.1, the input item features are projected into an embedding space. It seems that features are assumed to be discrete. Is the approach still adapted for continuous features?\n- In Section 3.2.1, it is unclear why the sequence of click interactions, the sequence of \"unclick\" interactions, and the sequence of dislike interactions are all three considered independently. These sequences might have be linked together. For example, the user may first click on an item A, then check a similar item B and dislike it. One could have simply considered all the interactions in sequence, and after that extract the item representations for each interaction type. Additionally, it is not fully clear if the different Transformer encoders (C-Encoder, U-Encoder and D-Encoder) share some parameters or are totally independent.\n- What is the motivation for using a residual connection in Equations (1), (2) and (4)? No justification is provided on that, and the experiments do not report results with/without these to show the need for this.\n- The scalars d_ci^(v) and d_uj^(v) in Equation (1) represent the contribution of clicked and unclicked items to the user intention. d_ci^(v) is given by a softmax and is thus strictly positive, however d_uj^(v) is based on the output of an MLP and it is therefore unclear if this quantity is positive or not.\n- The MLP used in the definition of d_uj^(v) takes as input the concatenation of F^(v), which contains the base embeddings of all g user features, and z_uj^(v) which was output by the U-Encoder. How is the concatenation performed here given that there are g+1 vectors? Do these vectors have the same dimensions, given that some are base embeddings and some are output by the Transformer encoder?\n- In Equation (2), it is unclear what the terms time_ci^(v) and time'_cm^(v) are trying to capture. Given that z_ci^(v) and z_cm^(v) are the output of the Transformer encoder, they should already capture temporality/sequentiality through position embeddings. What is the need then to add a fully-connected layer that takes as input the embedding output by the Transformer *and* the positional embedding? This seems redundant and unnecessary.\n- In Equation (4), what are zl_ci^(v) and zl_uj^(v) intended to capture? It is unclear why it is needed to aggregate z_ci^(v) and n'_c^(v) rather than directly using z_ci^(v) in the definition of intent_k^(v).\n- The quantity inten_k^(v) in Equation (4) is supposed to capture the interest of user v according to the k-th intent. However inten_k^(v) depends through f_ci^(v) and f_uj^(v) on the candidate item considered. It is unclear why the intention of the user would depend on a candidate item, which should be totally independent from the user as it can be any item.\n- The multihead attention introduced in Equation (5) lacks motivations. What is the expected role of the different heads in this context?\n- If one omits the bias beta_k in Equation 4, which has only a marginal role, then the sum over the different intents in Equation 6 will actually result in ignoring c_ik^(v) and u_jk^(v), and the notion of intents altogether. Indeed, zl_ci^(v) does not depend on k and sum_k c_ik^(v) = 1. A similar observation holds for zl_uj^(v) and sum_k u_jk^(v). Then the sum over intents in Equation 6 removes all dependencies with respect to an intent: sum_k intent_k^(v) = sum_i d_ci^(v) * f_ci^(v) * zl_ci^(v) + lambda * sum_j d_uj^(v) * f_uj^(v) * zl_uj^(v). Therefore, the prediction made by the model mostly ignores intents, despite intent disentanglement being presented as a main feature of the approach.\n- In Equation 6, the dot product term is positive but it is unclear if the output of the MLP term is positive or not.\n\n### Empirical methodology\n\nThe sequential recommendation models used as baselines (DIN, SASRec, DFN, SDR) were designed for next-item recommendation: based on a past sequence of interactions, predict the next item. However the experimental setup corresponds to a top-k recommendation scenario: given the past history of interactions, predict the set of test items.\n\nThe sequential baselines are then not very appropriate for a task where the test set contains more than a positive item per user, because the different items in the test set won't be all correspond to the next item following the training sequence. Therefore, one needed to either consider a next-item recommendation setup with only one test item to predict, or compare instead against other top-k recommendation baselines.\n\nThe authors could have considered the state-of-the-art recommendation model LightGCN [1] as a potential, more recent baseline.\n\n[1] Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yong-Dong Zhang, Meng Wang: LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation. SIGIR 2020: 639-648\n\n### Clarity\n\n- It was not fully clear at the beginning of the paper what the \"unclick\" interaction means. An unclick interaction seems to suggest the user had first clicked on an item and then removed the click. But in fact in this paper an unclick interaction simply corresponds to a lack of click interaction. This should be made more clear and unclick interaction should instead be replaced with lack of click.\n- The choice to denote the positive interaction as \"click\" and the negative interaction has \"dislike\" creates an asymmetry which can be confusing for the reader. One could question one does not consider the positive interaction \"like\" which is the positive counterpart of \"dislike\" and exhibits more confidence than a click, which can be more noisy in its positiveness.\n- In Section 4.3, it is not fully clear what the variant c&un of the proposed model consists in.\n\n### Update after author response\n\nI thank the authors for their detailed response to my comments and questions. The extensive efforts that were put in the response is commendable and appreciated. However, there are still several points which, despite the authors' response, remain problematic for me (detailed below), and I will thus keep my initial score.\n- **Efficiency & Complexity.** The runtime reported by the authors for the proposed model indeed seems reasonable in comparison to that of DFN. However, the complexity indicated in the response does not seem correct as the reported Transformer complexity ignores the dependence on the length of interaction sequences.\n- **Empirical methodology.** The response clarified that the experiments follow a next-item recommendation setting. This will need to be explained in the experimental setup section as in its current version the paper and the supplementary material lack too many details about the evaluation procedure. It is also still not clear whether at inference time, all items are considered as candidates or if a sampling is performed (which should be avoided [2]). With respect to the argument that LightGCN cannot be used as a baseline because of the next-item recommendation setting, I do not agree with the authors. Some of the baselines considered in the paper, such as DeepFM and AutoInt, were not initially designed either for the next-item recommendation task. Therefore, LightGCN could be adapted to the task in the same way as it is done for other baselines. This can be simply done by retraining the model each time new samples are added to the training set.\n- **Choice of different behaviors.** The authors claim that \"like\" behavior is much more rare than \"dislike\" behavior, however I did not see justifications or references to explain why there would be such an asymmetry.\n- **Motivations for using LayerNorm.** The authors mention that using a simple dot product as opposed to a cosine similarity leads to mode collapse, and thus normalization is needed. Then, why not use a simple cosine similarity, by dividing the dot product by the vector norms, instead of using LayerNorm? This argument did not convince me that LayerNorm is necessary or needed in the proposed model. This design choice seems to be guided by the fact that Transformer architectures adopt this, without evidence that this is needed in the proposed model (which vastly differs from a vanilla Transformer). For this choice and others (e.g., use of residual connections) I would have liked to see ablation studies to validate their necessity.\n- **General user intentions should be independent of the candidate item.** I still find problematic and unintuitive that user intentions depend on the candidate item. The user-item matching should intuitively be done by comparing a user representation (independent of the specific candidate item) and an item representation (independent of the specific user). Otherwise, the matching process boils down to matching an item with itself, which makes little sense. Having a user representation which depends on the candidate item also seems to lead to inefficient and non-scalable inference, as in the inference phase all items in the datasets need to be considered as candidate items for each user.\n- **About the disentangled intentions.** The authors describe beta_k as the center under the k-th intention, and zl_ci and zl_uj as offsets to these intention centers. However, zl_ci and zl_uj are not specific to any intention as they do not depend on intent k; it is then unclear how could they be at the same time the offset from all intent centers beta_k. Moreover, the authors' response did not address the fact that the influence of intents on Eq. 6, with or without beta_k, remain very minor as detailed in my original comment.\n\n[2] Krichene, W., & Rendle, S. (2020). On Sampled Metrics for Item Recommendation. Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 1748–1757. \n\n### Update 2\n\nI thank the authors again for the additional clarifications provided in the second round of discussion. In light of the satisfactory response brought by the authors, and although I still consider that some parts of the proposed approach are slightly arbitrary and that significant work will be needed to polish the paper based on our discussion, I will raise my score.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The paper did not discuss the limitations of this work and its potential negative societal impact.\n\nA limitation of the work is in the type of feedback considered. The approach is based on \"click\", \"lack of click\" and \"dislike\" signals. However a platform could have access to other types of signals (\"like\", \"add to basket\", \"add to wishlist\", \"remove from wishlist\", etc.) and it is unclear if the proposed approach could directly use this feedback.\n\nAdditionally, the approach seems to suppose that user/item features are discrete and did not cover how continuous features could be handled.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}, {"review_id": "PhaaEJRCjit", "reviewer": "Reviewer_jp7S", "summary": "The paper proposes a curriculum disentangled recommendation model to learn disentangled representations from multi-feedback data. Specifically, the authors design a co-filtering routing mechanism to capture the relations among multi-feedbacks and denoise the representations at the feature level. Then they utilize the curriculum learning to evaluate sample difficulties to do denoising at the label level. The extensive experiments show the effectiveness of the proposed method on recommendation tasks.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros:\n-\tThe proposed method tries to learn the disentangled representation for the recommendation problem from multi-feedback with noisy information. The task is well-motivated and important for real-world scenarios.\n-\tThe paper originally considers two-level noisy information. One is feature-level noise, and another is label-level noise.\n-\tThe paper is well-written and easy to follow.\n\nCons:\n-\tThe authors should give the definitions of feature level noise and label level noise. The label-level noise is easy to understand. But the feature level noise is confusing. For example, what kind of click action would be regarded as noise? The noisy unclicked items might be useless items that are different from the noise in the labels.\n-\tThe hyper-parameters of curriculum learning need to be searched and predefined manually. It is difficult to find the optimal parameters for different datasets, especially in the online scenario. \n-\tHow to decide the number of k latent intentions is not clear.\n-\tThe code is not provided. It is hard for readers to reproduce the results in the experiments.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "No discussions about the potential negative societal impact.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}]

Author rebuttal:

Meta-review: Overall strong scores, and reviewers are mostly aligned in finding the paper to be above the bar. Scores were initially more mixed, though after the discussion the weakest score was improved (to "borderline"). Reviewers praised the originality of the idea, and found the model performance through the experiments convincing enough. Several issues were raised regarding clarity and other specific details, and an in-depth discussion took place to resolve most of these issues.

Ultimately the scores are strong, and the rebuttal was fairly persuasive. The reviewers do highlight several issues (and in spite of the strong scores, the reviews are not particularly gushing in the text). But the issues raised seem mostly in terms of clarity, or otherwise are adequately  addressed during the rebuttal phase.
Decision: Accept (Poster)