Curriculum Disentangled Recommendation with Noisy Multi-feedback
Year: 2022

Abstract:
Learning disentangled representations for user intentions from multi-feedback (i.e., positive and negative feedback) can enhance the accuracy and explainability of recommendation algorithms. However, learning such disentangled representations from multi-feedback data is challenging because i) multi-feedback is complex: there exist complex relations among different types of feedback (e.g., click, unclick, and dislike, etc) as well as various user intentions, and ii) multi-feedback is noisy: there exists noisy (useless) information both in features and labels, which may deteriorate the recommendation performance. Existing disentangled recommendation works only focus on positive feedback, failing to handle the complex relations and noise hidden in multi-feedback data. To solve this problem, in this work we propose a Curriculum Disentangled Recommendation (CDR) model that is capable of efficiently learning disentangled representations from complex and noisy multi-feedback for better recommendation. Concretely, we design a co-filtering dynamic routing mechanism which simultaneously captures the complex relations among different behavioral feedback and user intentions as well as denoise the representations in the feature level. We then present an adjustable self-evaluating curriculum that is able to evaluate sample difficulties for better model training and conduct denoising in the label level via disregarding useless information. Our extensive experiments on several real-world datasets demonstrate that the proposed CDR model can significantly outperform several state-of-the-art methods in terms of recommendation accuracy3.

heading: 1 Introduction
text: Recommenders aim to capture the user’s preferences from different aspects of information for more accurate prediction[1–4] . Learning disentangled representations that can uncover and disentangle the latent explanatory factors hidden in user behavioral data has recently been shown as an effective way to discover users’ intentions, improving both recommendation accuracy and explainability [5–9]. Multi-feedback, normally including the positive feedback (e.g., click) and negative feedback (unclick and dislike, etc.), is of great significance to depict the user’s unbiased and various intentions [10]. Learning disentangled representation from multi-feedback is able to capture various user intentions more accurately, leading to improvement of accuracy and explainability in recommendation.
However, learning such disentangled representation from multi-feedback that can best serve recommendation is quite challenging due to two reasons. i) Multi-feedback is complex: different types of ∗Equal contributions †Corresponding Authors. 3Our code will be released at https://github.com/forchchch/CDR
35th Conference on Neural Information Processing Systems (NeurIPS 2021).
feedback in multi-feedback data have complex relations with each other, and the relations between multi-feedback and various user intentions are also complicated. For instance, a user may unclick an item because of disinclination, getting tired of seeing the same type of item previously clicked too many times, as well as lacking enough time to view the item in detail. ii) Multi-feedback is noisy: the large amount of feedback such as unclick brings a lot of noisy information (useless for recommendation) that may severely deteriorate the recommendation accuracy. For example, a user may unclick an item because he truly dislikes this item or he has interests in the item but is interrupted by others before making the click decision. As such, completely incorporating the unclick feedback as historical features to extract user intention representations may run the risk of bringing feature-level noise, while directly regarding the unclick behavior as negative samples to train the model will probably introduce label-level noise. Existing works on disentangled representation learning merely rely on the positive user feedback to extract the user intentions, failing to handle the complex relations and noisy (useless) information hidden in multi-feedback data.
To tackle these challenges, we propose a Curriculum Disentangled Recommendation (CDR) model that is able to accurately discover various user intentions from different kinds of user feedback. The CDR model consists of two core components, i.e., the co-filtering dynamic routing mechanism and the adjustable self-evaluating curriculum strategy, which together learn disentangled representations as well as captures the complex relational dependencies and filters out noise from multi-feedback to achieve more accurate recommendation. More concretely, the proposed routing mechanism utilizes the dependencies among different feedback to accurately discover the user preferences, followed by the intention aggregation which further helps to learn disentangled representations for users. Our adjustable self-evaluating curriculum then guides the model towards better optima by reweighing samples according to the self-evaluated difficulty. The proposed curriculum can further enhance the model performance by making the model learn from multi-feedback data of different difficulties at different learning paces, thus preventing the model from overfitting the data with improper labels. We conduct extensive experiments on several real-world datasets to demonstrate that our proposed CDR model can significantly beat baseline methods in terms of recommendation accuracy.
To summarize, our contributions are listed as follows. (1) We propose a Curriculum Disentangled Recommendation (CDR) model to learn disentangled representations for recommendation from multi-feedback with complex relations and noisy information. (2) We propose a co-filtering dynamic routing mechanism to simultaneously capture complex relations among different sorts of feedback and various user intentions as well as filtering out feature-level noise in multi-feedback. (3) We propose an adjustable self-evaluating curriculum to guide the model towards better optima by alleviating the impact of label-level noise in a more controllable and convenient way.

heading: 2 Related Work
text: Recommendation Based on Multi-Feedback Apart from positive feedback, negative feedback data are also essential for capturing user preference [11]. Early efforts [12–14] regard all unclick data equally as negative feedback and decrease the confidence compared to click data. Later works utilize additional information [15–18] or reinforcement learning [19–21] to distinguish real negative signals from all the unclick data. A recent method [22] also recognizes negative feedback from the clicked news by reading dwell time. However, all these methods merely select better negative training samples and ignore multi-feedback as features to learn user interests. DFN [10] simultaneously incorporate click, unclick, and dislike feedback simply by rough attention mechanism and concatenation, which is insufficient to capture the user’s comprehensive and accurate interests.
Disentangled Representation Learning Disentangled representation learning aims to learn various hidden explanatory factors behind observable data in different parts of the learned vector presentation [23]. Many variants of variational auto-encoder (VAE) [24] have been studied to improve the disentanglement of the learned representation [25–27] by adding regularization terms to decrease the mutual information between different parts of the learned vector. Disentangled representation learning has also found its application in recommendation [5–8, 28, 29] by learning disentangled user preference from user positive feedback to improve both the performance and interpretability. Different from these works, our work focuses on learning more comprehensive disentangled interests with multi-feedback.
Curriculum Learning Curriculum learning [30, 31] aims to design a dynamic sample reweighting strategy throughout the training process to improve performance and training efficiency. Traditional methods mostly follow an easy-to-hard paradigm, i.e., assigning higher weights to easier samples in earlier training, where the “easiness” measurement can be both domain-knowledge-based [32– 34] and loss-based [35, 36]. These methods can only be effective for specific tasks, and “easyto-hard” is sometimes sub-optimal compared with “harder first” [37]. Recent works propose to automatically learn a curriculum by reinforcement learning [38–40], meta-learning [41], etc. However, the optimization process of these curricula is time- or resource-consuming, which is too costly for abundant recommendation data. Our method provides an efficient and also flexible curriculum that is beyond the easy-to-hard limit and more adjustable and interpretable than automatic methods.

heading: 3 Method
text: In this section, we introduce the proposed Curriculum Disentangled Recommendation (CDR) model (Figure 1) to mine comprehensive and accurate user intentions from users’ noisy multi-feedback.

heading: 3.1 Notations and Problem Formulation
text: Notations We denote φ(a,b) as the inner product of two vectors LayerNorm(a) and LayerNorm(b), where LayerNorm refers to Layer Normalization operation and φ(a,b) measures the similarity between vector a and b. sim(keyi,query) is the normalized similarity between query and keyi on set Q that is composed of all the keys: sim(keyi,query) = exp(φ(keyi,query))∑ i′∈Q exp(φ(keyi′ ,query)) . Multi-feedback based prediction The vth user’s historical behavior contains his or her clicked item sequence c(v) = [c(v)1 , c (v) 2 , · · · , c (v) m ], unclicked (i.e., presented but not clicked) item sequence u(v) = [u (v) 1 , u (v) 2 , · · · , u (v) n ], and disliked (i.e., press to the dislike button or low rated) item sequence d(v) = [d(v)1 , d (v) 2 , · · · , d (v) l ], where each term in these sequences represents an item that the v(th) user interacted with. Our goal is to learn users’ disentangled intentions and then accurately predict svt, i.e., the vth user’s preference towards the candidate item t(v). Besides, we utilize the profile (e.g., age and gender) of the vth user to aid the learning and prediction process.

heading: 3.2 Co-filtering Dynamic Routing
text: The routing mechanism takes as input user profile, candidate item feature, and user multi-feedback history, and makes the final prediction by the following three steps. It first utilizes the relations behind different kinds of feedback to discover where the user’s true interests locate. Then, the model aggregates the user’s disentangled intentions from the useful behavior. Finally, it predicts the user’s preference towards the candidate item based on the learned intentions.

heading: 3.2.1 Interests Mining
text: The user’s interests could be reflected by his or her various kinds of feedback. Ignoring any kind of feedback may lead to incomplete or inaccurate preference modeling. However, the noise hidden behind these data makes it infeasible to directly use multi-feedback to learn the user’s interests. Thus, how to make use of the relations of different kinds of feedback to discover where the user’s true interests locate is the key in this step.
First, we project each item in c(v), d(v), and u(v) into the embedding space by concatenating its ID and category embedding. Thus, we obtain the clicked embedding sequence h(v)c = [h (v) c1 ,h (v) c2 , · · · ,h (v) cm], unclicked embedding sequence h(v)u = [h (v) u1 ,h (v) u2 , · · · ,h (v) un ], and disliked embedding sequence h(v)d = [h (v) d1 ,h (v) d2 , · · · ,h (v) dl ]. We also project the information of the user profile into the embedding space and obtain the user profile feature F(v) = {F(v)1 ,F (v) 2 , · · · ,F (v) g }, where g is the number of user profiles. We then utilize the power of transformer encoder [42] to obtain better representation for each item with fully interactions: [z(v)c1 , z (v) c2 , · · · , z (v) cm] = C-Encoder(h (v) c ), [z (v) u1 , z (v) u2 , · · · , z (v) un ] = U -Encoder(h(v)u ), [z (v) d1 , z (v) d2 , · · · , z (v) dl ] = D-Encoder(h (v) d ), where the z (v) ci , z (v) ui , z (v) di refer to ddimensional feature vectors containing the vth user’s interests.
We then need deeper thinking towards the relations among multiple feedback to discover user’s true interests. The first thing to consider is where the user’s interests will locate. We note that not only the click behavior contains user interests, but the unclick sequence generated by the recommender algorithms could also contain rich information about user interests. Therefore, utilizing both kinds of feedback could help to learn comprehensive user interests. The second noteworthy thing is how to discover the user’s true interests from the noisy data. To filter out the noise information in these historical sequences, we need the disliked item sequences that could reflect the user’s strong negative intentions and have high confidence. Thus, we average these strong negative features and regard it as the negative intention of the vth user: n(v) = 1l ∑l i=1 z (v) di . This highly confident negative intention could be used to filter the noise in both clicked and unclicked history items. We adopt the similarity between each item and the negative intention to judge whether the item should contribute to the user intentions. Based on the intuition that higher similarity to the negative intention should contribute less to the user preference, the contribution of each item to the user intentions can be formulated as follows:
n ′(v) c = n (v) + bc1, z ′(v) ci = z (v) ci + σ1(Wc1z (v) ci + bc2),
d (v) ci =
exp(−φ(z ′(v) ci , n ′(v) c ))∑m
j=1 exp(−φ(z ′(v) cj , n ′(v) c ))
,
n ′(v) u = n (v) + bu1, z ′(v) uj = z (v) uj + σ1(Wu1z (v) uj + bu2),
d (v) uj = σ(−φ(z ′(v) uj , n ′(v) u ) +MLP ([F (v); z(v)uj ])),
(1)
where σ1 is the ReLU activation function and σ is the Sigmoid activation function. d (v) ci and d (v) uj represent the contribution of each clicked item and unclicked item to the user intention, respectively. We obtain d(v)ci by calculating the similarity between each clicked item and the negative intention, and we also introduce the parameters Wc1,bc1,bc2 to avoid some outliers that disobey our intuition. As for the unclicked item sequence, there exists more noise. Thus, we utilize not only the information of the user negative intention but also the user profile feature F(v) to measure the importance of each
unclicked item. We first concatenate user profile features and the unclicked item feature together, and then use a two-layer MLP to judge whether the unclicked item is important.
Besides the noise effect, we also take the time and candidate item factors into consideration. Intuitively, more recently clicked items and items having a higher similarity to the candidate item will contribute more to the user preference towards the candidate item. The formulation is as follows:
time(v)ci = z (v) ci + σ1(Wc2[z (v) ci ; pi] + bc3), time ′(v) cm = z (v) cm + σ1(Wc3[z (v) cm; pm] + bc4),
cand(v)ci = z (v) ci + σ1(Wc4z (v) ci + bc5), z ′(v) ct = h (v) t + bc6,
f (v) ci = sim(time (v) ci , time ′(v) cm ) + sim(cand (v) ci , z ′(v) ct ),
cand(v)uj = z (v) uj + σ1(Wu2z (v) uj + bu3), z ′(v) ut = h (v) t + bu4, f (v) uj = sim(cand (v) uj , z ′(v) ut ).
(2)
We consider the impact of both time and candidate factors for the clicked items. The candidate item feature h(v)t ∈ Rd and the most recently clicked item feature z (v) cm measures the importance of each clicked item. pi is the position embedding of each item and encoded for more accurate time factor consideration. However, for the unclick sequence, we only consider the candidate factor, because the time of unclicking one item has low relations with the user’s current interests. f (v)ci and f (v) uj are the importance of each clicked item and unclicked item, respectively.

heading: 3.2.2 Intention Aggregation
text: After considering how each kind of feedback will contribute to the user’s intentions, we then conduct intention aggregation to obtain the vth user intentions under various latent categories. Assuming that the intentions of all users can be decomposed to K latent categories and each latent category has its prototype mk ∈ Rd, k = 1, 2, · · · ,K, we can predict the probability of each item belonging to the kth latent category by their similarities:
c (v) ik = sim(mk, z (v) ci ), u (v) jk = sim(mk, z (v) uj ). (3)
We then use the highly confident negative intention to filter the click and unclick historical item features by a residual structure. The kth intention of the vth user could be aggregated from all the filtered clicked and unclicked item features, z1(v)ci and z1 (v) uj as Eq. (4). βk is the bias for the k th latent category. λ < 1 is the prior confidence of the unclicked item. inten(v)k is the v th user’s interest under the kth latent category.
z1(v)ci = z (v) ci +MLP ([z (v) ci ; n ′(v) c ; z (v) ci − n ′(v) c ; z (v) ci n ′(v) c ]),
z1(v)uj = z (v) uj +MLP ([z (v) uj ; n ′(v) u ; z (v) uj − n ′(v) u ; z (v) uj n ′(v) u ]), inten(v)k = m∑ i=1 d (v) ci · f (v) ci · c (v) ik (z1 (v) ci + βk) + λ l∑ j=1 d (v) uj · f (v) uj · u (v) jk · (z1 (v) uj + βk).
(4)

heading: 3.2.3 Prediction
text: Based on the learned intentions, we can predict the user’s preference towards the candidate item by calculating the inner product between the candidate item and user disentangled intentions. However, there also exist some users who might have little historical behaviors. So we also manage to discover the interests of the vth user from his or her profile feature and the candidate item feature, aiming to capture some common patterns, e.g., females may show high interests in fashion articles. We collect these information in Fall = [F (v) 1 , · · · ,F (v) g ,h (v) t1 , · · · ,h (v) tr ], where h (v) tj ∈ R d r is one field of the candidate item embedding h(v)t . Then the multi-head attention [42] is used to capture the interactions of different fields of features:
[f1,f2, · · · ,fg+r] =MultiheadAttention(Fall). (5) By simultaneously considering the user’s interests learned from the historical multi-feedback and feature interaction, we can predict the user preference towards the candidate item svt as Eq. (6), where the first term is the preference inferred from user feedback (<> is the inner product) and the second term is from the feature interaction between the user profile and item features.
svt = K∑ k=1 < inten(v)k , h (v) t > +MLP ([f1;f2; · · · ;fg+r]). (6)
Our loss function is designed as follows, composing of the widely adopted binary cross-entropy loss and the regularization term for disentanglement aiming to minimize the similarities among different intentions. yvt is the ground truth label, and DT is the training set.
L = − 1|DT | ∑
(v,t)∈DT
yvt log σ(svt) + (1− yvt) log(1− σ(svt)) + K∑ i=1 ∑ j 6=i φ(mi,mj) (7)

heading: 3.3 Adjustable Self-evaluating Curriculum towards A Better Self
text: All the aforementioned methods could help to capture the user’s comprehensive true interests from the input features. However, the noise existing in the training labels will also mislead our model to suboptimal parameters. To tackle the problem, we leverage the idea of curriculum learning to denoise [31]. However, it is proved in [36] that different curriculum strategies (e.g., easy to hard, hard example mining, etc.) can be effective for different dataset settings, and thus we need a flexible curriculum design to adapt to complex recommendation scenarios.
To this end, we propose a novel adjustable self-evaluating curriculum for recommendation, which algorithm is shown in Algorithm 1. Our goal is to obtain the optimal parameters θ for our recommender F . When training on the batch B, we first calculate the prediction result σ(sj) for each sample and the difference between the ground truth label yj and our prediction. Then, we obtain the importance of each sample through a Gaussian distribution, where the sample with a closer difference to a preset value thre will get higher importance. Here, the hyperparameter thre ∈ (0, 1) reflects the sample difficulty we hope the model to focus on. Concretely, if thre approaches 1, the harder samples will get higher weights. While if thre approaches 0, the model will lay more emphasis on the easier samples. In our experiment, we can adjust the values for thre to set the curriculum of different difficulty levels for our model to improve itself. Finally, we optimize the parameters by the reweighted loss with an existing optimization method π. In the algorithm, µ controls the degree of concentration of the model and τ is a time-weight-decay factor. As the training goes on, µ is becoming smaller and smaller by timing τ , which makes the Gaussian distribution smoother and smoother so that all the samples almost get equal weights for training in the end. This time-varying process conforms to the human learning process: after we gradually improve ourselves by a scheduled curriculum, we should review all the samples to further consolidate the knowledge in our mind. Since in the early stage the model has already learned enough knowledge, it could be more robust and less likely to be influenced by the noisy samples. By adjusting the values of τ , we can control the curriculum learning speed, where a larger τ means a slower learning process.
Algorithm 1 Adjustable Self-evaluating Curriculum towards A Better Self 1: input: {(xvt, yvt)}(v,t)∈DT , π(·), `(·, ·), τ < 1, counter, interval, thre, F 2: initialize: θ, µ 3: for B ⊂ DT do 4: for (xj , yj) ∈ B do 5: sj = F (xj), dj = abs(yj − σ(sj)); 6: wj = exp(−µ× (dj − thre)2); 7: wj ← wj/ ∑|B| j=1(wj);
8: end for 9: θ ← θ − π(∇θ ∑|B| j=1 wj`(sj , yj) + ∑K i=1 ∑ j 6=i φ(mi,mj));
10: counter ← counter + 1; 11: if counter%interval == 0 then 12: µ← µ× τ ; 13: end if 14: end for
Discussion. Our proposed curriculum elegantly converts the designing process of a curriculum training strategy to a hyper-parameter search process on thre, µ and τ , which improves the flexibility, controllability, and explainability of curriculum design, while goes beyond the conventional assumption of “easy-to-hard” [30, 35, 32, 33]. Meanwhile, compared to automatic curricula [38, 40, 43, 39], our method requires almost no extra time and memory overhead for learning and applying the curriculum, which is efficient and conforms to the demand of recommendation scenarios.

heading: 4 Experiments

heading: 4.1 Experimental Setup
text: Datasets We conduct our experiments on four real-world datasets: WeChat5D, MovieLens-1M[44], Amazon Sports[45] and Amazon Beauty[45]. The datasets statistics are shown in Table 1. WeChat5D is a mobile article recommendation dataset of WeChat Top Stories and itself has different kinds of user feedback. All data are preprocessed via data masking to protect user privacy. For the Amazon and MovieLens dataset, we regard the ratings that are larger than 2 points as click feedback, while the rest as the dislike feedback. Since these three datasets have no information about the items that are recommended to the users but are unclicked, we simulate a simple recommendation environment. For each piece of user like or dislike interaction, we generate four pieces of unclick interactions for this user. They include three items that are sampled from the top popular items at that time and one item randomly sampled from all the items. This simulates the simplest recommendation rule, i.e., recommendation based on item popularity. The randomly sampled item simulates the scenario that the recommenders would always recommend something to explore the customer’s potential interests [46]. Note that all the user’s unclick interactions are not in the user clicked or disliked item sets. More specifically, based on the two Amazon datasets and the MovieLens-1M dataset, for each piece of user like or dislike interaction, we generate four pieces of unclick interactions for this user. They include three items that are sampled from the top popular (top 3000 for Amazon datasets and top 1/2 for MovieLens-1M) items at that time and one item randomly sampled from all the items. The whole dataset is chronologically divided to the train, valid, and test dataset by the ratio of 8:1:1. Note our training and testing phase follow the sequential recommendation setting. For example, if one user’s historical behavior is a sequence {1, 2, 3, · · · , 18, 19, 20}. Then, we will generate the validation and test samples as follows: two validation samples {[1, 2, 3, 4, · · · , 16], [17]}, {[1, 2, 3, 4, · · · , 16, 17], [18]} and two test samples {[1, 2, 3, 4, · · · , 16, 17, 18], [19]}, {[1, 2, 3, 4, · · · , 16, 17, 18, 19], [20]}, where the first term in [ ] represents the historical information we use for prediction and the second term is the next item for prediction. We always use all the user’s real historical behaviors as the sequential input to the models to predict the next item the user will click.
Baselines We compare our approach with the state-of-the-art (SOTA) methods. DeepFM [47] and AutoInt [48] are recommenders based on feature interaction. DIN [49], SASRec [50], DFN [10], SDR [9] are methods based on the user’s sequential historical behavior. Specifically, DFN considers different kinds of user feedback and concatenates these features together. SDR only utilizes positive click feedback to capture the user’s disentangled interests. For fair comparison, we add the feature interaction module to all the baselines if the model originally does not utilize the user profile.
Implementation and hyper-parameters We implement our method in Tensorflow and use the Adagrad [51] optimizer for mini-batch gradient descent that is suitable for sparse data, while the size of each mini-batch is 256. All the mentioned transformer encoders are four-head and one-layer. We cap the maximum sequential historical behavior length to 30 for all datasets. We fix µ in the curriculum to 10 and the other hyper-parameters are then tuned using random search. The search space is listed as follows. More detailed experimental settings can be found in our appendix.
• The number of latent intentions K ∈ {1, 2, · · · , 8}.
• The prior confidence for the unclicked data λ ∈ {0.1, 0.2, · · · , 1.0}.
• The learning rate ∈ {0.0001, 0.001, 0.01, 0.1, 1.0}.
• The hidden size of each field of feature ∈ {32, 64, 128, 256}

heading: 4.2 Recommendation Performance
text: We evaluate the performance of our proposed method on the classical click-through-rate (CTR) prediction task and utilize a widely-used metric Area Under Curve (AUC) for evaluation. We also follow [52] to use the RelaImpr to measure the relative improvements over the base model (i.e.,DeepFM in our experiment). The results are shown in Table 2.
We observe that our approach outperforms the baselines significantly, both on the dense MovieLens dataset and the sparse Amazon and WeChat5D dataset. We can see that on the MovieLens dataset where each user has rich historical behaviors, the models that utilize the sequential historical behavior obtain much better performance. However, DFN fails to accurately capture the user’s interests compared to other sequential models in MovieLens-1M. This is likely because DFN uses three kinds of feedback and in this dataset, the user has rich feedback that contains more noise (introduced by more unclick behavior), while DFN fails to filter the noise. Our method performs best on datasets that have different sparsity, mainly benefiting from both our model design and the curriculum training strategy, which is discussed in Section 4.3 and Section 4.4, respectively.

heading: 4.3 Multi-feedback
text: We validate the effectiveness of our method on capturing the user’s preference from different kinds of feedback. We compare the following four situations, our complete method (complete), our model without curriculum (w/o CL), our model that only utilizes the clicked historical feedback without curriculum (click), and our model that directly utilizes the clicked and unclicked historical feedback(i.e, the output of the C-Encoder and U-Encoder) to aggregate the user’s intentions without curriculum (c&un). The result is shown in Table 3.
By comparing the click and c&un results, we conclude that directly utilizing the unclick historical behavior will not improve the model performance. A possible reason is that the noise in unclick feedback will prevent the model from capturing the user’s true interests. However, with our proposed co-filtering mechanism to filter the noise and locate the user’s true preference, the model could more precisely capture the user’s intentions and bring higher performance (comparing the results of w/o CL with that of c&un). Furthermore, the results of complete and w/o CL indicate that the curriculum learning strategy takes a further step to help the model to learn better parameters.

heading: 4.4 Curriculum Exploration
text: In this part, we explore what kind of curriculum strategy would benefit our model. We respectively explore the impact of the curriculum difficulty thre and the curriculum learning speed τ .
Impact of Curriculum Difficulty As the difficulty of our curriculum is controlled by the parameter thre (larger thre means curriculum of higher difficulty), we set different values for thre to train our model, while fixing the other hyper-parameters the same as the random grid search result. The performance of the model trained with different values of thre is shown in Fig 2(a). The results on Amazon Sports dataset show the typical “easy-to-hard” curriculum pattern [31]. Learning from the easier samples first will make the objective smoother, thus more easily reaching the global optimal. In contrast, the results on the Amazon Beauty dataset show a different curriculum scenario: although learning easy samples first could achieve comparably good performance, learning from the samples that have difficulty of about 0.8 could lead to better performance. It matches the spirits of “hard example mining” [37], focusing more on the harder samples that are more informative for the model helps the model to discriminate the samples better. Although the results on the two datasets show that they have different suitable curriculum, there is one phenomenon in common, curriculum concentrating on the too hard samples (thre = 1.0) results in bad performance. This is quite reasonable for our scenario, because there exists label-level noise. While the noisy data always cause higher prediction error than the clean data [53], focusing on the noisy data will absolutely cause the learned parameters sub-optimal.
Impact of Curriculum Speed Fixing the most suitable curriculum difficulty thre, we change the hyper-parameter τ to see at what pace the model should learn. Smaller τ means faster adapting to all the samples considering that smaller τ will make the Gaussian function smooth faster. From the results in Fig 2(b), we can see that too fast or too slow curriculum is not good enough, and the best paces for different datasets are different. Moreover, we observe that if τ equals 1, the performance on both datasets will drop4. This phenomenon is easy to understand from human learning. If we always concentrate on the problems of some particular level of difficulty, we cannot perform well in exams when the problems of different levels of difficulty are present to us. For the Amazon Sports dataset, since we fix thre = 0.0, it always concentrates on the easiest samples and hardly learns the more difficult samples, and thus suffer dramatically performance drop when test. While for the Amazon Beauty dataset, since thre = 0.8, it more focuses on its errors during all the training process, and thus not suffers as much performance drop as the Amazon Sports dataset.

heading: 4.5 Other Studies on Explainability
text: Disentangled Intentions We validate the disentanglement of our learned different intentions. We calculate the similarity between each item and each of K prototype intention, and assign each item to the intention that has the highest similarity with it. Then under each intention, we calculate the number of items belonging to each item category and plot the results in Figure 4. We can see that the learned intentions have disentangled meanings. For example, in Figure 4(d), intention 0 means interests in “Sports”, while intention 5 mainly means preference towards “Entertainment”. However, the disentanglement on the MovienLens-1M dataset is not so promising, almost all intentions are highly related to “Comedy”, “Action”, and “drama”. This phenomenon is probably because the long historical behavior in MovieLens-1M brings great challenges for disentanglement as mentioned in [9].
Mining Interests from Unclick Sequence We also conduct an ablation study to validate our claim that there also exist user’s interests in the unclick sequence and our model can locate these interests from these noisy data. One example is shown in Figure 3 and we can see that this user clicked topics about “Social livelihood”, “Technology” and “Human history”. Our interests mining algorithm discovers the user’s interests in “International News” and “Education” from the unclick feedback. When the candidate item about “International News” comes, our model can make the right prediction based on these located interests, showing that our model could mine the user’s true interests that do not hide in the clicked history from the user’s unclick history, thus capturing users’ comprehensive and accurate intentions.
4we don’t plot the performance on Amazon Sports dataset because its performance drops to 0.5645 when τ equals 1.0.

heading: 5 Conclusion
text: In this paper, we propose to learn disentangled representations for user’s intentions from multifeedback. The proposed routing mechanism models the complex relations among multi-feedback and various user intentions, and tackles the impact of noise brought by the multi-feedback. Experimental results show that the learned disentangled representations from multi-feedback could capture comprehensive user intentions and consequently improve the recommendation performance. Moreover, the proposed curriculum further alleviates the impact of data with improper labels in different datasets by adjustable hyperparameters, which can serve as an efficient plug-in for recommendation models. Future work may include explicitly locating the noisy sample labels in multi-feedback data.

heading: Acknowledgement
text: This work is supported by the National Key Research and Development Program of China No. 2020AAA0106300 and National Natural Science Foundation of China (No. 62050110, No. 62102222).

Review summaries:
The paper proposes a curriculum disentangled recommendation model to learn disentangled representations from multi-feedback data. Specifically, the authors design a co-filtering routing mechanism to capture the relations among multi-feedbacks and denoise the representations at the feature level. Then they utilize the curriculum learning to evaluate sample difficulties to do denoising at the label level. The extensive experiments show the effectiveness of the proposed method on recommendation tasks.

The paper introduces an approach to take into account different types of positive and negative feedback in a recommendation model, while accounting for the possible noise contained in the feedback. The proposed model relies in particular on three types of feedback (click, lack of click and dislike). The sequences of different types of feedback are first encoded with a Transformer and then aggregated through a series of MLPs and attention mechanisms into vectors supposed to capture and disentangle the intents of the user.  Independently from this module, the paper also proposes a curriculum learning approach to train the model and allocate with flexibility more weight on either the easier samples or the harder samples at the beginning of the training.  The proposed approach is evaluated on 3 public datasets and 1 proprietary dataset, and compared against several baselines.

This paper proposes a model to learn disentangled representations for user intentions from complex and noisy multi-feedback. The model can capture complex relations among multi-feedback and various user intentions, and filter out feature-level noise in multi-feedback. Moreover, this paper proposes an adjustable curriculum to alleviate the impact of label-level noise in a more controllable way.