The Minority Matters: A Diversity-Promoting Collaborative Metric Learning Algorithm
Authors: Shilong Bao
Qianqian Xu
Zhiyong Yang
Yuan He
Xiaochun Cao
Qingming Huang
Venue: NeurIPS 2022 Accept (2022)
Keywords: Collaborative Metric Learning
Machine Learning
Recommendation System

Abstract: Collaborative Metric Learning (CML) has recently emerged as a popular method in recommendation systems (RS), closing the gap between metric learning and Collaborative Filtering. Following the convention of RS, existing methods exploit unique user representation in their model design. This paper focuses on a challenging scenario where a user has multiple categories of interests. Under this setting, we argue that the unique user representation might induce preference bias, especially when the item category distribution is imbalanced. To address this issue, we propose a novel method called Diversity-Promoting Collaborative Metric Learning (DPCML), with the hope of considering the commonly ignored minority interest of the user. The key idea behind DPCML is to include a multiple set of representations for each user in the system. Based on this embedding paradigm, user preference toward an item is aggregated from different embeddings by taking the minimum item-user distance among the user embedding set. Furthermore, we observe that the diversity of the embeddings for the same user also plays an essential role in the model. To this end, we propose a diversity control regularization term to accommodate the multi-vector representation strategy better. Theoretically, we show that DPCML could generalize well to unseen test data by tackling the challenge of the annoying operation that comes from the minimum value. Experiments over a range of benchmark datasets speak to the efficacy of DPCML.

The Minority Matters: A Diversity-Promotin Collaborative Metric Learning Algorithm
Shilong Bao
Qianqian Xu
Zhiyong Yang!
Yuan He
Xiaochun Cao
Qingming Huang 3,4,788
State Key Laboratory of Information Security, Institute of Information Engineering, CAS School of Cyber Security, University of Chinese Academy of Sciences Key Lab. of Intelligent Information Processing, Institute of Computing Technology, CAS School of Computer Science and Tech., University of Chinese Academy Sciences Alibaba Group
6 School of Cyber Science and Technology, Shenzhen Campus, Sun Yat-sen University Key Laboratory Big Data Mining and Knowledge Management, CAS Peng Cheng Laboratory
baoshilong@lie.com ac. en, xuqianqian@ict cn. heyuan. hy@alibaba-ind ieaibbbaainnn.. com, caoriaochun@mall. sysu. edu. en, {yangzhiyong21, qmhuang @ucas ac.
Abstract
Collaborative Metric Learning (CML) has recently emerged as a popular method in recommendation systems (RS), closing the gap between metric learning and Collaborative Filtering. Following the convention of RS, existing methods ex- ploit unique representation in their model design. This paper focuses challenging scenario where a user has multiple categories interests. Under this setting, we argue that the unique especially when the item category distribution is imbalanced. To address this is- sue, we propose novel method called Learning (DPCML), with the hope of considering the commonly ignored minority interest of the user. The key idea behind DPCML include a multiple of representations for each user in the system. Based on this embedding paradigm, user preference toward item is aggregated from different embeddings by taking the minimum item- user distance among the user embedding set. Furthermore, we observe that the diversity of the embeddings for the same user also plays an essen- tial role in the model. To this end, we propose a Diversity Control Regularization Scheme (DCRS)
representation might induce preference bias,
Collaborative Metric
accommodate the multi-vector representation strategy better. Theoretically, we show that DPCML. could generalize well to unseen test data by tackling the challenge of the annoying operation that comes from the minimum value. Experiments over range of benchmark datasets speak to the efficacy of DPCML.
Introduction
Recommender system (RS) is well-known building block in eCommerce, which can assist buyers to find products they wish to purchase by giving them the relevant recommendations The key recipe behind RS is to learn from user-item interaction records [45, 30, 31. 29, 19]. practice, since user preferences are hard to collect, such records often exist as implicit feedback [48, 1, 42] where only indirect actions are provided (say clicks, collections, reposts, and etc.). Such property of implicit
*Corresponding authors.
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

feedback raises a great challenge to RS- RS-targeted machine learning methods and thus stimulates a wave of relevant studies along this course [56, 63, 51].
Over the past two decades, most literature follows a typical paradigm known as the One-Class Collaborative Filtering (OCCF) [35], where the items not being observed are usually assumed to be of less interest for the user and labeled as negative instances. In the early days, the vast majority of studies in the OCCF community focus on Matrix Factorization (MF) based algorithms, where the preference of a specific user to an item is conveyed by the inner product between their embeddings [60, 6]. Recently, a milestone study known as Collaborative Metric Learning (CML) [18] pointed out that the inner product involved in MF violates the triangle inequality, resulting in sub-optimal topological embedding space. To fix this, CML proposes a novel framework to overcome such a problem by borrowing the strength from metric learning. Practically, CML has achieved promising performance over a series of RS benchmark datasets. Hereafter, many efforts have been made along the research direction to improve CML [44, 41, 36, 2, 54, 47, 65, 62, 43]. More discussions of the related work are presented Appendix.A.
However, through the lens of a critical example in the practical scenarios (shown in Sec.2.2), we notice that users usually have multiple categories of preferences in real-world RS. Moreover, such interest groups are often not equally distributed, where the amount of some groups dominates the others. Unfortunately, as shown in Fig.3, this case, the existing studies might induce prefer- ence bias since they tend to meet the majority interest while missing the other potential preference. Therefore, in this paper, we ask:
How to develop effective CML-based algorithu to accommodate the diversity of user preferences?
Contributions. In search of an answer, we propose a novel algorithm called Proorottig Collaborative Metric Learning (DPCML). The key idea is to explore the diversity of user interest which spans multiple groups of items. To this end, we propose a multi-vector user representation strategy, where each user has set of C' embeddings. To find out the score of a given item embedding go. we aggregate the results from the user embeddings Iu gē ... .u by taking the minimum distance s(u, v) = min gg
11? Then we will recommend the item with the smallest s value.
In this way, we can focus on all potential items that fit one of the users' interests well, both for the majority and the minority interests. Meanwhile, we observe that the diversity of the embeddings among the same user representation set also plays an important role better achieving our goal. Therefore, we further present novel diversity control regularization scheme.
Taking a step further, we continue to ask the following question:
Could CML generalize well under the multi-vector representation strategy?
To the best of our knowledge, such problem remains barely explored in the existing literature. To solve the problem, we then proceed to explore the generalization bound for DPCML algorithm. Here the major challenges fall into two aspects: 1) The risk of DPCML could not be expressed as a sum of independently identically distributed (i.i.d.) loss terms, making the standard Rademacher Complexity-band based [3, 33] theoretical arguments unavailable; 2) The annoying minimum operation are not continuous, which cannot be analyzed easily in the Rademacher complexity framework. Facing these challenges, we employ the covering number and e-net arguments to derive the general- ization bound. On top of this, we show that DPCML could induce a small generalization error with high probability. This supports the effectiveness of DPCML from theoretical perspective.
Finally, we conduct empirical studies over range of RS benchmark datasets that demonstrate the superiority of DPCML.
2 Methodology
2.1 Preliminary
In this paper, we focus on how to develop an effective CML-based recommendation system on top of the implicit feedback signals (say clicks, browses, and bookmarks). Assume there are a pool of users and items in the system, denoted by 21 = {u1, U2...., uup] and I = {01, 12 ..., ".z))

---

02
02-
2.5
(a) MovicLens-
(b) MovicLens- 10M
(a) MovieLens-IM
(b) MovieLens- 10M
Figure 1: Statisties of preference diversity.
Figure 2: The item category distribution.
respectively. For each user u, U, i 1,2, [2) let D {ui U2 ...... denote the set of items that user Uy has interacted with (i.e., observed user-item interactions) and the rest of the items (i.e,, unobserved interactions) are denoted by D., {vī 122 ..... ny Du. = Dtt U U and [Z n + n2 In the standard settings of OCCF, one usually assumes that users tend to have higher preference for the items contained in D than the items in Pu,- Therefore, given target user M4; E ll and his/her historical interaction records, the goal of RS is to discover the most interested N items by recommending the items with the top-N (bottom-N) score. The top-N item list denoted as Iy.
where ni. ny are
the number of observed/unobserved interactions user u, We have
2.2 Motivating Example
We start by definition of the preference diversity of users.
Definition 1 (Preference Diversity). As- sume that there exists an attribute set T T(2)) T() in typi- cal RS, where T(e;) {t,, 12, 22.,2,,,7777 .,,,,,,,, contains the attribute information of item Uy (e.g., the genres of movie) and T, is the number of attributes. Given a user u; and interaction records D the preference diversity is defined follows:
Horor
SG+
HorrY
Uur Eerrendding
enteent
pestrnnce
Preference
(6) OPCML [Ours)
Figure An illustration shows the benefit of our proposed algorithm when user has multiple categories of preferences. Taking example, we assume that Sci- Fi/Horror is the majority/minority interest of the user while Cartoon is an irrelevant movie type. It is easy to see that if the item embed- dings distributed shown in the figure, we can hardly find a single user embedding that simultaneously captures both interests.
Σ
(T(()))TTv) Ø0
Div(u:)= = NY.MEDD myy Pn
[Dt, [[Dit -
movies
where I(xx) is an indicator function, i.c., returns 1 if the condition æ holds, otherwise is returned.
Remark 1. Intuitively, the range of Div(u:) among [0, 1], and its value measures the diversity of U, preference to a certain extent. That is say, if items among the historical interaction records of users are irrelevant, there should induce large value (e.g., Div(u;) = 1), implying the diversity of their preferences. If the opposite is the case, the value small. This means users may have narrow interests where only some unique attributes appeal to them.
Based on Def.1, we visualize the user preferences on two real-world benchmark datasets, including MovieLens-Ini and MovieLens-100 Appendix.C. C. Here we adopt the movie genres as the attribute set T because such information is easy obtain. The results are shown in Fig. 1. From the results, we can make the following observations. First, only a few users have limited interest. Moreover, most of the users have diversity value spaning (0, 0.8), suggesting that they have multiple categories interests. Finally, one can notice that there are very few users with high preference diversity (at the lower-right corner) in both figures. This is a convincing case in the real-world recommendation since most users usually have interests in couple of movic genres but not all.
The detailed information of datasets listed in Tab. in

---

Motivation and Discussion. Through the above example, the key information is that users usually have multiple categories of preference in real-world recommendations. This poses critical chal- lenge to the current CML framework. Specifically, following the convention of RS, the existing CMI.-based methods leverage unique representations of users to model their preferences. Facing the multiplicity of user intentions, such paradigm may induce preference bias due to the limited expressiveness, especially when the item category distribution is imbalanced. Fig.2 visualizes the item distribution on MovieLens-Im and MovieLens- 10m datasets. We see that both of them are imbalanced. this case, as shown in Fig.3-(b) CML would pay more attention to the majority in- terest of users making the unique user embedding close the items with the science fiction (Sci-Fi) category. In this way, the minority interest of the user (i.e., Horror movies) would be ignored by the method, inducing performance degradation. This motivates us to explore strategies on top of CMI..
2.3 Seversity-Promotie Collaborative Metric Learning
Recall that the critical recipe behind CML-based algorithms is to seek a metric space (usually adopt- ing the Euclidean space) such that user preferences could be naturally specified by their distance toward different items. To do this, the traditional CML-based methods usually represent each user and each item as a vector, respectively. Different from them, taking the preference diversity of users into account, we propose adopt C' (C > 1) different embeddings for each user and represent each item as one single vector in a joint Euclidean space.
Concretely, each user u, is projected into the metric space via the following lookup transformations [49, 50, 55]:
gú, P. u. Ve, ui, c [C], uN U, where g.. E 'd is a representation vector of user us: [C] is the set {1, (1......,C), P. E M is a learned transformation weight; d is the dimension of space and u; E Blu|| is one-hot encoding that the nonzero elements correspond its index of particular user u4. Similarly, we apply the following transformation to each item ey:
go, = Q Uj Vuy E I,,
2)
where 9r, E Rd is the embedding of item My; Q IR Z|xd is the learned transformation weight and By R is one-hot embedding of item dz-
In what follows, given a target user u;, we need to find out a score function to express the user preference toward an item in the context of multiple representations of users. Here we define the score function by taking the minimum item-user Euclidean distance among the user embedding set:
(N; 03) = min |g Ivy 11 Vuz E I. cE(C)
Equipped with this formulation, focus the potential items that fit one of the user preferences. If user ue has interacted with item Uy. there should be a small value with respect to s sw, v2). If the opposite is the case, then expect should be satisfied to reflect the relative preference of in the learned Euclidean space:
large s(u, vy). Mathematically, the following inequality
s(u,, n < s(u;, Vos E Di,,, Vuz E D,,,"
(4)
Therefore, given the whole sample set D = U
Duy, we adopt the following pairwise learning
u.Elt
problems [18, 44, 24, 57] to achieve such goal:
min RD..g
(5)
where, Vogi D.t, u. Vog E Di,, we have
RD
1
Σ
(ot
u
n
Th
M.Eu
k=

---

(o 0x) = max(0, A s(u,, 2/t) s(u,, U/ v)))
(6)
and À > a safe margin.
According to (5), we have the following explanations. At first, optimizing the above problem could pull the observed items close to the users and push the unobserved items apart from the observed items. This achieves our goal of preserving user preferences in the Euclidean space. Then, as shown in Fig.3-(c), equipped with multiple set of representations for each user, DPCML would exploit different user vectors to focus on different interest groups. In this sense, the minority interest groups can also be modeled well. Last but not least, one appealing property is that, DPCMI, also preserves the triangle inequality for the items falling into the same interest group.
2.4 Diversity Control Regularization Scheme
In practice, we note that a proper regularization scheme is crucial to accommodate the multi-vector representation strategy. Here we focus on the diversity within the embedding sets of given user. Such diversity is defined as the average pairwise distance among the C' user embeddings for user i.c.
0g M 2C(C-1) c1,C2E|C]
Σ
19 golll
Based on the definition, argue that one should attain proper ōg.u, to get a good performance since extremely large/small values of d might harmful for the generalization error. It easy to see that if 6g.u, is extremely small, the embeddings for given user very close to each other such that the multi-vector representation strategy degenerates to the original single-vector represen- tation. This increases the model complexity with few performance gains and obviously will induce overfitting. On the other hand, a too large diversity might also induce overfitting. might be a bit confusing at first glance. But, imagine that when some noise observations extremely interests far away from the normal patterns exist the data, having a large diversity will make easier to overfit such data. Moreover, it is also natural assumption that user's interests should not be too different, as validated in Fig.1. 1. In this sense, the distance across different user embeddings should remain moderate magnitude.
Therefore, controlling a proper diversity is essential for the multi-vector representation. To do this, put forward the following diversity control regularization scheme (DCRS):
ÔD =
(y()).
I
Ni
where, we have
(1:) max (0, 61 )g max (0, ōg.u; -52,
and d, 82 are two threshold parameters with d3 I 62. Intuitively, optimizing (7) ensures that the diversity of user's vectors lies between d1 and 82.
2.5 Optimization
Finally, we arrive at the following optimization problem for our proposed DPCML.:
min Cp(g) == Ro. + n Ω.p.
(8)
where y trade-off syper-parameter.
When the training is completed, one can easily carry out recommendations by choosing the items with the smallest s s(u, vy), Vuj, 0j I.
2.6 General Framework of Joint Accessibility
Now, we expect to provide another intriguing perspective of our proposed method. As we discussed in Aec.A.2, equipped with multiple set of representations for each user, our proposed algorithm could be treated generalized framework against the joint accessibility issue. To see this, if
5

---

restrict the user and item embeddings within a unit sphere, then the score function (3) degenerates to
s(u, 0ÿ) min (1 -G,g nE(C]
s.t. lg 1, Vu; U,
II 1, Voy E
where Iú, E xd represents the transpose vector of g. E R Therefore, to minimize (9), one only needs maximize the following equivalent problem:
8 (u4, Ug) max cE(C]
s.t.
l 1, Vu, UI
(10)
1g,, 1, Vuj EI,
which is exactly the original form of the joint accessibility model.
3 Generalization Analysis
In this section, we present a systematic theoretical analysis of the generalization ability of our pro- posed algorithm. Following the standard learning theory, deriving uniform upper bound of the generalization error relies on the proper measure of complexity over the given hypothesis space H. The most common complexity to achieve this is the Rademacher complexity [3, 33, 23], which is derived from the symmetrization technique as an upper bound for the largest deviation over a given hypothesis space H::
ED
sup (D(RR) RD
SEH
However, the standard symmetrization technique requires the empirical risk RD to be a sum of inde- pendent terms, which is not applicable for the CML-based methods since they usually involve sum of pairwise terms in (5). For instance, with respect to (5), the terms ((1) (v) 2y and ef (u) (g)
are interdependent long one of them is the same (i.e., 2i 6j ur 0g
Therefore, we turn to leverage another complexity measure, i.c. covering number, to overcome this difficulty. The necessary notations summarized as follows.
Definition 2 (e-Covering). [21] Let (F,p) p) be (pseudo) metric space, and G S F.F {fi..... fx is
said be an e-covering of i Ú B(fs,e), i.e., Vg G. 3i such that p(g, fi) S
i=1
Definition (Covering Number). [21] According to the notations Def.2, the covering number of G with radius is defined as:
N(c9. min Ee covering over 9 with size n}
With the above definitions, we further have the following assumption and lemma to help us derive the generalization bound.
Assumption 1 (Basic Assumptions). We assume that all the embeddings of users and items are chosen from the following embedding hypothesis space:
HR = (g:g9 Rd llell <r}.
(11)
where gú, H.R. u, E U, c [C] and go, E HR. Dy E I.
Lemma 1. [28, 25, 64] The covering number of the hypothesis class HR has the following upper bound:
3r
log (e; Hn: p) < dllog
(12)
where d is the dimension of embedding space.

---

Based on the above introductions, we have the following results. Due to space limitations, please refer to Appendix. for all proofs in detail.
Theorem 1 (Generalization Upper Bound of DPCML). Ler [((D(g)) be the population risk of LD(g). Then, V g E H.s. with high probability, the following inequation holds:
2d 3r.
(D(g) (E(E(EE(
S\
(13)
N
where we have
(4 (4+ 22
Ñ=
4r2
22
Intriguingly, we see that our derived bound does not depend C. This is consistent with the over-paramettrrizaion phenomenon [8, 34J. On top of 1hm.1, we have the following corollary.
Corollary 1. DPCML could enjoy a smaller generalization error than CML.
Therefore, conclude that DPCML generalizes to unseen data better than single-vector CML and thus improves the recommendation performance. This supports the superiority of our proposed DPCML from theoretical perspective. In addition, we also empirically demonstrate this in the experiment Sec.4.3 and Appendix.C.6.44
Table 1: Basic Information of the Datasets. %Density is defined as
# Ratings #U sersx #leess
X 100%.
Datasets
MovieLens- IM
Steam-200k
CiteULike-T
MovieLens-10M
Domain
Movie
Game
Paper
Movie
#Users
6,034
3,757
5,219
69,167
#ltems
3,953
5,113
25,975
10,019
#Ratings
575,271
115,139
125,580
5,003,437
%Density
2.4118%
0.5994%
0.0926%
0.7220%
Experiments
In this section, proposed method is applied to wide range of real-world recommendation datasets to show its superiority. Please refer to Appendix. C for more results about experiments.
4.1 Experimental Setups
To begin with, we perform empirical experiments several common recommendation benchmarks: MovieLens-Ing Steam-200k, CiteULike and MovieLens- 00m. The detailed statistics terms of these datasets summarized in 1.b.1. For the datasets with explicit feedbacks, follow the previous works [14, 44] and transfer them into implicit feedback. Secondly, we evaluate the performance with five metrics. including Precision (P@ N). Recall (R @ N), Normalized Discounted Cumula- tive Gain (NDCG@ @ N), Mean Average Precision (MAP), and Mean Reciprocal Rank (MRR). Moreover, compared proposed method with 14 competitive competitors: a) Item-based CF method, itemKNN [27]: b) MF-based algorithms, including the combination of MF and deep Iearn- ing models and multi-vector MF-based approaches: GMF, MLP, NeuMF [14], M2F [12] and MGMF [12]; c) CaL-based methods, including UniS [35], PopS [53], 2stS [44], HarS [18, 11],
TransCF [36], LRML [41], AdaCML [62] and HLR [43].
4.2 Overall Performance
The experimental results of all the involved competitors are shown in Tab.2 and Tab.5 (in Ap- Ce5. Consequently, we can draw the following conclusions: 1) In most cases, the best
7

---

Table 2: Performance comparisons MovieLenns lm and Steam-200k datasets.
Type
Method
NDCG
P665
NDCG@
Item-based
itemKNN
12.24
2.90
12.41
12.43
4.29
12.79
8.34
26.16
GMF
14.10
2.81
14.33
14.28
4.08
14.73
8.29
29.51
MI.P
13.95
2.78
14.22
14.06
3.98
14.56
8.30
29.39
MF-based
NeuMF
16.43
3.20
16.87
16.73
4.68
17.40
9.69
33.23
M2F
8.61
1.84
9.36
7.60
2.30
8.67
2.95
20.40
MGMF
17.38
3.51
18.08
17.63
5.05
18.52
10.12
35.15
UniS
18.34
18.79
35.77
MovieLens-Im
PopS
12.96
3.11
13.30
12.82
4.41
13.40
7.59
28.61
2st8
21.07
4.84
21.35
21.81
7.07
22.29
14.42
40.36
HarS
24.88
25.38
24.89
8.25
25.77
15.74
45.15
CMI-hased
TransCF
10.03
1.84
10.31
10.90
3.09
11.20
7.07
23,66
LRMI.
17.15
3.52
17.56
17.45
5.12
18.08
10.42
34.36
AdaCML
19.06
4.12
19.31
19.74
6.23
20.20
13.30
37.36
HLR
21.10
4.80
21,53
21.61
7.06
22.28
13.95
40.71
DPCML1
19.12
4.14
19.34
19.90
6.27
20.29
13.24
37.55
Ours
DPCM122
25.18
25.35
8.51
45.32
hem-based
itemKNN
12.58
9.47
13.23
6.47
3.90
7.23
11.74
23.33
GMF
12.57
6.17
13.29
14.22
6.86
15.39
9.72
28.38
ML.P
17.07
9.63
17.49
16.89
8.49
17.67
15.15
34.54
MF-based
NeuMF
17.36
17.95
17.41
8.79
18.45
15.11
35.55
M2F
11.33
5.69
11.95
11.44
5.73
12.98
6.43
25.05
MGMF
12.51
13.25
14.45
6.88
15.55
9.63
28.40
UniS
20.92
Steam-200k
PopS
18.05
11.58
18.76
14.94
7.98
15.78
15.13
34.04
2st8
25.20
14.62
26.20
23.97
11.91
25.35
21.48
46.17
HarS
26.66
27.93
24.94
12.78
26.63
23.25
48.84
CMI-based
TransCF
13,30
6.61
13.58
15.26
7.09
15.89
11.08
26.29
LRMI.
14.91
7.48
15.43
16.49
8.06
17.51
12.24
31,89
AdaCML
23.02
23.38
22.35
11.31
23.23
19.88
42,03
HLR
20,30
11.65
20.96
19.79
9.88
20.94
17.06
39.26
DPCMLI
25.39
14.84
26.56
23.88
12.11
25.25
22.26
46.79
Ours
DPCML2
29.88
28.70
51.95
UniS
HarS
DPCML1
DPCML2
20
MAP(%)
15
10
5
Drama
Comady
Actio
Thiller
Adventure
Romance
Crima
sci-fi
Fanmayy
Children
Mystory
War
Homor
Animation
Musical
Wessen
Film-tul Holl
Decumentrr
IMA
Figure 4: Fine-grained performance over each interest group MovieLens- 10m dataset.
performance of CML-based methods consistently surpasses the best MF-based competitors. This suggests that it is necessary to develop CML-based RS algorithms. 2) Our proposed method con- sistently surpasses all the competitors significantly on all datasets, except the results for MAP and MRR on CiteULike. Even for the failure results, the performance fairly competitive compared with the competitors. This shows the effectiveness of our proposed algorithm. 3) Compared with studies targeting joint accessibility (i.c., M2F and MGMF), our proposed method significantly out- performs M2F and MGMF all benchmark datasets. This shows the advantage of the CMs-based paradigm that deserves more attention along this direction in future work.
4.3 Quantitative Analysis
Fine-grained Performance Comparison. Fig.4 presents the MAP metric over each interest group (movie genre) MovieL
We can observe that our proposed framework could not only significantly outperform their single-vector counterparts the majority interests but also improve the performance of minority groups in most cases. Especially, compared with HarS, the performance

---

Table 3: The diversity performance comparison of CML-based algorithms on Beeam-200k and MovieLens- Im datasets. Here a higher value implies more diverse recommendation results.
Steam-2008
Method
MaxDiv@2
MaxDiv@10 MaxDiv@5
MaxDiv@20
UniS
1.354
4.750
23.520
117.927
HarS
1.752
6,809
40.378
236.794
DPCMLI w/o DCRS
1.633
5.857
30.425
155.193
DPCMLI
18222
6.713
34.727
179.065
DPCML2 w/o DCRS
2.958
11,398
65.398
365,458
DPCML2
2.977
11.42
65.952
369.876
MovieLens- Im
UniS HarS
1.739
6,142
30.127
140.095
2.443
8.826
46,390
244.078
DPCMLI w/o DCRS
1623
5.857
29,500
140,057
DPCMLI
1.744
6,195
30.755
145.615
DPCML2 w/o DCRS
2,827
10.423
55.612
292.089
DPCML2
3.144
11.498
60.696
313,086
improvement of DPCML2 on minority interests is sharp. This shows that DPCML could reasonably focus on potentially interesting items even with the imbalanced item distribution.
Recommendation Diversity Evaluation. We test the performance of DPCML against CML- based competitors with max-sum diversification (MaxDiv) [4]. The diversity results shown in Tab.3. We observe that: a) For methods within the same negative sampling strategy (i.e., UniS and DPCML1, HarS and DPCML2), values. This suggests the improvement of DPCML in terms promoting recommendation diver- sity. b) In most cases (except for DPCML1 w/o DCRS the MovieLens- Im dataset), DPCML outperforms other competitors even without regularization. c) Most importantly, equipped with the regularization term DCRS, DPCML could achieve better diversification results against w/o DCRS.
proposed DPCML could achieve relatively higher max-sum
Effeet of the Diversity Control Regularization. Fig.5 illustrates a 3D-barplot based on the results observe that the proposed regularization
of grid search Steam-200k. From the results, scheme could significantly boost performance on all metrics. Moreover, there would induce different performances with different diversity values. This suggests that controlling proper diversity of the embeddings for the same user is essential to accommodate their preferences better.
Empirical Justification of Corol.1. Fig.7 shows the empirical results on Heam-22004 dataset. Based on these results, we can see that, with increase of C, the empirical risk (i.e., training loss) of DPCML (C > 1) is significantly smaller than CML (C 1). In addition, DPCML could substantially improve the performance the validation/test set. Thus, we can conclude that DPCMI. could induce smaller generalization error than traditional CML. This is consistent with 1
Sensitive Analysis of C. Fig.8 demonstrates the performance of DPCML methods with different C on Steam-200k dataset. We observe that a proper C' could significantly improve the performance. Besides, leveraging too aggressively for DPCML2 may adversely hurt the performance since models optimized with hard samples are more likely to lead to the over-fitting problem with the increasing parameters.
Sensitivity analysis of 1). We investigate the sensitivity of E (0. 1,31, 5, 10, 20, 30) for recommen- dation results on the Steam-200k dataset. The experimental results listed in Tab.6 and Tab.7. We can conclude that a proper (roughly 10) could significantly improve the performance, suggesting the essential role of the proposed diversity control regularization scheme.
Training Efficiency. Since DPCML includes multiple user representations, it will inevitably intro- duce extra complexity to the overall optimization. We further investigate the efficiency of our pro- posed algorithm, presented in Fig.6. This trend suggests that our proposed algorithm could achieve competitive performance with acceptable efficiency.
Ablation Studies DCRS. In order to show the effectiveness of our proposed DCRS, we compare its performance with the its three variants: a) w/o DCRS. b) DCRS 8, and c) DCRS- 82. Please refer to Appendix. C.6.8 the details of each variant. The empircal results on Steam-200k dataset are provided in Tab.8 and Tab.9. From the above results, we can see that: In most cases, only

---

(a) DPCMLI (MRR) (b) DPCML2 (MRR)
(a) CiteULike
(b) MovieLens- 10m
Figure 5: Sensitivity analysis about d and 82 on Steam-200k datasets.
Figure 6: Training efficiency comparison
among CML-based competitors.
540
3.7
22.4
25.
(a) DPCMLI
(b) DPCML2
(a) P@3
(b) P@5
Figure 7: Empirical justification of Corol.1 Figure 8: Sensitive Analysis of different C.
employing one of the two terms of DCRS could still improve the recommendation performance. However, none of them could outperform our proposed method. This strengthens the effectiveness of our proposed regularization scheme.
DCRS for MF-based Systems. We attempt to apply the proposed diversity control regularization scheme (DCRS) for M2F [52, 12]. In addition, we further explore the effectiveness of DCRS for the general framework of joint accessibility (GFJA, (q) in the main paper). The experimental results are summarized in Tab. 10. The overall performance suggests the superiority of our proposed method against the current multi-vector-basse competitors.
5 Conclusion & Future Remarks
This paper pays attention to developing an effective CML-based algorithm when users have multiple categories of interests. First, point out that the current CML framework might induce preference bias, especially when the item category distribution is imbalanced. To this end, we propose a novel algorithm called DPCML. The key idea is to include multiple representations for each user in the model design. Meanwhile, a novel diversity control regularization scheme is specifically tailored to serve purpose better. To see the generalization ability of DPCML unseen test data, also provide high probability upper bounds for the generalization error. Finally, the experiments over range of benchmark datasets speak the efficacy of DPCML. However, following the paradigm of CML, a possible limitation of DPCML is that it generally applies to implicit feedback but not explicit feedback since CML only cares about the relative preference ranking instead of concrete magnitude. In the future, we will explore how to improve the recommendation diversity based on explicit feedback.
Acknowledgements
This work was supported in part by the National Key R&D Program of China under Grant 2018AAA0102000, in part by National Natural Science Foundation of China: U21B2038, 61931008, 62025604, 61971016. 6212200758 and 61976202, in part by the Fundamental Re- search Funds for the Central Universities, in part by Youth Innovation Promotion Association CAS, in part by the Strategic Priority Research Program of Chinese Academy of Sciences, Grant N028002000000 in part by the China National Postdoctoral Program for Innovative Talents under Grant BX2021298, and in part by China Postdoctoral Science Foundation under Grant 2022M713101.
10

---

References
[1] Askari, B., Szlichta, J., and Salchi-Abari, A. Variational autoencoders for top-k recommenda- tion with implicit feedback. In SIGIR, pp. 2061-2065, 2021.
[2] Bao, S., Xu, Q.. Ma, K., Yang, Z., Cao, X., and Huang, Q. Collaborative preference embedding against sparse labels. In ACM MM, 2079-2087, 2019.
[3] Bartlett P. L. and Mendelson, S. Rademacher and gaussian complexities Risk bounds and structural results. In COLT, volume 2111, pp. 224-240, 2001.
[4] Borodin, A., Lee, H. C., and Ye, Y. Max-sum diversification, monotone submodular functions and dynamic updates. In SIGMOD/PODS, pp. 155-166, 2012.
[5] Canévet, O. and Fleuret, Efficient sample mining for object detection. ACML, pp. 48-63, 2014.
[6] Chen, J., Lian, D., and Zheng, K. Improving one-class collaborative filtering via ranking-based implicit regularizer. In AAAI, pp. 37-44, 2019.
[7] Curmei, M., Dean, S., and Recht, B. Quantifying availability and discovery in recommender systems via stochastic reachability. In ICML., pp. 2265-2275, 2021.
[8] Dar, Y., Muthukumar, V., and Baraniuk, R. G. A farewell to the bias variance tradeoff? an overview of the theory of overparameterized machine learning. 2021.
[9] Dean, S., Rich, S., and Recht, B. Recommendations and agency: the reachability of collaboratively-filee filtered information. In FAT, pp. 436 445, 2020.
[10] Ding, J., Quan, Y., He, X., Li, Y., and Jin, D. In IJCAI, pp. 2230-2236, 2019.
[11] Gajic, B., Amato, A., and Gatta, C. Fast hard negative mining for deep metric learning. Pattern Recognition, 10::957795, 2021.
[12] Guo, W., Krauth, K., Jordan, M. I., and Garg, N. The stereotyping problem in collaboratively filtered recommender systems. EAAMO, pp. 6:1 6:1-6:10,
10, 2021.
[13] He, X., Zhang, H., Kan, M., and Chua, T. Fast matrix factorization for online recommendation with implicit feedback. In SIGIR, pp. 549-558, 2016.
[14] He, X., Liao, L., Zhang. H., Nie, L., Hu, X., and Chua, Neural collaborative filtering. In WWW, pp. 173-182, 2017.
[15] He, X., Du X., Wang, X., Tian, F., Tang, J., and Chua, T. Outer product-based neural collabo- rative filtering. In IJCAI, pp. 2227-2233, 2018.
[16] Heckel, R. and Ramehandran, K. The sample complexity of online one-class collaborative filtering. In ICML, pp. 1452-1460, 2017.
[17] Henriques, J. F., Carreira, Caseiro, R., and Batista, Beyond hard negative mining: Efficient detector learning via block-circulant decomposition. In ICCV, pp. 2760-276) 2013.
[18] Hsieh, C.-K., Yang. I.., Cui, Y., Lin, T.-Y., Belongie, S., and Estrin, D. Collaborative metric learning. In WWW, pp. 193-201, 2017.
[19] Jiang, M., Cui, P., Chen, X., Wang, F., Zhu, W., and Yang. S. Social recommendation with cross-domain transferable knowledge. IEEE TKDE, 27(11):3084 3097, 2015.
[20] Kingma, D. and Ba, J. Adam: A method for stochastic optimization. In ICLR, 2015.
[21] Ledoux, M. and Talagrand, M. Probability in Banach Spaces: isoperimetry and processes. 1991.
[22] Lee, D., Kang, S., Ju, H., Park, C., and Yu, Bootstrapping user and item representations for one-class collaborative filtering. In SIGIR, pp. 1513-1522, 2021.

---

[23] Lei, Y., Ding. L., and Bi, Y. Local rademacher complexity bounds based on covering numbers. Neuroconputiing. 218:320-330, 2016.
[24] Lei, Y., Ledent, A., and Kloft, M. Sharper generalization bounds for pairwise learning. In NeurlPs, 2020.
[25] Li, S. and Liu, Y. Sharper generalization bounds for clustering. In ICML, pp. 6392 6402, 2021.
[26] Li, Z., Xu, Q. Jiang, Y., Ma, K., Cao. X., and Huang, Q. Neural collaborative preference learning with pairwise comparisons. IEEE Trans. Multim.. 23:1977-1989, 2021.
[27] Linden, G., Smith, B., and York, Amazon. com recommendations: Item-to-item collabora- tive filtering. IEEE Internet computing, (1):76-80, 2003.
[28] Long, P. M. and Sedghi, H. Generalization bounds for deep convolutional neural networks. In ICLR 2020, 2020.
[29] Lv, Y., Zheng, Y., Wei, F,, Wang, C., and Wang, C. AICF: attention-based item collaborative filtering. Adv. Eng. Informatics, 44:101090:1-11, 2020.
[30] Ma, J., Zhou, C., Cui, P., Yang, H., and Zhu, W. Learning disentangled representations for recommendation. In NeurlPS, pp. 5712-5723, 2019.
[31] Ma. J., Zhou, C., Yang, H., Cui, P., Wang. X., and Zhu, W. Disentangled self-supervisinn in sequential recommenders. KDD, pp. 483-491, 2020.
[32] McDiarmid, C. Concentration. In Probabilistic methods for algorithmic discrete mathematics, pp. 195-248. 1998.
[33] Mohri, M., Rostamizadeh, A., and Talwalkar, Foundations of Machine Learning. MIT Press, 2012.
[34] Nakkiran, P., Kaplun, G., Bansal, Y., Yang, T. Barak, B., and Sutskever, I. Deep double descent: Where bigger models and more data hurt. In ICLR, 2020.
[35] Pan, R., Zhou, Y., Cao, B., Liu, N. N., Lukose, R. M., Scholz, M., and Yang, Q. One-class collaborative filtering. In ICDM, pp. 502-511, 2008
[36] Park, C., Kim, D., Xie, X., and Yu, H. Collaborative translational metric learning. In ICDM, pp. 367-376, 2018.
[37] Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z., Desmaison, A., Antiga, L., and Lerer, A. Automatic differentiation in pytorch. 2017.
[38] Rendle, S. and Freudenthaler, C. Improving pairwise learning for item recommendation from implicit feedback. In WSDM, pp. 273-282, 2014.
[39] Rendle, S., Freudenthaler, C., Gantner, 7.,, and Schmidt-Thieme, I.. BPR: bayesian personal- ized ranking from implicit feedback. In UAI, pp. 452-461, 2009.
[40] Takács, G. and Tikk, D. Alternating least squares for personalized ranking, In RecSys, pp. 83-90, 2012.
[41] Tay, Y., Tuan, L. A., and Hui, C. Latent relational metric learning via memory-based atten- tion for collaborative ranking. In WWW, pp. 129-739, 2018.
[42] Togashi, R., Kato, M., Otani, M., and Satoh, S. Density-ratio based personalised ranking from implicit feedback. In WWW, pp. 3221-3233, 2021.
[43] Tran, V., Salha-Galvan, G., Hennequin, R., and Moussallam, M. Hierarchical latent relation modeling for collaborative metric learning. In RecSys, pp. 302-309, 2021.
[44] Tran, V.-A., Hennequin, R., Royo-Leteller, J., and Moussallam, M. Improving collaborative metric learning with efficient negative sampling. In SIGIR, pp. 1201-1204, 2019.
12

---

[45] Wang, C., Zhou, T., Chen, C., Hu, T., and Chen, G. Off-policy recommendation system without exploration. In PAKDD, volume 12084, pp. 16-27. Springer, 2020.
[46] Wang, H., Chen, B., and Li, W. Collaborative topic regression with social regularization for tag recommendation. In ICAI, pp. 2719-2725, 2013.
[47] Wang, H., Li, Y., and Frimpong, F. Group recommendation via self-attention and collaborative metric learning model. IEEE Access, 7:164844-164855, 2019.
[48] Wang, M., Gong, M., Zheng, X., and Zhang. K. Modeling dynamic missingness of implicit feedback for recommendation. In NeurIPS, pp. 6670-6679, 2018.
[49] Wang, W., Feng. F., He, X., Nie, L., and Chua, Denoising implicit feedback for recommen- dation. In WSDM, pp. 373-381, 2021.
[50] Wang, X., He, X., Wang, M., Feng, F., and Chua, T. Neural graph collaborative filtering, In ACM SIGIR, pp. 165-174, 2019.
[51] Wang, X., Wang, R., Shi, C., Song, G., and Li, Q. Multi-component graph convolutional collaborative filtering. In AAAI, pp. 6267-62744 2020.
[52] Weston, J., Weiss, R. J., and Yee, H. Nonlinear latent factorization by embedding multiple user interests. In RecSys, pp. 65-68, 2013.
[53] Wu, G., Volkovs, M., Soon, C. L., Sanner, S., and Rai, H. Noise contrastive estimation for one-class collaborative filtering. In SIGIR, pp. 135-144, 2019.
[54] Wu, H., Zhou, Q., Nie, R., and Cao, Effective metric learning with co-occurrence embedding for collaborative recommendations. Neural Networks, 124:308-318, 2020.
[55] Wu, Q., Zhang, H., Gao, X., Yan, J., and Zha, H. Towards open-world recommendation: An
inductive model-based collaborative filterigg approach. In ICML, pp. 11329-1 11399, 2021.
[56] Xu, D., Ruan, C., Körpeoglu, E., Kumar, S., and Achan, K. Rethinking neural vs. matrix- factorization collaborative filtering: the theoretical perspectives. In ICML, pp. 11514-11524, 2021.
[57] Yang, T. and Ying, Y. AUC maximization in the era of big data and AI: A survey, ACM Computing Surveys, 2022.
[58] Yang, Z., Ding, M., Zhou, C., Yang, H., Zhou, J., and Tang, J. Understanding negative sam- pling in graph representation learning. In KDD, pp. 1666-1676, 2020.
[59] Yao, Y., Tong, H., Yan, G., Xu, F., Zhang, X., Szymanski. B. K., and Lu, J. Dual-regularized one-class collaborative filtering with implicit feedback. www, 1293):1099-11 129, 2019.
[60] Zhang, Q. and Ren, F. brior-based bayesian pairwise ranking for one-class collaborative filter- ing. Neurocomputing, 440:365-374, 2021.
[61] Zhang, Q. and Ren, Double bayesian pairwise learning for one-class collaborative filtering. Knowl. Based Syst., 229: 107339, 2021.
[62] Zhang, T., Zhao, P., Liu, Y., Xu, J., Fang. J., Zhao, L., Sheng, V. S., and Cui, Z. Adacml: Adaptive collaborative metric learning for recommendation. DASFAA, volume 11447, pp- 301-316, 2019.
[63] Zheng, Y., Tang, B., Ding, W., and Zhou, H. A neural autoregressive approach to collaborative filtering. In ICML, pp. 764-773, 2016.
[64] Zhou, D. The covering number in learning theory. J. Complex., 18(3):739-767, 2002.
[65] Zhou, X., Liu, D., Lian, J., and Xie. X. Collaborative metric learning with memory network for multi-relational recommender systems. In IJCAI, pp. 4454 4460, 2019.
13

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [N/A]
(c) Did you discuss any potential negative societal impacts of your work? [No] There are no immediate societal impacts associated with our research.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes]
(b) Did you include complete proofs of all theoretical results? [Yes] See the supplemen- tary materials.
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either the supplemental material or as URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Sec.C.4 in the Appendix.
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [N/A]
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [N/A]
4. If you are using existing assets (e-g., code, data, models) or curating/releasing new assets...
(a) your work uses existing assets, did you eite the creators? [Yes]
(b) Did you mention the license of the assets? [Yes] See Sec.C
(c) Did you include any new assets either in the supplemental material or as a URL? [N/A]
(d) Did you discuss whether and how consent obtained from people whose data you'l re using/curating? [No] We use open-source software and datasets.
(c) Did you discuss whether the data you are using/curating contains personally identifi- able information or offensive content? [N/A]
5. If you used crowdsoureing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [N/A]
14


Reviews:
[{"review_id": "lOd8gVbasjP", "reviewer": "Reviewer_Jwxi", "summary": "This work attacks the collaborative metric learning (CML) based recommendation algorithms. The critical point is that the current studies might overlook the minority taste of users in practice due to the limited representation capability, leading to inevitable preference degradation. Based on this recognition, this paper presents a simple and effective approach (i.e., Diversity-Promoting CML, DPCML) to alleviate this preference bias. Then, a decent theoretical guarantee is derived, suggesting the superiority of DPCML against CML in theory. Finally, empirical studies ascertain the effectiveness of the proposed method.", "questions": "The authors should correct my minor concerns listed in the weakness to make their work more complete. ", "limitations": "There are no potential societal impacts of this paper.", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\n1.\tClear motivation and good novelty.  \n2.\tSound/important theoretical analysis. The provided theoretical results present novel insights into the strengths of DPCML against the current CML-based recommendation approaches. This is challenging to derive the generalization bound and may arise much more research attention in the community. \n3.\tConvincing validations. The authors conduct comprehensive experiments to show the effectiveness of DPCML against the current CML-based algorithms, and the experiment results seem promising.\n4.\tThe presentation of this paper is good and easy to understand.\n\nWeakness:\nThe writing of this paper is roughly good but could be further improved. For example, there are a few typos and mistakes in grammar:\n1.\tRow 236 in Page 4, “…show its superiority.”: I think this sentence should be polished.  \n2.\tRow 495 in Supp. Page 15: “Hard” should be “hard”.\n3.\tRow 757 in Supp. Page 29: “…training/validation/test” should be “…training/validation/test sets”. \n4.\tRow 821 in Supp. Page 31: “Fig.7” should be “Fig.12”.\nLast but not least, each theorem and corollary appearing in the main paper should be attached to its corresponding proof link to make it easy for the reader to follow.\n\nThe primary concerns are motivation, methodology soundness, and experiment persuasion. I believe this is a qualified paper with good novelty, clear theoretical guarantees, and convincing empirical results. \n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "Lwj9B9_UL1J", "reviewer": "Reviewer_1VF6", "summary": "This paper focuses on Collaborative Metric Learning (CML) based recommendation systems (RS). The authors point out that the current CML-based studies might result in a limited recommendation performance due to the insufficient consideration of the multiple categories of user preferences in practical RS. To alleviate this problem, they propose a novel algorithm named Diversity-Promoting Collaborative Metric Learning (DPCML). The secret of success behind the DPCML is to design multiple representations for each user to focus on different interest groups (i.e., both majority and minority). Going a step further, the authors theoretically prove that the proposed DPCML outperforms the existing CML-based methods and could generalize well to unseen test data. Finally, experiments over a wide range of benchmark datasets demonstrate the efficacy of DPCML.", "questions": "1. Some parts of the paper should be reorganized to help readers understand this work better.\n2. The authors should complement a further explanation of the Joint Accessibility problem to make this concept concise and understandable.\nMore questions should be found in the above weakness part.\n", "limitations": "None.", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The motivations and contributions of this paper are clear. \n2. The proposed method is novel. Concretely, this work presents a simple but effective method, DPCML, which leverages multiple user representations to capture user interests and adopt a diversity control regularization term to serve their purpose better.\n3. Significantly, this paper is technically valuable. The authors further demonstrate the effectiveness of the proposed algorithm from a challenging theoretical perspective. In my opinion, the generalization analysis of CML-based algorithms is not an easy task, which is barely explored and could motivate more effective work in this direction. \n4. The experiment is persuasive.  They conduct comprehensive experiments on four benchmarks and compare DPCML with 14 state-of-the-art competitors. The empirical results show the superiority of DPCML.\n\nWeaknesses:\n1. Some parts of the paper should be reorganized to help readers understand this work better. For example, the proof of Corollary 1 should be mentioned in the main paper. Moreover, the empirical verification should be followed with Corollary 1 at the end of that part. \n2. There are some typos in this paper, such as:\na) Row 186: “when the training is completed…” Here “completed” should be “complete”.\nb) Row 274: “…the performance of the validation/test set…” Here “of” should be “on”.\nc) Row 716 (in Appendix): “NCF” should be NeuMF.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "8PuRHwDulpH", "reviewer": "Reviewer_7cbi", "summary": "This paper presents a technique for item recommendation which augments a standard collaborative metric learning (CML) with multiple user representations. An additional loss term is introduced in order to encourage diversity among representations for the same user. The proposed approach is compared to alternative matrix factorization (MF) and CML based techniques on common benchmark datasets. Additionally, theoretical justification for the proposed approach in the form of a generalization bound is presented.", "questions": "- What is the intuition behind wanting to ensure representations for the same user aren't too dissimilar in $\\psi_g(u_i)$?\n- Were other diversity-promoting regularizers explored?\n- Does the diversity-promoting regularizer improve results for MF based systems like those in [1,2]?\n- Does the proposed approach actually result in more diverse recommendations?\n- Were similar computational resources allocated to tuning baseline hyperparameters?\n\n", "limitations": "The paper does not discuss any limitations of the proposed approach. Providing some discussion of failure modes would strengthen the paper.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths\nItem recommendation is an important problem, and the experiments demonstrate that the proposed approach performs well in the settings considered.\n\nThe work here suggests that it may be worth revisiting ideas previously developed in the context of MF based techniques in the CMF framework.\n\n### Weaknesses\nThe idea of using multiple user representations is not new and goes back to at least [1], which is identical to the approach presented in [2] (only the later is cited in this work). Showing that something that works in the MF setting also works in the CML setting is a limited contribution.\n\nThe diversity-promoting regularizer is not sufficiently explored. The paper does not adequately motivate the desire to prevent representations for the same user from being too different, the second term of the unnumbered equation between (7) and (8). I find this counterintuitive and would have appreciated more commentary on this, and further exploration of the regularization strategy in general. Does including this regularizer in an MF setting like that explored in [1, 2] improve those results? Were alternative regularizers considered? How sensitive are the results to $\\eta$?\n\nThe extent to which the propose system actually promotes diversity (the motivation for the approach) is not explored outside the result in Figure 5. Providing more more evidence that the proposed approach actually results in more diverse recommendations (for example, using well know measure of diversity [3]) would significantly strengthen the paper.\n\nThe paper introduces several additional hyperparameters ($C$, $\\eta$, $\\delta_1$, $\\delta_2$), and the results here seem highly dependent on their exact choice (Figures 6, 8, 9). This is a significant drawback and I wonder if similar computational budgets were allocated to tuning baselines?\n\nI found the paper somehow difficult to read and poorly organized. There are numerous grammatical, incomplete sentences, and unreadable figures. A couple I explicitly noted are listed below.\n\n- Line 22: “giving them the relevant recommendations” => “giving them relevant recommendations”\n- Line 268-269: The sentence “Moreover, there would induce different performances with different diversity values.” is missing a noun.\n- Figure 6 and 7 are impossible to read without zooming to around 400%.\n- It would help if the numerical order of Figures 6-9 should match the order they are discussed in the subsequent paragraphs.\n\n### References\n1. Jason Weston, Ron J Weiss, and Hector Yee. Nonlinear latent factorization by embedding multiple user interests. In ACM Conference on Recommender systems, 2013.\n2. Guo, Wenshuo, et al. The Stereotyping Problem in Collaboratively Filtered Recommender Systems. Equity and Access in Algorithms, Mechanisms, and Optimization. 2021.\n3. Borodin, Allan, Hyun Chul Lee, and Yuli Ye. Max-sum diversification, monotone submodular functions and dynamic updates. ACM SIGMOD-SIGACT-SIGAI symposium on Principles of Database Systems. 2012.\n\n### Update Post Rebuttal\nIn their reply, the authors have provided thorough answers to my questions and further experimental results which address my concerns. I have raised my score accordingly.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept