Consistent Collaborative Filtering via Tensor Decomposition
Authors: Shiwen Zhao
Guillermo Sapiro
Venue: Accepted by TMLR ()

Abstract: Collaborative filtering is the de facto standard for analyzing users’ activities and building recommendation systems for items. In this work we develop Sliced Anti-symmetric Decomposition (SAD), a new model for collaborative filtering based on implicit feedback. In contrast to traditional techniques where a latent representation of users (user vectors) and items (item vectors) are estimated, SAD introduces one additional latent vector to each item, using a novel three-way tensor view of user-item interactions. This new vector extends user-item preferences calculated by standard dot products to general inner products, producing interactions between items when evaluating their relative preferences. SAD reduces to state-of-the-art (SOTA) collaborative filtering models when the vector collapses to 1, while in this paper we allow its value to be estimated from data. Allowing the values of the new item vector to be different from 1 has profound implications. It suggests users may have nonlinear mental models when evaluating items, allowing the existence of cycles in pairwise comparisons. We demonstrate the efficiency of SAD in both simulated and real world datasets containing over 1M user-item interactions. By comparing with seven SOTA collaborative filtering models with implicit feedbacks, SAD produces the most consistent personalized preferences, in the meanwhile maintaining top-level of accuracy in personalized recommendations. We release the model and inference algorithms in a Python library https://github.com/apple/ml-sad.


Published in Transactions on Machine Learning Research (July/2023)
Consistent Collaborative Filtering via Tensor Decomposition
Shiwen Zhao
sushao@apple.com
Apple
Charles G Crissman Work done while at Apple
charleyc@gmail.com
Guillermo R Sapiro
gsapiro@apple.com
Apple
Reviewed on OpenReview: https: // openreview. net/ forum? id= HqTuA
Abstract
Collaborative filtering the de facto standard for analyzing users' activities and building recommendation systems for items. In this work we develop Sliced Anti-symmmefric De- composition (SAD), new model for collaborative filtering based implicit feedback. In contrast to traditional techniques where latent representation of users (user vectors) and items (item vectors) are estimated, SAD introduces one additional latent vector to cach item, using novel three-way tensor view of user-item interactions. This new vector ex- tends user-item preferences calculated by standard dot products to general inner products, producing interactions between items when evaluating their relative preferences. SAD re- duces stato-of-thc-art (SOTA) collaborative filtering models when the vector collapses to I, while in this paper allow its value to be estimated from data. Allowing the values of the new item vector to be different from 1 has profound implications. It suggests users may have nonlinear mental models when evaluating items, allowing the existence of cycles in pairwise comparisons. We demonstrate the efficiency of SAD in both simulated and real world datasets containing over 1M user-item interactions. By comparing with seven SOTA collaborative filtering models with implicit feedbacks, SAD produces the most consistent personalized preferences, in the meanwhile maintaining top-level of accuracy in personal- ized recommendations. We release the model and inference algorithms in Python library https://githn https///gggtabbb.c.maaa////////
1 Introduction
Understanding preferences based users' historical activities is key for personalized recommendation. This is particularly challenging when explicit ratings on many items are not available. In this scenario, historical activities are typically viewed as binary, representing whether a user has interacted with an item or not. Users' preferences must be inferred from such implicit feedback with additional assumptions based on this binary data.
One common assumption is view non-interacted items as negatives, mcaning users are not interested them; items that have been interacted often assumed to be preferred ones (Hu al. 2008 Pan et al. 2008]. In reality however, such an assumption is rarely met. For example, lack of interaction between user and an item might simply be the result of lack of exposure. It is therefore more natural to assume that non-interacted items are combination of the ones that users do not like and the ones that users are not aware of (Rendle et al. 2009
With this assumption, Rendle et al. (2099) proposed give partial orders between items. Particularly, they assumed that items which users have interacted with are more preferable than the non-interacted ones. With this assumption in mind, Bayesian Personalized Ranking (BPR) was developed to perform personalized

---

Published in Transactions on Machine Learning Research (July/2023)
2
4
2
3
4
1
1
1
A112
1113
$114
2
3
4
-1
-1
?
2
?
+
+
?
1
?
2
X121
Xy23
X124
2214
14
X114
2
+
?
+
3
?
1
-1
3
X131
1132
A134
A324
3
+
?
?
-1
-1
-1
-1
?
4
X141
M142
A143
F234
4424
5544
?
?
1
1
2/41
1242
X243
?
?
X341
*342
A343
4454
5524
5
?
+
?
1
X441
X443
X534
D
?
?
-1
X
A541
Ks4p
Ase7
(a) Data transformation to form three-way binary tensor D
(b) Tensor parameter X
Figure 1: Diagrams provide visualizations of both transformed observation D and parameters X underlying the observation (Rendle et al. 2009).
rocommendations. In BPR, the observed data are transformed into a three- way binary tensor D with the first dimension representing users. The other two dimensions represent items, encoding personalized preferences between item pairs (Figure Iap. Mathematically, this means that any first order slice of D at the u-th user, Du:, is represented as pairwise comparison matrix (PCM) between items. The (i,j)-th entry when observed, duiz E [),,1}, is binary, suggesting whether w-th user prefers (dusj 1) the i-th item over the j-th one, or the other way around (duij = -1). The tensor D is only partially observed, with missing entries where there no prior knowledge to infer any preference for particular user. The recommendation problem becomes finding parsimonious paramcterization of the generative model for observed entries in D and estimating model parameters which best. explain the observed data.
The model used in BPR assumess that among the observed entries in D, the probability that the u-th user prefers the i-th item over the j-th item can be modeled as a Bernoulli distribution (Hu et al. 2008
p(duij 1|X) 1 exp aait))
where X = [Iuaz} E Raxmxm is the collection of unknown parameters of the Bernoulli distribution (Figure
[b). In fact, Luij is the natural parameter of a Bernoulli distribution. Rendle et al. (2009) decompose the natural parameter by
Fuij == Rui Taj E IR.
(2)
In the equation, Zui (Zwy) can be interpreted as the strength of preference the i-th (j-th) item for the u-th user. In other words, users' strengths of preference on different items are represented by scalar values denoted as Tui- The relative preferences between items are therefore characterized by the differences between their preference's strengths for a particular user. With this decomposition, Zuij becomes w-th user's relative preference on the i-th item over the j-th item.
By further letting Fui be represented as a dot product betwoen dimensional user vector &a R and an item vector 7/ R*
Fui (E., 11))
Rendle et al. 2009 reveal the connection between their proposed BPR model and traditional collaborative filtering models such as matrix factorization.
The factorization in equations 2 and provides a parsimonious representation of the original Equation Without the factorization, could model Fuij independently. This model requires free
2

---

Published in Transactions on Machine Learning Research (lll)
parameters, and model fitting would result in Fuij = oo for duiz = 1, and Zuij = -oo for duij = -1. The rests of Baij with missing duij are left unknown. The parsimonious representation, in contrast, reduces the number of parameters to (n + m) x k. The value of Tuiz now becomes coupled with Tuit for t * j, and entries in D jointly impact parameter estimate of Zusj- This highlights a fundamental assumption behind collaborative filterin:: information can be learned from items that share similar pattern when they are interacted by users, and one user can learn from another user who interacts with similar set of items.
Equation has direct link to the Bradley- Terry model often studied when analyzing PCM for decision making (unter 2004 Weng & Lin 2011). This model can be at least dated back to 1929 (Zermelo) 1929 One property of the model is its transitivity: The relative preference Zuij can be expressed as the sum of relative preferences of Luit and Eutj, for any user with t + i and & + j. However, in reality this transitivity property is less frequently met. Only around 3% of real world PCM's satisfy complete transitivity (Mazurek & Perzina 2017). The violation of the property more conceivable when users are exposed to various types of items. For example, in an online streaming platform, one favorable movie could become less intriguing after subscriber watches different style/genre.
In this paper, we extend the original BPR model (which is one of the most fundamental models in recommen- dation systems) to allow non-transitive user ratings. In particular, we extend Equation 2 to a more general form by proposing a new tensor decomposition. We denote our new model SAD (Sliced Anti-symmetric Decomposition). The new tensor decomposition introduces a second set of non-negative item vectors Ti for every item. Different from the first vector 7); the new vector contributes negatively when calculating relative preferences, producing counter-effects to the original strength of users' preferences; see Section f for more details. Mathematically, the new vector extends the original dot product in Equation 3 to an inner product. The original BPR model becomes a special case of SAD when the values in T; are all set to 1, and the inner product reduces to standard dot product. When contains entries that are not 1, the transitivity property no longer necessarily holds. While assigning an 4 regularization to the entries in to encourage its values being to reflect prior beliefs, SAD is able to infer its unknown value from real world data. We derive an efficient group coordinate descent algorithm for parameter estimation of SAD. Our algorithrm results in a simple stochastic gradient descent (SGD) producing fast and accurate parameter estimations. Through a simulation study first demonstrate the expressiveness of SAD and efficiency of the SGD algorithm. We then compare SAD to seven alternative SOTA recommendation models in three publicly available datasets with distinct characteristies. Results confirm that our new model permits to exploit information and rela- tions between items not previously considered, and provides more consistent and accurate results as we will demonstrate in this paper.
2 Related Works
Inferring priority via pairwise comparison. The Bradley-Terry model (Hunter 2004 Weng & Lin 2011 Zermelo, 1929) has been heavily used along this line of research. In the Sowlley-Terry, model, the probability of the i-th unit (an individual, a team, or an item) being more preferable than the j-th unit (denoted as j) is modeled by
p) X11(A;; + Ag):
(4)
where X represents the strength, or degree, of preference of the i-th unit The goal is to estimate A for all units based on pairwise comparisons. The link to Equation ① becomes clear once we omitting user index and sot x; = log(A.). In fact, the original BPR model can be viewed as an extension to the Bradlcy-Terry model to allow personalized parameters, and the strength of preferences are assumed to be dot products of user and item vectors as in Equation
Various algorithms have been developed for parameter estimation of this model. For example, Hunter (2004) developed a class of algorithms named a minorizing function Q is maximized to find the next parameter update at every iteration. Refer to the work by Hunter & Lange (2004) for more details related to MM. Weng Lin (2011) proposed a Bayesian approximation method to estimate team's priorities from outputs of games between teams. Most recently, Wang et al. (2021) developed bipartite graph iterative method to infer priorities from large and sparse
(MM) for parameter estimation. In MM,
3

---

Published in Transactions on Machine Learning Research /uly)2023)
pairwise comparison matrices. They applied the algorithm to the Movie-Lens dataset to rank movies based on their ratings aggregated from multiple users. Our paper is different from aforementioned models in that we model user-specific item preferences under personalized settings.
Tensor decompositions for recommendation. Compared to traditional collaborative filtering methods using matrix factorizations, tensor decompositions have received less attention in this field until recently. The BPR model can be viewed as one of the first attempts approach the recommendation problems using tensor analysis. As discussed in Section [ by making the assumption that interacted items are more preferrable compared to non-interacted ones, user-item implicit feedback are represented as a three-war way binary tensor (Rendle et al. 2009). In their later work, the authors developed tensor decompasition models for personalized tag recommendation (Rendle & Schmidt-Thiee [ 2010).. The relationship between their approach and traditional tensor decomposition approaches such as Tucker and PARAFAC (parallel factors)
decompositions (Kiers 2000 Tucker
1966) was discussed.
Recently, tensor decomposition methods have been used to build recommendation systems using information from multiple sources. Wermser et al. (2011) developed context aware tensor decomposition approach by using information from multiple sources, including time, location, and sequential information. Hidasi & Tikk (2012) considered implicit feedback and incorporated contextual information using tensor decomposition. They developed an algorithm which scaled linearly with the number of non-zero entries in a tensor. A comprehensive review about applications tensor methods in recommendation can be found by Frolov & Oseledets (2017) and references therein. Different from leveraging multiple sources of information, the SAD model developed in this paper considers the basic scenario where only implicit feedback are available, the scenario that is considered in BPR model (Rendle et al., 2009). Our novelty lies in the fact that propose more general form of tensor decomposition for modeling implicit feedback.
Deep learning in recommendation models. Deep learning has attracted significant attention in recent years, and the recommendation domain is no exception. Traditional approaches such as collaborative filter- ing and factorization machines (FM) have been extended to incorporate neural network components (Chen jet al. 2017: He et al. 2017 Xiao el al., 2017). In particular, Chen et al. (2017) replaced the dot product that has been widely used in traditional collaborative filtering with a neural network containing Multilayer Perceptrons (MLP) and embedding layers. Chen ct al. (2017) and Xiao et al. (2017) introduced attention mechanisms Vaswani el al. 2017) to both collaborative fillering and FM (Rendle) 2010). Mostly recently Rendle et al. (2020) revisited the comparison of traditional matrix factorization and neural collaborative fil- tering and concluded that matrix factorization models can be as powerful as their neural counterparts with proper hyperparameters selected. Despite the controversy, various types of deep learning models including convolutional networks, recurrent networks, variational auto-encoders (VAFs), attention models, and combi- nations thereof have been successfully applied in recommendation systems. The work by Zhang al. (2019) provided an excellent review on this topic. This line of research doesn't have direct link to the SAD model considered in the current work. However, provide brief review along this line since our model could be further extended to use the latest advances in the area.
3
Notation
We use n to denote the total number of users in a dataset and m to denote the total number of items. Users are indexed by E [1, n.]. Items are indexed by both i and j mm]...m.. We use k to denote the number of latent factors, and use h E [1,... k] to index factor. Capital letters are used to denote matrix a lensor, and lowercase letters to denole scalar is denoted as D E R=xmxm and the (u, i,j)-th entry is denoted as duiz. Similarly, the user latent matrix is denoted as E R and its u-th column is denoted as &u to represent the user vector for w-th user. We use gh to denote the h-th row (the h-th factor) of E viewed as a column vector.
vector. For example, the three-way tensor of observations
4 Tensor Sliced Anti-symmetric Decomposition
We start with the original BPR model. The relative preference Tuiy defined in Equation 2 forms a three-way tensor X E [ The BPR model (Rendle et al. 2009) can be viewed as one parsimonious representation
4

---

Published in Transactions on Machine Learning Research (023)
of tensor X. Let Eu RR and n R* denote the user and item vectors respectively, and let Enu (ma)) indicate the h-th entry in Eu (m.). Equations and can be re-written as
X Chu(Hn (Hr))
Am1
where Xu: is the first order slice of X al u-th user,
Hy = r/" 01.
n Rm being the h-th row of item matrix H E Rkxm. 1 E RR is used to denote a vector of all I's and being the outer product.
4.1 Anti-symmetricity of Xu::
As discussed in Section Xu:: represents parsimonious representation of parameters of PCM. We formalize the property of X as follows:
Property 4.1. For every user u, the first order slice of X, is an anti-symmetric with X.:: -
This can be shown casily by letting Puij = p(duij = 1) = p(i >u j) and noting that the relative preference Zusj is the natural parameter of the corresponding Bernoulli distribution with Tuij = log(pwis/ Puij)). Note that Zuij is also known the log-odds or logit.
The decomposition introduced in BPR can be further written as
X
Σ
Enu (v) c1 - 1or").
(5)
k=1
Note that the anti-symmetricity is well respected in the equation. Intuitively, the decomposition suggests that for the u-th user, her item preference matrix Xu:: can be decomposed as a weighted sum of k anti-symmetric components, each of which is the difference of a rank one square matrix and its transpose.
4.2 Generalization of BPR
By replacing 1 with arbitrary vector ry E R"" the new square matrix n/" or - on " is still anti-symmetric, and Property [.1 still holds for the resulting X With this observation, generalize Equation by proposing a new parsimonious representation of X
Xus Σ Cna(T" ot - -mon"") h=1
(6)
In this work we require entries in Th to be non-negative.
The rationale will become clear in Sec- Rkxxn, H (1,,, , Rkxm and T = 72 .,..... -T)T Rixm (we use R4 to denote the set of non-negative real numbers), introduce the proposed Sliced Anti-symmetric Decomposition (SAD).
tion 4.3 Furthermore, by letting =\t (2,, En)
Definition 4.1. We define the Sliced Anti-symmetric Decomposition (SAD) of X to be the matrices H, satisfying Equation 1 above for every user index u. We denote this by
SAD X = (E,H,T)
(7)
4.3 Interpretation of SAD
To understand the interpretation of SAD, start by re-writing equations and in BPR
Juss = (Eu, 1p.) (Eu, n1)
(Eawthi EneTps)
hm1
5

---

Published in Transactions on Machine Learning Research (02l)
The term ShuTlh can be interpreted as the strength of preference of the u-th user the i-th item from the hith factor. The overall strength of preference, Ruis is the sum of contributions from the k individual factors. Accordingly, the relative preference over the (i,j)-th item pair for the u-th user can be viewed as the difference between the preference strengths of the i-th and the j-th items from the u-th user.
This interpretation has direct link the Bradley-Tens Terry model 44 as previously mentioned, in which the strength of the i-th item is described as a positive number );. Here the strength of preference is viewed user specific and is represented by real mumber zui log Aui-
SAD extends the original equations 2 and 3 by introducing a new non-negative vector for every item (a
column in T in Equation 7 We can re-write Equation i as follows for every item pair and j:
Fuij (fu, deddigg(, (Su: M55 diag(r))
Σ (EhuMhaThy Tha))
(8)
(ss (ddag(())) in Equation denotes the inner product with a diagonal weight matrix having 7; on the diagonal. To be a proper inner product, we require t; to be non-negative, resulting in a positive semi-definite matrix diag(7,).
The first term on the right hand side of Equation g describing the preference strength of the i-th item, now becomes dependent on Thjs the h-th entry in тj of the j-th rival. When Thi is bigger than 1, it increases the effect of Enu Similarly, the second term on the right hand side suggests that when The is bigger than 1, it strengthens the effect of the j-th item. The opposites happen when either Thy or The is smaller than 1. Therefore, while respecting the anti-symmotricitt, the new non-negative item vector т₃ can be viowed a counter-eflect acting upon the strength of relative preferences, penalizing the strength when greater than 1, while reinforcing when smaller than 1. In real world applications, a user's preference indeed may be influenced by different items. For example, during online shopping, one favorable dress may become less intriguing after customer sees a different one with different style/color that matches her needs. In an online streaming platform, one favorable movie could become less interesting after a subscriber watches another one with different style/genre. SAD allows us to capture these item-item interactions by introducing new set of vectors T₃.
To summarize, interpret the three factor matrices in SAD follows:
represents the user matrix. Each user is represented by a user vector Eu E R*
H represents the left item matrix, which is composed of left item vectors denoted as y E R*. It contributes to the strength of preference on the i-th item via an inner product with user vector f..
T' represents the right item matrix, which contains non-negative right item vectors denoted R4. This set of vectors defines the weight matrices of inner products between Ni and &u- It produces counter-effects to the original preference strengths, with values bigger than 1 adding additional strength to rival items in pairwise comparisons, and value smaller than 1 producing the opposite effect. When T = 1, the model reduces to the original BPR model.
In SAD estimate the value of T from data. As discussed in Section I encourage the values of entries in T to be 1 unless there is strong evidence from the data suggesting otherwise. This is achieved by adding an 4 regularization centered around to the entries in T independently.
The 4 regularization has another side effect. In Equation S multiplying by any constant e to The and 1/c to Thy results the same objective function, causing H and T to be unidentifiable. The additional 4 regularization around mitigates the identifiability problem by discouraging any constant multiplication that moves Thy away from 1, making the joint objective function identifiable between H and T.

---

Published in Transactions on Machine Learning Research (02l)
4.4 The transitivity problem
In social science involving decision makings, PCMs have been investigated extensively (Saaly & Vargas, 2013] Wang et al., 2021). It is usually assumed that a PCM holds the transitivity property, resulting in the following observation introduced in Section [ The relative preference of the (i.j)-th item pair can be derived from the sum of relative preferences the (i, t)-th and (t,j)-th item pairs, with t * and t * J. Zuij = Juit + Zutj- The original BPR meets this property nicely. After introducing 7 SAD, this property no longer necessarily holds. One can show that t; = Tj 7 for ternary (i.j.t) is sufficient condition for transitivity in SAD. In our model, allow the violation of this property, making the proposed model more realistic given the fact that complete transitivity is met only in 3% of real world applications (Mazurek & Perzina 2017
4.5 Inference algorithms
To estimate model parameters, we maximize the log likelihood function directly. The log likelihood given observed entries in D can be re-written
O)()) = Σ 1(duss = - 1))ees - log (1 + exp (-Zuiz)):
(9)
where 1(-) is the indicator function, and the sum taken with respect to non-missing entries in D with < j. Here we require i < to prevent us from double counting.
We take the derivatives with respect to the columns of E, H, and T, resulting in following gradients
ô log p(D()) = Wais (1/ ① T, n 2 TT).
0E. a log p())( = Waisfa ② Tz+
On; 1 log (D = - WaifEu 5u O Ti:
(10)
On,
0 log (D))) = - WaefEu su ① Th Ôπ: älog ( Waish Om,
θT,
from the (n, observation. Here Wuij (duuij -1) exp(=xai))/(1 + exp (-rwiy)) and is the element-wise product (Hadamard product).
Equation 10 allows us to create a stochastic gradient descent (SGD) algorithm (Algorithm to optimize the negative the log likelihood. During optimization, we add an 4 penalty with weight wi to the entries in T' independently to encourage their values to be 1. In addition, we add an le independent penalties with weight Wa to both and for further regularization.
We also develop efficient Gibbs sampling algorithm for full posterior inference under Probit model setup. By drawing parameter samples from posterior distributions, the Gibbs sampling algorithm has the advantage of producing accurate uncertainty estimation of the unknown parameters under Bayesian inference. We replace the logistic function in Equation with p(duij = 110) = (z where (I is the cumulative distribution function (CDF) of the standard Guassian distribution centered at Zuij. By assigning spherical Gaussian priors to E, H, and T, full conditional distributions can be derived. More details can be found in Appendix.
5 Simulation Study
We first evaluate the performance of SAD and the SGD algorithm on simulation data, with the goal of examining the performance of our algorithm with true parameters known ahead. We choose n 20 users, = 50 items, and k = 5, resulting in = E R5x20 H E R5x50 and T E R5x50 We consider two scenarios in
7

---

Published in Transactions on Machine Learning Research JJuly20233
Algorithm SGD for parameter estimation of SAD
Require: n, m, k; D RRxxmxm, P. W1, W2 Initialization E E R&xm H. Rkxm T R x -E, while Convergence not met do for и n do
pi learning rale, wi, w2 weights for I and l2
Equation 7
for Every item in interacted set do
Random select item j from non-interacted item set
Calculate dmu, dn:: dny, drg, dr,
b Equation 10
Eu + fu + p. (df. 2w2Eu)
!i + p. (dy; 2w2Thi)
π - 7 + (dnı 2w27T)
7 - T, + (dr, - wi w > 1] + w 1 < <1)
[|]: Entry-wise indicator to 7 E RR
Tj s + p (dr, - wil(T; > + wi < 1])
end for end for end while
the simulation. In the first simulation (Siml) we set. T to 1, effectively reducing SAD to the generalive model of BPR (Rendle et al. 2009). In the second scenario (Sim2), we set small proportion of T to either 0.01 or 5, the other entries are set to 1. For user matrix E and left item matrix H, their values are uniformly drawn from the interval [-2, We calculate the preference tensor with Equation and draw an observation tensor D from the corresponding Bernoulli distributions.
We first examine the performance of SAD with complete observations in Sim2 validate if method is able to generate accurate parameter estimation. We run the SGD algorithm with learning rate 0.05. The weight of the l regularization assigned to T is set to 0.01, and the weight of the l2 regularization is set to 0.005. Initial values of parameters are randomly drawn from a standard Gaussian distribution. The number of latent factors is set to the true value. In reality when select the best value of After 20 epochs, f is able to recover the sparse structure of the true parameters of T up permutation of factors (Figure 2 The user matrix and left item matrix converge to the truc parameter values as well (Figure 3)-
unknown, cross validation can be used to
Next examine the performance of SAD in both Siml and Sim2, under the scenarios with missing data. To be more specific, we randomly mark 2% of D missing to mimic missing at random. Note that in the real world, observations could have more complex missingness structures. As a comparison, we run BPR under the same contexts.
Estimated f, MSE: 0.304
n.2
0J
B3
811361
1.0
0562
u5
0U
27
2.52
0./ 09 03 07001
100
35 06 05 09 08
09
18
10
15
16
18
19
21
22
24
25
27
28
29
30
32
35
36
38
39
41
True T
05
21
1 L.0
1.0
5.0
50
50
2.0
2010
10
10 1.0
10
10
15 16 18
21
24
25
27
28 30
35
38 39
Figure 2: Comparison of f with ground truth. Factors are re-ordered in T to match true T.
The convergences of SAD in Siml are shown in Figure 4a
together with the estimated sparsity of T. Here the sparsity of T. is defined as the percentage of the entries in T. with [T 1| < 0.05. When the percentage

---

Published in Transactions on Machine Learning Research (July/ )J23
Estimated g
MSE: 0.013
Estimated 8
07533 +B 130
MSE: 0.085
MSE: 0.025
MSE: 0.025
Estimated '.
MSE: 0.032
Estimated
Estimated
True 51
True 5²
True 5
True 5
True 5
Estimated n
MSE: 0.124
Estimated n?
MSE: 0.043
Estimated ni
MSE: 0.171
Estimated n
MSE: 0.040
Estimated ñs
MSE: 0.107
True n
True n?
True n
True n
True n)$
Figure 3: Comparison of Ê, A with their ground truth. Factors are subject re-order and sign flips.
le4
100%
log(p) - log(pa))
log(p) log((gg)
4.875
80%
4.875
80%
60%
60%
biissig
3,250
mnssinn
3,250
40%
40%
1.625
20%
1.625-
20%
0.000
200
400
600
800
1000
0%
0.000
800
1000
056
100%
100%
100%
100%
T sparsity
75.0%
80%
75.0%
80%
50.0%
60%
Gunss
40%
40%
25.0%
20%
25.0%
20%
0.0%
0%
0.0%
0%
200
400
600
800
1000
200
400
600
800
1000
Iteration
Iteration
(a) Likelilhood Sparsity (Sim1)
(b) Likelilhood Sparsity (Sim2)
100%
||x Xxoee -xxeelllttmmn (r/lmmm)
10a.
SAD BPR
1|x - Xooel ((((mmm))
10a
SAD
80%
BPR
80%
Suiss %
60%
biissiu
60%
10a-
40%
20%
20%
201.
10a
200 0
400
600
800
1000
200
400
600
800
1000
Iteration
Iteration
(c) Convergence to True Parameter (Siml)
(d) Convergence to True Parameter (Sim2)
4b show changes of log likelihood (Equation during SGD optimization. log(pe,) the log likelihood at 0-th iteration. Bottom row 4b shows the true sparsity (86%) as black line. 4c and 4d show the Frobenius distance between estimated parameter X and true parameter Xtrue during SGD optimization. Note that SAD (solid line) achieves a much lower distance compared with BPR (dashed line) in 4d under wide range of percentages of missingness.
Figure 4: Convergence in Simulation Study. Top rows in 4a and
of missingness is at small or medium levels, SAD able to converge to a sparsity close to 1, suggesting the effectiveness of the I regularization. It bocomes more challenging when the percentage of missingness the trajectories of the mean squared error (MSE) between X and the true
surpasses 70%. under different missingness percentages for both SAD and BPR. Both SAD and BPR able to converge to
9

---

Published in Transactions on Machine Learning Research (
low MSE with small/meding percentages of missingness. Similarly, with high percentage of missingness, the performance of both models begins to deteriorate. We conclude that SAD has a performance on par with BPR when data are simulated from the generative model of BPR.
Results for Sim2 are shown in Figure 4b Note that the true sparsity of is 86%. SAD is able to generate an accurate estimation of the sparsity under small/medium percentages of missingness. When evaluating both models using MSE, SAD is able to achieve a much lower value due to its correct specification of the generative model (Figure 4dh.
6 Applications for Real Data
We select three real world datasets to evaluate SAD and compare against SOTA recommendation models. The datasets selected contain explicit integer valued ratings. Nonetheless, we mask their values and view them as binary. The explicit ratings are used as a means to evaluate models' consistency pairwise comparison after model fitting (dctails below). The first datasct used is from the Nctflix Prize (Bennett et al. 2007)- The original datasel contains movie ratings of 8, 8,921 movies from 478,533 unique users, with total number of ratings reaching to over 50M. We randomly select 10, 000 users as our first dataset. The resulting dataset contains 8, 693 movics with over 1M ratings from the 10, 000 users. For the second dataset, we choose the Movie-Lens 1M dataset (Harper & Konstan 2015). It contains over 1M ratings from 6, 040 users on 706 movies. As a third dataset, we consider the reviews of recipes from Food-Com (Majumder itt al. 2019). The complete dataset has 1.1M reviews from 227K users on 231K recipes. We select the top 20K users with the most activities, and filter out recipes receiving less than 50 reviews. The resulting dataset has 145K reviews from 17K users (users with zero activity are further removed after filtering recipes) on 1.4K recipes.
The three datasets have distinct characteristics. Among the three datasets, the Food-Com review dataset has the least user-item interactions, even when the most active users/pon pooplar recipes are selected. The maximum number of items viewed by a single user is 878.
also has the largest number of users. The Netflix dataset is the most skewed, with the number of items interacted by a user ranging from as low as I as high as over 8K. The Movie-Lens dataset contains the largest number of user-item interactions, and is most uniformly distributed. Some details of the three datasets can be found in Table A in Appendix.
We choose seven SOTA recommendation models to compare with SAD. Their details are listed in Table 3. For each of the model considered, we perform a grid search to determine hyperparameters. Models are chosen based on their goodness-of-fi using log likelihood. We evaluate the models using a comprehensive leave one interaction out (LOO) evaluation [Bayer et al. 2017 He et al. 2017 2016), in which we randomly hold out one user-item interaction from training set for every user. Users who have only one interaction are skipped. We create 20 such LOO sets for each dataset considered. The dimension of latent space during evaluation is set to 500 for all models and datasets. The choice of the latent dimension can be further optimized using methods such as across validation. In this work choose the same number across comparing SOTA models such that they can be evaluated on a common ground.
1
Σ
Σ
(anoj 1 j) + @njol(j > o))
(11)
(17m)
1)
jEL
1
Σ JET
Σ
(1(bnoj > 001(00 1) 1(xnjo 0)1(j o))
(12)
We consider two aspects of model's performance during evaluation: consistency and recommendation. The consistency is defined as whether model's prediction matches user's actual pairwise preference. During evaluation, the hold out item o and other items j in interacted set I, of user u are arranged such that o >~ j based on users' actual ratings. The mean of their predicted preference (Equation I., the percentage of consistent predictions (Equation 12 and the modian of per user percentage of consistency are reported in Table SAD has the most consistent results among all eight recommendation models except in one scenario, in which our model is second best.
10

---

Published in Transactions on Machine Learning Research (023)
Table I: Model evaluations across 20 LOO datasets. When evaluating consistency, item pairs between hold out item o and other interacted items j for a user are ordered based on the user's actual ratings (o > J). The mean of their predicted preference (Equation [1], the percentage of predictions that match with actual ratings (Equation 12], and the median of per recommendation, the percentages of random hold out items that are ranked higher than 20 (out of 100) using two different ranking method (M1 and M2) are shown. See Table in Appendix for details about each of the comparing models.
percentage of match
reported. When evaluating
Consistency
Recommendation
Dataset
Model
mean Juaj
match (%)
per user (%)
(%)
(%)
SAD
0.024 0.012
33.7 0.5
18.7 = 0.3
83.3 = 0.3
83.9 + 0.3
BPR
-0.019 0.012
32.6 + 0.5
12.7 0.3
81.8 = 0.4
81.7 + 0.3
SVD
-0.824 = 0.032
1.11
0.0
0.5
2.1 0.3
0.3
MF
-0.018 0.058
8.5
0.0
7.
45.4
Netflix
PMF
0.003 + 0.015
3.3
2..9
21.1
20.2
FM
-0.538 0.316
0.2
8.40.11
8.2
0.3
NCF
0.025 0.043
13.1
5.
86.0 = 0.3
1..9222
8-VAE
0.011 =
2.91
9.7
35.7 3.1
31.
SAD
0.120 + 0.014
0.6
29.9
82.9 0.4
82.1 + 0.4
BPR
0.083 + 0.012
0.6
2.62
0.5
76.9 0.4
SVD
0.324 = 0.011
0.4
0.0
3.1
3.1 0.2
Movie
MF
0.021
1..0
0.0
17.
0.7
-Lens
PMF
0.027 + 0.016
3..7
26.1
2.8
22.8 0.5
FM
0.103 0.025
2..7
18.1
0.3
0.4
NCF
-0.241 = 0.346
2.90
14.5
9..4
90.1 2.1
B-VAE
0.120 0.301
2.1
10.9
71. 0.3
1 0.7
SAD
0.329 0.002
14.9 0.5
0.0
24.7 0.2
23.9 + 0.2
BPR
-1276 0.009
0.5
0.0
0.3
21.3 0.3
SVD
-5.152 = 0.039
1.1
0.0
0.0
0.0
Food
MF
-1.956 0.161
2.4
0.0
27.8
6.1
-Com
PMF
-0.331 = 0.031
4.1
0.0
21.0 2.6
FM
5.321 0.247
9.7
0.0
3.2
35.1
NCF
-11.362 = 0.852
8.9
0.0
382
37.1 1 5.5
8-VAE
-3.127 0.426
10.2
0.0
16.
16.1 3.7
To evaluate models' performance in recommendation, we create an item set containing 100 non-interacted items by randomly sampling for each user. We combine the hold out item with the 100 items to form a test set, and examine whether models are able to rank the hold out item high in the test set (Hit Ratio (He et al. 2017)). SAD faces some unconventional challenges (hence opportunities) in producing ranking when violation of transitivity exists. Consider three items i, j and t. With transitivity, if user has i X> and j t, then i > / must hold. However, SAD can result in a scenario in which > i, forming preference cycle among the three items, in which case no ranking can be inferred. We propose two methods using pairwise comparisons for SAD the evaluation. In the first method (M1), the number of non-interacted items the test set that more preferrable than the hold out item is dubbed as its rank. In a second method (M2), use the ratings from interacted items kept in training set and calculate the proportion of interacted items to rank items in the test set. We
that are less preferable than a test item. The proportion is used as calculate the percentage of hold out items that are ranked higher than 20 in all the hold out items. SAD is among the lop three best models that rank the hold out items in top 20 (Table D We argue that since our model's predictions match better with user's actual ratings, is able to bring additional information by introducing diversity of items into recommendation, while respecting users's potential preferences. In Appendix, we illustrate examples in which SAD produces model predictions consistent with true ratings while other SOTA models fail.
11

---

Published in Transactions on Machine Learning Research (2ll)
7 Discussions
We proposed new tensor decomposition model for collaborative filtering with implicit feedback. In contrast to traditional models, we introduced a new set of non-negative latent vectors for items. While respecting anti- symmetricity parameters, the new vectors generalized the standard dot products for calculating user-item preferences to general inner products, allowing items to interact when evaluating their relative preferences. When such vectors were all set to I's, our model reduced to standard collaborative filtering models. We allowed their values to be different from enabling the violation of the known transitivity property. The proposed model generated accurate recommendations across multiple real world datasets examined.
The existence of violation of transitivity has profound implication in real world applications. The items that violate the property form cycles in the directed graph implied by pairwise preferences of a user. For example, consider three items i, and t. Transitivity implies if user u has i i and j > t, then f > must hold. Violation of transitivity suggests > i, forming a preference cycle among the three items. When SAD detects such violations, even strong prior, I regularization, suggesting otherwise, it indicates that users themselves may not have linear mental model in terms of ranking items. When making recommendations, the items forming cycle can be selected as group, and users who are not certain which one in the group is the most relevant can be exposed to the entire group, increasing the possibility of producing a hit.
When ovaluating consistency in Section G we used users' actual ratings to determine pairwise preference between items. Many items in the datasets however had collided ratings due to the limited values those ratings can take. We chose the unevenly rated items with partial orders as ground truth during evaluation. The partial orders are a proxy for users' true preferences, and using them is the best whether our model is able to produce consistent predictions. For evenly rated items, there is no ground truth available to allow us to evaluate the performance.
do to evaluate
This topic touches one fundamental aspect in our existing rating system - using an integer valued score with limited range as a sole reflection of user's preference. Traditional collaborative filtering models pander to this system, by assuming there is a linear ranking among items, and the ranking is dictated by real number representing the preference strength. However, reality can hold evidence that violates the assumption. For example, Mazurek& & Perzinal (2017) demonstrated only a small proportion of real world pairwise comparisons satisfy complete transitivity. Li et al. (2013) showed the ubiquity of user providing very different rating scores on closely correlated items, producing self-contradictions. In addition to noisiness in user ratings, the observed self-contradictions is a manifestation of the collision between user's mental model and the rating system. SAD innovates by providing a novel methodology to mathematically parameterize this mental model.
There can be parameterizzations that both respect anti-symmetric property of X.:: and at the same time allow the violation of transitivity property other than Equation For example, instead of introducing the new vector T; R4 for every item, one can model Fuij as
Fuij Σ Eun Tbj)
with Th: = (Thithisssss E Rmxm. The in this parameterization can be viewed as an item interaction matrix for h-th factor. Additional structures can be assigned to Th: to enable parsimoniousness, such as letting The be symmetric (in order to make Xa:: anti-symmetrcc and modeling as a low rank matrix with They = (ame Shy), and are Shg E RR* In our current work we focused on an efficient model with a simple interpretation. We delegate the research of exploring alternative parameterizations to future work.
In this paper, restrict our scope to consider collaborative filterings for implicit feedbacks. SAD can be applied to datasets beyond the scope. For instances, datasets with explicit ratings contain partial orders that can be leveraged directly during model fitting, instead of being used to evaluate model consistency in a post- hoc manner as in current work. Such datasets include the ones considered in Section 6 and numerous others such as Yandex challenge and KDD Cup 2012, Other datasots that contain actual pairwise comparisons such as the one created by Paaviccennko & Ustalov (2021) are a natural fit to SAD as well. We expect the power of SAD can be further enhanced with neural network components integrated.
12

---

Published in Transactions on Machine Learning Research /2ll)
References
Immanuel Bayer, Xiangnan He, Bhargav Kanagal, and Steffen Rendle. A generic coordinate descent frame- work for learning from implicit feedback. In Proceedings of the 26th International Conference on World Wide Web, pp. 1450- 2017.
James Bennett, Stan Lanning, et al. The Netflix prize. In Proceedings of KDD Cup and Workshop, volume 2007, pp. 35. New York, NY, USA., 2007.
Jingyuan Chen, Hanwang Zhang, Xiangnan He, Liqiang Nie, Wei Liu, and Tat-Seng Chua. Attentive collab- orative filtering: Multimedia recommendation with item-and component-level attention. In Proceedings of the 40th International ACM SIGIR Conference on Reseorch and Development in Information Retrieval, pp. 335 344, 2017.
Evgeny Frolov and Ivan Oseledets. Tensor methods and recommender systems. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 7(3):c1201, 2017.
Scott Graham, Jun-Ki Min, and Tao Wu. Microsoft recommenders: Tools to accelerate developing recom- mender systems. In Proceedings of the 13th ACM Conference on Recommender Systems, pp. 542 543, 2019.
F Maxwell Harper and Joseph A Konstan. The Moviclens datasets: History and context. ACM Transactions on Interactive Intelligent Systems (TISS) 5(4):1-19, 2015.
Xiangnan He, Hanwang Zhang, Min Kan, and Tat-Seng Chua. Fast matrix factorization for online recommendation with implicit feedback. In Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval, pp. 549-558, 2016.
Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. Neural collaborative filtering. In Proceedings of the 26th International Conference on World Wide Web, pp. 173 182, 2017.
Balázs Hidasi and Domonkos Tikk. Fast ALS-based tensor factorization for context-aware recommendation from implicit feedback. In Joint European Conference on Machine Learning and Knowledge Discovery in Dalabases, pp. 67 82. Springer, 2012.
Yifan Hu, Yehuda Koren, and Chris Volinsky. Collaborative filtering for implicit feedback datasets. In 2008 Eighth IEEE International Conference on Data Mining, pp. 263-272. Icee, 2008.
Nicolas Hug. Surprisc: A Python library for recommender systems. Journal of Open Source Software, 5(52): 2174, 2020. doi: 10.21105/joss.0211 URL httpp:://aii.crr//022210///////0 02174
David R Hunter. MM algorithms for generalized Bradley-Terry models. The Annals of Statistics, 32(1): 384-406 2004.
David R Hunter and Kenneth Lange. tutorial on MM algorithms. The American Statistician, 58(1):00-37, 2004.
Henk AL Kiers. Towards a standardizod notation and terminology in multiway analysis. Journal of Chemo- metrics: A Journal of the Chemometrics Society, 14(3):105-12 122, 2000.
Bin Li, Ling Chen, Xingquan Zhu, and Chengqi Zhang. Noisy but non-malicious user detection social recommender systems. World Wide Web, 16(5):677- 699, November 2013. ISSN 1573-1413. doi: 10.1007/ s11280-012-0161-9, URL
Dawen Liang, Rahul G Krishnan, Matthew D Hoffman, and Tony Jebara. Variational autoencoders for collaborative filtering. In Proceedings of the 2018 World Wide Web Conference, pp. 689-698, 2018.
Bodhisattwa. Prasad Majumder, Shuyang Li, Jianmo Ni, and Julian McAuley. Generating personalized recipes from historical user preferences. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (FMNL.PIICCNLP) pp. 5976-599 2019.
13

---

Published in Transactions on Machine Learning Research (02l)
Jifí Mazurek and Radomir Perzina. On the inconsistency of pairwise comparisons: An experimental study. Svienlifie: Papers of the Universily of Pardubice. Series D, Facully of Economics and A Administmtion, 2017.
Andriy Mnih and Russ R Salakhutdinov. Probabilistic matrix factorization. In Advances in Neural Infor- mation Processing Systems, pp. 1257-1264, 2008.
Rong Pan, Yunhong Zhou, Bin Cao, Nathan Liu, Rajan Lukose, Martin Scholz, and Qiang Yang- One- class collaborative filtering. In 2008 Eighth IEEE International Conference on Data Mining, pp. 502 511. IEEE, 2008.
Nikita Pavlichenko and Dmitry Ustalov. comparisons. arXiv preprinl arXin:21 2.00:2110.11 14990, 2021.
An evaluation dataset for crowdsourced pairwise
Steffen Rendle. Factorization machines. In 2010 IEEE International Conference Data Mining, pp. 995- 1000, 2010. doi: 10.1 10.T109/TCDM.2010
Steffen Rendle and Lars Schmidl-Thieme. Pairwise interaction lensor factorization for personalized tag recommendation. In Proceedings of the Third ACM International Conference on Web Search and Data Mining, pp. 81-90, 2010.
Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thiena Thieme. BPR: Bayesian person- alized ranking from implicit feedback. In Proceedings of the Twenty- Pifth Conference on Uncerlainty in Artificial Intelligence, pp. 452 422, 2009.
StefTen Rendle, Walid Krichene, Li Zhang, and John Anderson. Neural collaborative filtering vs. matrix factorization revisited. Fourteenth ACM Conference on Recommender Systems, pp. 240-2400200000 2020.
Thomas L Saaty and Luis G Vargas. The analytic network process. In Decision Making with the Analytic Network Process, pp. 1-40. Springer, 2013.
Aghiles Salah, Quoe-Tuan Truong, and Hady W Lauw. Cornac: A comparative framework for multimodal recommender systems. the Journal of Machine Learning Research, 21:1-5, 2020.
Ledyard R Tucker. Some mathematical notes on threc-mode factor analysis. Psychometrika, 31(3)2779-33 -11,, 1966.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkorcit, Llion Jones, Aidan N Gomez, Lukass Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems, pp. 5998 6008,00888 2017.
Haomin Wang, Gang Kou, and Yi Peng. An iterative algorithm to derive priority from large-scale sparse pairwise comparison matrix. JEEE Transactions Systems, Man, and Cybernetics: Systems, 2021.
Ruby Weng and Chih-Jen Lin. A Bayesian approximation method for online ranking. Journal of Machine Learning Research, 12(1), 2011.
Hendrik Wermser, Achim Rettinger, and Volker Tresp. Modeling and learning context-aware recommendation scenarios using tensor decomposition. In 2011 International Conference on Advances in Social Networks Analysis and Mining, pp. 137 144. IEEE, 2011.
Jun Xiao, Hao Ye, Xiangnan He, Hanwang Zhang, Fci Wu, and Tat-Seng Chua. Attentional factorization machines: Learning the weight of feature interactions via attention networks. Tn Proceedings of the 26th International Joint Conference on Artificial Intelligence, IJCAI'17, pp. 3119 3125, Melbourne, Australia, 2017. AAAI Press. ISBN 978-0-9992411---3..
Ernst Zermelo. Die berechnung der turnier-ergebnisse als ein maximumproblem der wahnschennichkkettsccc.. nung. Mathematische Zeilschrift, 29(1):436 460, 1929.
Shuai Zhang, Lina Yao, Aixin Sun, and Yi Tay. Deep learning based recommender system: A survey and new perspectives. ACM Computing Surveys, 52(1):1-38, 2019.
14

---

Published in Transactions on Machine Learning Research (223)
A Properties of Real World Datasets
Table 2: Properties of the three real word datasets used in Section
Quantiles of #Ratings/User (min/5%/500% 92000%/99////// max)
Dataset
# Users
# #Ttems
# Ratings
Sparsity
Netflix
10, 000
8, 693
1, 044, 318
98.80%
(1/6/46, 4000/00 88, 237)
Movie-Lens
0.10
3, 706
000, 209
95.53%
(20/23/ 9 /56 314) (1/1/4/30/878)
Food-Com
17, 482
1, 1,358
145,431 431
99.39%
B
Gibbs Sampler for Posterior Inference of SAD
We derive an efficient Gibbs sampling algorithm as complement the SGD algorithm in the main paper. The Gibbs sampling algorithm has the advantage of producing accurate uncertainty estimation of unknowns under Bayesian inference by drawing parameter samples from the posterior distribution. The algorithm is an application of Bayesian Probit regression to the current setting. Specifically, we replace the original logistic paramcterization in Equation [ with the following equation:
p(dnij = 1/0) == ((Fwi));
(13)
where (( is the CDF of a Gaussian distribution with mean Fuij and variance By augmenting the model with hidden tensor Z {Zagh, where Zuij Kuij + euij and Euij. iid N(0, 1), the Probit model is equivalent to
Zuij >0 0
duij
Zuij S 0
With this new model, an efficient Gibbs sampling algorithı can be derived. As toy example, assign spherical Gaussian priors to rows of E, H and T' independently. With the likelihood defined in Equation [3] the following conditional posterior distributions can be derived.
Posterior of Zuij
N4 N4 if duis = 1 N (Kuij; 1) if duis = -11
Zuig E, H,
where N+ (4, and N N- are truncated Gaussian distributions on positive and negative quadrants respectively. Posterior of Eu with u 1,
E.| C., H.T No (2E()V)) >D)
where (21))-1 (WE) +1,
711 7/17111
T22 - T22
7k1Tk2 T6k2Tk1
MI1TI3 113TI
7/2 - T22
TRITKa T]k3Tk1
T
7/2T13 - 713T12
7/22 T23 - 1/23 T22
7k2 Mes Tk2
11. 1Im
7im 71mm1
72.m T2m
172m T2m
T.. Tkm
Mom Tk.
E 1m(mm11//2xk
and 25
[2u12, Zu13,
Zu23, Zw24.' ... -2,mm-1.m 1,m] Rm(m-1)/2
15

---

Published in Transactions on Machine Learning Research (023)
Posterior of 7/ with i i= 1....
W,, H Mis ~ (T)) TT 2x, E.),
where (E (V7) 1
{17Th
f21
Tar
f171
221
T-
f171,44
22172,011
fu1Tmm
Tkm
{12711
Eurtu1
ER2THl
f1271,111
2-272,1-1
V?
4171144
f2272.011
R 1)xk
f12Tim
finTi
E2u T21
Tk
Ean T2,51
Ekn Th
T2n
Tkitt
finTim
T2m
21a
Tha
2102
h=1 Ehi Th2
21,1111
21,11
Th.
Ziim +
Eii TasTiim
4
2211
Eh27H7711
22,6, +
(=(m-1)
22,4.41 + h=1 MieTraaee
22im E hum Exa7hahho
Znal D
h=1 Enn Tha
Zn,i,-1 h= Enn
Zn,itt11
h=1 i41
Znim >k A= Ehn ThaThim
16

---

Published in Transactions on Machine Learning Research (223)
In the above notation, take advantage of the anti-symmetric property of Zu: and assume Zuij Zuji when i >>> j.
Posterior of with j = 1, m
2, N ( E3).
where (Ez))- (VV)
E17
721
Ek1781 ExrMe
fum2
f21
f2172,111
Ea 7 f
f1m
f217
7am
f12111
E227/21
Exatjer
f12m1-11
fuzMa.j-1 522
V3
E 1)xk
Exalllon
Einm1
E2n7
Eka7
EleTi--1
Ean
Ekn M Ek i+
22
-2151
h=1 Thi
-2152 +
h=1 Th171
-11.5-1
-1 El 1
-11..+1
h=1 EN T
-21mm
EN Thm Mh) 112TH17A3
2251
E pmmm
-
N EN2Th.g-
-225.4+1 + h= 1127
- 222m
hum The Mij
-Znjji-1
Th 17b5
-kniss41
Enm Th.g-
Znjm
Dten ShnThm
17

---

Published in Transactions on Machine Learning Research (023)
Similar to 2,, we take advantage of the anti-symmetric property of Zu: and assume Zui -ufé when i> j.
C
Methods Considered in the Real World Application
Table Specifics hyperparameters for models used when applying to real world datasets.
Model
Parameter
Values
Implementation
Supplementary Code
Learning Rate
[0.001, 0.002, 0.005, 0.01, 0.02, 0.05, 0.1] [2, 10, 20, 50]
SAD
# Epochs
Reg
[0.05, 0.01, 0.005, 0.001]
I Reg
0.01
Implementation
Supplementary Code
Learning Rate
[0.001, 0.002, 0.005, 0.01, 0.020.0220000 0.05, 0.1] 2, 10, 20, 50] [0.05, 0.01, 0.005, 0.001]
BPR Rendle al. 2009
# Epochs
l2 Reg
Implementation
Surprise (package)
(uug
2020
Learning Rate
[0.001, 0.002, 0.005, 0.01, 0.02, 0.05, 0.1]
SVD
# Epochs
5, 10, 20, 50) [0.05, 0.010 0.005, 0.001]
Regularization
Implementation
Cornac (package)
Salah et al. 2020
Learning Rate
[0.001, 0.002,000000 0.005, 0.01, 0.02, 0.05, 0.1
Matrix Factorization (MF)
# Epochs
2, 5, 10, 20, 50] [0.05, 0.01, 0.005, 0.001]
Reg
Implementation
Cornac (package)
Salah al. 2020
Probabilistic Matrix Factorization (PMF)
Learning Rate
[0.001,0.0 0.002, 0.005, 0.01, 0.02, 0.05, 0.1] [,,5, 10, 20, 50] [0.05, 0.01, 0.005,00,000000 0.001]
MMiih & Salakhutdinov 2008
# Epochs
À Reg
Implementation
RankFM (package)
Learning Rate
[0.001, 0.002, 0.005, 0.0,
Factorization Machine (FM) Rendle
2010
# Epochs
2, 10, 20, 50] [0.05, 0,010 0.005, 0.001]
12 Reg
Implementation
MSFT recommenders (package
Graham et al. 2019
Learning Rate
[0.001, 0.002, 0.005, 0.01,0.022 0.05,
Neural Collaborative Filtering (NCF) He et al. 2017)
# Epochs
2, 10, 20, 50]
Batch size
[128, 256, 512, 1024]
Network
Three layers MLP with sizes [128, 64, 32
Implementation
MSFT recommenders (package) Graham al. 2019 [0.001, 0.002, 0.005, 0.01, 0.02, 0.05, (..05,0.1 0.1] 2, 10, 20, 50] 128,, 256, 512, 1024]
Variational AutoEncoder (A-VAE) Liang al. 2018
8 parameter
# Epochs
Batch size
Real World Examples
During LOO evaluation, true preferences between a hold out item and the rest user interacted items training set are determined based on users' actual ratings. Cases in which our model's predictions are consistent with true preferences while alternative models are not shown in Table 4 We select 10 such cases from each of the three real world datasets. Model predictions together with item descriptions are shown in the table.
18

---

Table 4: Examples where SAD produces a consistent prediction (Iusy > & Puij 0.5) while BPR fails (Buis 0 Puis < 0.5).
Published in Pransactions on Machine Learning Research (July 2023)
Fuij
Puéj
Dataset
u-th user
i-th item (rating)
j-th item (rating)
SAD
BPR
Netflix
'1381599'
Last of the Dogmen (5)
Look at Me (2)
0.35
0.59
-0.22
0.44
11243460*
Joe Kidd (5)
Scenes of the Crime (1)
0.80
0.69
-0.27
0.43
'581011"
The Professional (5)
The Bourne Identity (2)
0.80
0.69
-0.32
0.42
'632823'
Lara Croft: Tomb Raider: The Cradle of Life (4)
The Cookout (2)
0.27
0.57
-1.36
0.20
'429299'
The Worst Witch (4)
A Chorus Line (1)
0.85
0.70
-0.76
0.32
'127356'
Trading Spaces: Great Kitchen Designs and More! (4)
White Olcander (2)
0.30
0.57
-1.24
0.22
'2264661'
Free Tibet (5)
Native American Medicine (3)
1.25
0.78
-0.05
0.49
'581011'
The Dream Catcher (1)
The Bourne Identity (2)
0.90
0.71
-0.37
0.41
'1368371'
Zombie Holocaust (1)
Gothika (1)
0.71
0.67
0..11
0.42
12434460'
Moog (3)
Scenes of the Crime (1)
0.98
0.73
1.34
0.21
Movie-Lens
'1318'
High Fidelity (5)
Jimmy Hollywood (3)
0.85
0.70
1.26
0.22
1250'
American Beauty (5)
eXistenZy (3)
1.44
0.81
00.4
0.47
4116''
My Fair Lady (5)
Problem Child 2 (1)
0.51
0.63
-0.78
0.31
'153'
American Beauty (4)
Spice World (1)
1.07
0.75
-0.08
0.48
'2160'
Harold and Maude (4)
The Brady Bunch Movie (1)
0.34
0.58
-0.33
0.32
'4692'
Blade Runner (5)
The Newton Boys (3)
0.65
0.66
-0.38
0.41
6L
'2385'
Braveheart (4)
Voyage the Bottom of the Sea (3)
0.98 0.73
-0.56
0.36
'4756'
Galaxy Quest (3)
Felicia's 's Journey (2)
1.01 0.73
0..1
0.45
'4439'
The Maltese Falcon (4)
Entrapment (3)
0.45
0.61
0.57
0.36
'3021'
L.A. Confidential (5)
Fatal Attraction (3)
0.64
0.66
-0.37
0.41
Food-Com
148323'
Best Ever Banana Cake !ww Cream Cheese Frosting (5)
Crock Pot Garlic Brown Sugar Chicken (0)
0.16
0.54
3.51
0.03
4240088'
Glazed Cinnamon Rolls, Bread Machine (5)
Japanese Mum's Chicken (0)
0.31
0.57
0.38
0.41
'484223'
Crock Pot Stifado (5)
Best Ever and Most Versatile Muffins (3)
0.30
0.57
2..9
0.05
'733257'
Banana Banana Bread (2)
Low Fat Oatmeal Muffins (0)
0.32
0.58
2231
00009
'340980"
Funky Chicken !w Sesame Noodles (3)
Amanda's Thai Peanut (1)
0.56
0.64
-0.77
0.32
'573772'
Delicious Chicken Pot Pie (5)
Amish Oven Fried Chicken (1)
1.28
0.78
-1.1
0.23
'14775440'
Cinnabon Cinnamon Rolls by Todd Wilbur (4)
Amanda's Cheese Pound Cake (0)
0.81
0.69
2.59
0.07
'268644'
Baked Tilapia \w Lots of Spice (5)
Southern Fried Salmon Patties (2)
0.38
0.59
3.35
0.03
'762742'
Easy Peezy Pizza Dough & Bread Machine Pizza. Dong (5)
Fresh Orange Muffins (1)
0.98
0.73
-2.31
0.09
'212558'
Steak Chicken Fajitas (5)
Thai Style Ground Beef (3)
1.86
0.87
-0.11
0.47


Reviews:
[{"review_id": "tXGJL8tF5X", "reviewer": "Reviewer_WWns", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: \n1. The paper follows a good flow. \n2. References appear to be sufficient.\n3. Experiments conducted on multiple datasets and compared against seven existing approaches.\n\nWeaknesses:\n1. A lot of missing detail -- please see requested changes below.\n2. The notation could be better -- see below.\n3. Minor editing errors -- see below.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a novel tensor decomposition based model for collaborative filtering based on implicit feedback. The proposed tensor decomposition is named sliced anti-symmetric decomposition (SAD). In contrast to standard methods that rely on vector representations of users and items, SAD forms a three-way tensor by introducing an additional latent vector for each item, thereby capturing higher order interactions between user-item pairs. The newly introduced latent vector per item allows for the notion of users having a non-linear mental model when evaluating models, as opposed to a linear mental model assumed by existing approaches. Experiments are performed on several synthetic and real-world datasets and the proposed method is compared with seven existing methods. ", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Pg. 1, second paragraph, presents the statement \"It is therefore more natural to assume that non-interacted items are a combination of the ones that users do not like and the ones that users are not aware of.\" Are the authors claiming this or re-stating this? If the authors are re-stating this, the original source needs to be cited. If not, please see if your experimental results agree to this statement. \n\n2. $d_{uij} \\\\in \\\\{-1, 1\\\\}$ is used to represent if a user prefers the $i$-th item over the $j$-th or not. Could that authors clarify if the context of +1 or -1 here? That is, is +1 used for preference and -1 for otherwise?\n\n3. Pg. 2 has the sentence \"The model used in BPR assumes that among the observed entries in D, the probability that the u-th user prefers the i-th item over the j-th item can be modeled as a Bernoulli distribution\". Can the authors please cite the source?\n\n4. In (2) do we assume $x_{ui} > x_{uj}$? Is $x_{uij} > 0$. Please clarify.\n\n5. The second set of non-negative item vectors here are denoted as $\\tau_i$ for item $i$, can the authors give some insight onto what values could the entries of $\\\\tau_i$ take?\n\n6. Pg. 3 has teo sentences that slightly contradict each other -- \"Compared to traditional collaborative filtering methods\nusing matrix factorizations, tensor decompositions have received less attention in this field.\" and \"Recently, tensor decomposition methods have been used to build recommendation systems using information from multiple sources.\" Could the authors please re-phrase?\n\n7. Pg. 4 -- notation: vectors and scalars are denoted by lowercase letters and uppercase letters are reserved for matrices and tensors. This makes it hard for the user to identify the context in the text. The authors could consider simplifying notation by using lower case for scalars, lowercase boldface for vectors, uppercase boldface for matrices, and overline uppercase for tensors. E.g., $u$, $\\\\mathbf u$ $\\\\mathbf U$, and $\\\\overline{\\\\mathbf{U}}$.\n\n8. It would be beneficial to see the first order slice of three-way tensor X corresponding to user $u$ as a graphical representation. Maybe show a cube (three-way tensor) and highlight the first order slice.\n\n9. In Pg. 5, the notation used in (7) for SAD is commonly used for CP-decomposition. Is it possible to change the notation? If not, please clarify that in this paper, the notation in (7) is reserved for SAD.\n\n10. Figures 1 and 2 appears way ahead of its corresponding text. Please move it closer to the corresponding text. Also, can we look at quantitative number for Figures 1 and 2? Maybe MSE?\n\n11. Use \\eqref in the first line of section 4.3 for 2 and 3.change (Equation 4) to (4) in the second paragraph.\n\n12. (8) consists of two $\\\\tau$s -- $\\\\tau_i$ and $\\\\tau_j$. Where do they come from? It is not clear how we got to (8). Please provide clarification.\n\n13. The text claims that SAD learns the value of T from data but that is not the case in experimental studies. That is, T is set explicitly and not learnt from data. Please re-phrase to be consistent with experimental studies. \n\n14. Figure in Pg. 7 is not names. The title is missing. Should it be Figure 3? Many notations in this figure needs to be explained. For example, what are $p$, $p_{\\\\theta}$, what is the relationship between $D$ and $X_{true}$? Please clarify. \n\n15. The figure in Pg. 7 corresponding to Sim1 sets T=1 and for this case, SAD must simplify to BPR, however, the performances of SAD (with T=1) and BPR are not the same. Why so? On the other hand in Sim 2, T is not equal to 1, so the behavior of SAD and BPR are expected to be different, but they appear similar, especially near convergence. Why? Please clarify.  Also please label the vertical line in (b) \nas \"true sparsity\".\n\n16.Please provide a pseudo-code of the proposed algorithm. This pseudo-code would contain (9) in it. \n\n17. In section 5, please provide the dimensions of T. Please clarify how the hyper-parameters (learning rate, weight of L2 regularization, etc.) were selected? The number of latent factors k is set to the \"true value\". How can k be estimated in practice?\n\n18. In Table 1, please define match, per user M1 and M2 in the title. Provide references next to the model names, that are compared against. \n\n19. The dimension of the latent factor in real-world datasets is set to 500. Why? Can we cross-validate to find a dimension that works well?\n\n20. This paper proposes pairwise comparisons for evaluation, namely, M1 and M2. Can the authors also report results on standard metrics, if any?\n\n21. The discussion in section 7, mentions \"When making recommendations,\nthe items forming a cycle can be selected as a group, ...\" What does this mean? This is not mentioned anywhere in the maintext.\n ", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "7655dM60wD", "reviewer": "Reviewer_Kngd", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "My view on the proposed model requires to rewrite Equation (8) as $x_{uij} = \\sum_h \\xi_{hu}(\\zeta_{hi} - \\zeta_{hj})\\tau_{hi}\\tau_{hj},$\nwhere $\\zeta_{hi} = \\frac{\\eta_{hi}}{\\tau_{hi}}$. Let me denote (10) this equation.\n\n\n### Strengths\n\n* **A new model that can simultaneously model the ranking of items and the (non-)observation of this ranking**. As in BPR, the term $\\zeta_{hi} - \\zeta_{hj}$\nin equation (10) measures the extent to which item $i$ is preferred over item $j$ given feature $h$ ($\\eta_{hi} > \\eta_{hj}$), and the weighted sum of these terms (weighted only by $\\xi_{hu}$) indicates whether item $i$ is preferred over item $j$ given user $u$. The addition of $\\tau_{hi}\\tau_{hj}$ allows the reduction of $x_{uij}$ when two items have a small probability of being present in the data. Indeed, when two items are present in the data, $d_{uij}$ is either -1 or 1 and can be fitted with $x_{uij}$ having a large value, while when one of the two items is not present in the data, even if there is a clear advantage for one of the two items, the observation is $d_{uij}=0$, forcing $x_{uij}$ to be close to 0.\n\n### Weaknesses\n\n* **Equation (8) is misleading and the narrative would be more accurate if based on Equation (10).** \n* **The paragraph \"While respecting the anti-symmetricity [...]\" does not fit the current model.** In fact, in the current model, the term $x_{uij}$ is derived only from elements $i$ and $j$: other elements have no effect, the respective positions of the elements in the recommendation have no effect. To handle such a property, one has to look at models like in [1].\n* **The contribution of the simulation study is limited.** Indeed, the parameters $n$, $m$, and $k$ are tiny compared to what can be expected from real data. Moreover, unlike the authors, I see BPR outperforming the proposed approach (SAD) in both settings when the percentage of missing information is high, and I cannot distinguish a leader when the percentage of missing information is low (while SAD should outperform BPR in setting (sim2)).\n* **The contribution of the section \"Applications for Real Data\" is unclear.** First, the presentation of the criterion \"consistency\" is unclear, which forbids any conclusion from this criterion. Second, I do not see what information \"mean $x_{uij}$\" brings. Third, both consistency and recommendation seem to be related to the ranking of items, I do not understand what additional information the second criterion brings. Fourth, the use of non-interacted items to build a criterion is controversial because we do not know if they are non-interacted because the user is not interested in them or because he has not had the opportunity to consider them.   \n* **The paper should assess its relationship to the implicit feedback literature.** In my opinion, the main contribution of the paper is a way to \"hide\" the comparison between items when the comparison is not present in the data. This is typical of the implicit feedback literature. The paper should refer to this literature, compare to the state of the art article in the experimental part, and consider standard experimental protocols in the field. \n* **Details**\n\t* Equation (5) is the same as the equation in section 4. I do not see its purpose\n\t* Why Figures 1, 2 and 3 are plotted much before they are referenced ?\n\t* Figure 3's caption is missing\n\t* The reference is missing in the sentence \"Some details of the three datasets can be found in\"\n\n\n### Bibliography\n[1] Lijing Qin, Shouyuan Chen, and Xiaoyan Zhu. Contextual Combinatorial Bandit\nand its Application on Diversified Online Recommendation. SDM'14\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The article is interested in recommender systems. It introduces an extension of the BPR model, where the scalar $x_{uij}$ governing the probability that user $u$ prefers item $i$ over item $j$ is rescaled by a new set of parameters $\\tau_{hi}\\tau_{hj}$. In the experiments on 3 datasets, the proposed model enjoys a better consistency in terms of pairwise ranking than 7 other state-of-the-art models, including BPR. \n\n", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "* (Critical) Rewrite the narrative of the proposed approach based on equation (10).\n* (critical) Remove the paragraph \"While respecting the anti-symmetry [...]\".\n* (strengthening the work) Rethink the section \"Simulation study\".\n* (critical) Clarify the section \"Applications for real data\".\n* (strengthen the paper) Add comparison with implicit feedback literature\n", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "sOdzImjZyG", "reviewer": "Reviewer_aiYy", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n1. This paper studies the transitivity property followed by most existing collaborative algorithms. It's an interesting topic that has been overlooked. The argument is that people's mental model of evaluating items does not follow this property is valid and well explained. It's intuitive that cycles in pairwise comparisons may happen a lot in reality. It's important to model this phenomenon.\n2. The authors conducted experiments over multiple datasets with reasonable size. They also compared their approaches with multiple baselines.\n3. Most content of this paper is well-explained and easy to follow.\n4. The M1 and M2 metrics used in evaluations use pairwise comparison to rank items. It's nice to see the authors develop them as their proposed model relies on pairwise comparison while most evaluation metrics, which are developed for listwise ranking, may not be the best to evaluate their models.\n\nWeaknesses\n1. While the problem studied in this paper is interesting, the proposed solution is not very impressive. \n    * First of all, compared with most recent advancements in recommender systems, the proposed approach is very simple and incremental.\n    * Secondly, if we stick with the approach of adding one additional matrix to BPR, the proposed solution of modeling the third matrix as another item matrix may not be optimal. One alternative is to model the third matrix as an interaction matrix between item $i$ and item $j$, so $x_{uij} = \\sum_{h=1}^{k}\\xi_{hu}(\\eta_{hi} - \\eta_{hj})\\tau_{hij}$. $\\tau_{hij}$ can be further modeled as the inner product of two vectors, one from item $i$ and one from item $j$. This modeling approach is more intuitive to me than the one proposed in this paper as it assumes that which aspect (latent dimension) the user care about is determined by some interactions between the two items. \n2. While it's a strength that the authors use multiple baselines for comparison, it would be better if they can consider more variations of BPR and matrix factorization/collaborative filtering. Researchers have developed tons of algorithms to extend them and some have considered introducting additional matrices to it, such as https://arxiv.org/pdf/1711.08379.pdf.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors developed a new collaborative filtering algorithm, Sliced Anti-symmetric Decomposition (SAD), for recommender systems. This new algorithm extended Bayesian Personalized Ranking (BPR) by introducing one additional latent vector to each item. Collapsing this additional latent vector to 1 will result in the standard BPR model so this new algorithm can be considered as a generalization of BPR and matrix factorization. Allowing it to have values other than 1 can model users' nonlinear mental models when evaluating items by violating of the transitivity property that is followed by most existing collaborative filtering algorithms.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. I would suggest the authors to further push their proposed approach to be more innovative. Combining the idea in this paper with what they mentioned in their future work, integration with neural network components, may be a good direction to explore.\n2. Some typos need to be fixed, such as\n    * \"we show that SAD is not only able to more\" in Abstract.\n    * \"Some details of the three datasets can be found in. \" in Section 6.\n    * missing caption of Figure 3.\n", "broader_impact_concerns": "None.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: In this paper, the authors propose Sliced Anti-symmetric Decomposition (SAD) for collaborative filtering based on implicit feedback. SAD extends the Bayesian Personalized Ranking (BPR) model by introducing one additional latent vector to each item. In this way, users could use nonlinear mental models when evaluating items, allowing the existence of cycles in pairwise comparisons. Finally, the authors demonstrate the efficiency of SAD in both simulated and real-world datasets.

The proposed SAD method is novel and interesting. After discussions, the reviewers are generally positive with this paper. However, there still exists one concern regarding the dataset used in the experiments, which is shown below. When preparing the revision, the authors need to provide more discussions, and more experiments if possible.

---
The experimental part is done on data that do not contain the properties that the proposed model aims to capture. This section would be much more convincing if using data containing partial ranking on items and therefore may contain non-transitive relations, for example [1,2] or more recent.

1] Yandex personalized web search challenge. https://www.kaggle.com/c/yandex- personalized-web-search-challenge, 2013.

[2] KDD Cup 2012 Track 2. http://www.kddcup2012.org/

Decision: Accept (poster)