Explainable Recommender with Geometric Information Bottleneck
Authors: Hanqi Yan
Lin Gui
Yulan He
Venue: Submitted to ICLR 2023 (2023)
Keywords: Interpretability
Recommender System
Information Extraction

Abstract: Explainable recommender systems have attracted much interest in recent years as they can explain their recommendation decisions, enhancing user trust in the systems. Most explainable recommender systems rely on human-generated rationales or annotated aspect features from user reviews to train models for rational generation or extraction. The rationales produced are often confined to a single review. To avoid the expensive human annotation process and to generate explanations beyond individual reviews, we propose an explainable recommender system trained on user reviews by developing a transferable Geometric Information Bottleneck (GIANT), which leverages the prior knowledge acquired through clustering on a user-item graph built on user-item rating interactions, since graph nodes in the same cluster tend to share common characteristics or preferences. We then feed user reviews and item reviews into a variational network to learn latent topic distributions which are regularised by the distributions of user/item estimated based on their distances to various cluster centroids of the user-item graph. By iteratively refining the instance-level review latent topics with GIANT, our method learns a robust latent space from the text for rating prediction and explanation generation. Experimental results on three e-commerce datasets show that our model significantly improves the interpretability of a variational recommender using a standard Gaussian prior, in terms of coherence, diversity and faithfulness, while achieving performance comparable to existing content-based recommender systems in terms of rating prediction accuracy.  

Under review as a conference paper at ICLR 2023
EXPLAINABLE RECOMMENDER WITH GEOMETRIC IN- FORMATION BOTTLENECK
Anonymous authors Paper under double-blind review
ABSTRACT
Explainable recommender systems have attracted much interest in recent years as they can explain their recommendation decisions, enhancing user trust in the systems, Most explainable recommender systems rely on human-generated ratio- nales or annotated aspect features from user reviews to train models for rational generation or extraction. The rationales produced often confined single review. To avoid the expensive human annotation process and to generate ex- planations beyond individual reviews, we propose explainable recommender system trained on reviews by developing a transferable Geometric InformAtioN boTtlennck (GIANT), which leverages the prior knowledge acquired through clus- tering on a user-item graph built on user-item rating interactions, since graph nodes in the same cluster tend to share common characteristics preferences. We then feed user reviews and item reviews into a variational network to learn latent topic distributions which are regularised by the distributions of user/item estimated based on their distances to various cluster centroids of the user-item graph. By iteratively refining the instance-leved review latent topics with GIANT, our method learns a robust latent space from text for rating prediction and explana- tion generation. Experimental results on three e-commerce datasets show that model significantly improves the interpretability of a variational recommender us- ing the Wasserstein distance while achieving performance comparable to existing content- based recommender systems in terms of rating prediction accuracy.
INTRODUCTION
Typically, a recommender system compares users' preferences with item characteristies (e.g., item descriptions item-associated reviews) studies user-item historical interactions (e.g., ratings, purchases or clicking behaviours) in order to identify items that are likely of interest to users. In addition to predictive performance, interpretable recommenders aim to give rationale behind the rating given by a user on an item (Ghazimatin et al. 2020 Zhang et al., 2020). Most existing interpretable recommenders can either generate rationale extract text spans from given user- item review as explanations of model decisions. Both rationale generation and extraction require annotated data for training, e.g., short comments provided by users explaining their behaviours of interacting with items, or annotated centiment-bearinn aspect spans in reviews (Zhang et al. 2014 Ni et al. 2019 Chen et al. 2019 ei al., 2020; Tan al. (02]aa)
We argue that generating explanations based on a specific user-item review document suffers from the following limitations. First, some reviews may be too general to explain the rating, rendering them useless for explanation generation. For example, the review 'I really like the smartphone, will recommend it to my friends' does not provide any clue why the user likes the smartphone. Second, features directly extracted from review document may fail to reflect some global properties which can only be identified from implicit user-item interactions. For example, meaningful insights could still be derived from reviews towards items that are not directly purchased/rated by user but preferred by other like-minded users. Finally, explanation generation model from user/item reviews are often supervised by human-annotated rationales, which are labour-intensive obtain in practice.
To address the aforementioned limitations, we propose an AutoEncoder (AE) framework with vari- ational Geometric InformAtioN bo Ttleneck (GIANT) to incorporate the prior from user-item inter- action graph to refine the induced latent factors of user and item, and generate explanations in an unsupervised manner. For user-item pair, all reviews written by the user and reviews posted on the

---

Under review as a conference paper at ICLR 2023
U-I Graph
Geometric Information Bottleneck Regularisation
Recommendation Explanations from related reviews
Reviews from item
X
Don't miss the enarge tope This Tope really fits the bill
Encoddey
Z
Decodar
1
Based on reviews from This tope is lsftt transparent but is strong.
Preedct Rating
Reviews from user
Head
Based on the reviews from like-mined users. Byg Fibergiass threads, impossible to tear by hand, and sirong athasive that will never fall off will recommend this Hem
taps
Eneyped
Dacodar
(M)
2
with this product
donit you could go wrong
Figure I: An encoder-decoder structure with a geometric information bottleneck regularisation, which is derived from the U-I interaction graph and used as prior imposed Z. The latent variable 2 thus can capture the geometric affiliations in graph and be the soft cluster assignment distributions. It enables the of like-minded users Ug and similar items ip and for generating explanations beyond the imput user-item pair (i1, U2). Existing rationale extraction methods can only extract indicative words (shown in red) from the given review pair. Our method can generate rationale from reviews written by other like-minded users or on similar items, which are assigned into the same latent dimension/cluster.
item are fed into two separate encoders to infer latent factors, and predict rating based on the match of their latent factors. The latent variables are supposed to capture the key semantic information recover the original review text written by the user on the item. Different reviews assigned to different latent dimensions according to posterior distributions and we extract the representative words in each dimension summarize the cluster topic. The explanations are thus from the reviews related to the topic of the assigned cluster (See 6..3
Our proposed framework is illustrated in Figure 1 The geometric regularisation refers the cluster- based distance with Gaussian variance, which is derived by firstly clustering the users/items in the user-item interaction graph, and then caleulating the distribution of user/item its distance each cluster centroid by a Gaussian kernel ($ 4.1). To impose the regularisation on the latent variables of the AE framework taking the set of user reviews and item reviews input, we apply the KL divergence to minimize the discrepancy between the induced posterior distribution of users g(x), and items 201, and the geometric regularisation as the prior ($4.2). after linking the cluster to the review text encoder via prior-centraliiatton ($4.2).
Experimental results on the three commonly used benchmarking datasets show that the proposed method achieves performance comparable with several strong baselines in recommendation. More- over, the quantity and the quality analysis in interpretability show that our method can generate coherent, diverse and faithful explanations.
2 RELATED WORK
We review recommender systems, with particular attention to those built on Variational Autoencoder (VAE) (Kingma & Welling 2014) offering explanations of recommendation decisions.
Recommender with variational autoencoder Text data such user reviews, item or brand de- scriptions could be important for developing high-quality recommender system as they can be exploited address the sparsity issue in user-item interactions. HFT (McAuley & Leskovec 2013)
and CTR (Wang & Blei 2011) adopted Latent Dirichlet Allocation (LDA) (Blei et al. 2003) to
extract latent topics from review text. VAE can also be used for text modeling due its ability in extracting latent and interpretable features (Fei et al. 2021] Truong et al.) 2021 Wang et al. 2020)
Truong et al. (221)) argued that the commonly used isotropic Gaussian VAE is over-simplified and proposed BIVAE by introducing constrained adaptive prior (CAP) for learning user- and item- dependent prior distributions. More recently, review information is used to disentangle the latent
user intents at the finer granularity, for example, DisenGCN (Ma et al. 2019a) and DisenHan (Wang
al. 2020 used the graph attention mechanism to differentiate multiple relations and features.
Explainable Recommender System One popular interpretable method is train a language gen- eration model with the ground-truth explanations supplied, which can be the first sentence of a given review human-annotated text spans in the review text (Li et al. 2017 Chen al. 2019. Ni al.
2

---

Under review as a conference paper at ICLR 2023
2022). Feature-level explanations aim to provide the important item features the users care about when making decisions Zhang & Chen 2020). Explicit Factor Model adopted Matrix Factorization (MF) to predict review ratings and aligned each dimension of latent factors of the MF results with some item features, thus explaining the recommendation decision by the corre- sponding features (Zhang al. 2014]. Other approaches firstly processed the review text to extract user-aspectoopnioon and item- aapea---tct-oiiinn tuples. They then modeled the interactions between the two tuples to derive the rating and the indicative features (Chen et al. 2016; Wang et al.) 2018 TTan et al.] 202 Chen al., 2020). However, the aforementioned approaches suffer from two ma- jor issues. Some approaches require aspects to be extracted first in order to provide aspect-related predictive features as interpretations, thus heavily dependent on the quality of the aspect extractors employed; while others rely on human-anen annotated rationales to train supervised language generators for explanation generation. Our proposed approach can instead generate explanations with the only supervision signals coming from review ratings.
2019 Li et al.
3 GEOMETRIC INFORMATION BOTTLENECK
To consider implicit user-item interactions for developing interpretable recommender system, we propose to incorporate the geometric regularisation derived from user-item interaction graphs to learn the latent factors of users and items from review text in variational network. Although various work has been proposed to adopt richer priors to infer a more complex and realistic posterior in standard VAEs (Tomczak & Welling 2017 Zhao et al. 2017). our solution is different from existing approaches as we need to impose the prior knowledge learned from the graph modalíty to constrain the learning of latent variables in the text modality. In what follows, we show how this can be done under the theory of information bottleneck.
Based on the Information Bottleneck (IB) theory (Tishyy & Zaslavsky 2015), can train an encoder-decoder architecture I encode, decode, I, where I E X, 2 E Z. X are the input, the hidden, and the reconstructed representations, respectively, in order to preserve the meaningful information about X in Z while maximally compress the information of X. We achieve it by maximising the following equation:
On = 2) - (((;)))
(1)
where I(.) denotes the mutual information, ,8 is a Lagrange multiplier. The second term in Eq. (1) is to maximally compress X, equivalent to minimize the mutual information between X and Z, denoted as [(X; Z) S Les Ie is the upper bound.
In this paper, the goal is to build a mechanism which able to compress the information across different modalities, i.e., the text and graph, while keep the mutual information between the input X and compressed representation Z bounded by an upper bound as well.
Intuitively, we assume the n-th input x,, has representations xf, E X' and 2g E X under different sataaiiiin modalities satisfying the constraint, T(X X9) 2 Ta. since X' and X are derived from the same input, they should be relevant. Then, two independent well-trained encoder-dccoder architectures in the two modalities optimised by Eq. { are constrained by [Xx: ; 2t) I La and 1 1(X*; 29) < Le respectively. We derive a new upper bound for the multi- molti-modality as follows
[(x:; Z9)= - H(X*) - (Xx) - ( XX) + 1.
(2)
where H(-)) and H(-1-)) are entropy and conditional entropy, respectively. Therefore, imise the objective function using the input from the text modality, with a regularisation from another modality such user-item interaction graph, but still maintain new upper bound.
max-
0 I(X;; Zz) - 3- [(X:; 22).
(3)
According to (Alemi et al. 2017). the above lower bound can be approximated by applying the reparameterisation trick (Kingma Welling. 2013) with a random Gaussian noise e:
O = Ee-p(e)) logg(z") [f e)N/N + B (L(p(Zz |at (Z)r)
(4)
Recous, term: Q(x)))
KL-div, term: KL(Z)|29)
In the rest of paper, we use for text and g for graph. "The proof is shown in the Appendix B
"The proof is shown in the Appendix
3

---

Under review as a conference paper at ICLR 2023
where z/ is the training sample from one modality X (e.g., text), ž is the VAE-based recon- struction output, g(-) and p(-) are the posterior probabilities estimated by decoder and the encoder, respectively, r r(Z%) is the estimated distribution from another modality (e.g.. user-item interaction graph). Here, Eq. (4) has a similar form as 3-VAE which contains a reconstruction term Q(a))) defined by the posterior probability and KL-divergence term KL <L(ZZZZ)) regularised by the prior from another modality Z By this equation, we are able to guarantee that the information bottleneck between two modalities are:
Transferable - According to Eq. (2). the information bottleneck between modalities is still constrained by the global information bottleneck, i.e., can take the bound from one modality to guide the training on the other.
Practicable One challenge in the training of variational models is the choice of the prior distribution. By Eq. (2), we do not have to utilise a pre-defined prior but instead use the posterior constraints from other modalities, which are more flexible and efficient.
Interpretable The alignment between representations from different modalities and their corresponding probabilities defined in the geometric space makes it possible to use modal- ities offering better interpretability, such as review text, to explain the black-box features from other modalities.
4 GIANT FRAMEWORK
We propose a novel explainable recommender built on VAE with the geometric information bottle- neck. For given (user, item) pair, the input our recommender is a of reviews written by the user and a set of reviews on the item. As each user or item associated with multiple reviews, and some reviews might be very long, the Convolutional Neural Network (CNN) is used to encode reviews due to its efficiency in encoding long sequences, as have been previously studied in Chen eet al. (2018); Zheng al. (2017). We then generate the user or item representation by aggregating their associated multiple reviews by the attention mechanism. The final contextual vector for user u and item i denoted as (.(2) and (,) respectively
Given the contextual vectors, we employ two geometric regularised variational networks to infer their latent factors, in order to better capture their latent semantic topics. According to Eq. ( we use graph-based 2 as the transferable information bottleneck to regularise the optimisation of the text-based Z². To ensure feasibility and robustness, we propose an initialisation trick called prior-centralisation which uses the Gaussian kernel based geometric estimation to regularise the initialisation and optimisation of the encoders based on the transferable information bottleneck. In the experiments section, we verify the interpretability of our proposed method by the alignment between the two latent feature spaces from graph and text modalities.
4.1 DERIVING PRIORS FROM USER/ITEM CLUSTERS INTERACTION GRAPH
In the user-item bipartite graph G, node can be either a user or an item. For each user-item pair, an edge is created if the user has previously rated on the item and the rating score is higher than the average rating score calculated across all user-item ratings 4 We use the LightGCN (He et al. 2020 to encode our interaction graph and obtain the user and item embeddings.
Once user/item embeddings are learnt, we apply K-means on the learned node embeddings to derive
the cluster centroid vectors, denoted c(u) for users and C( for items, à f1, ..., ,)) We use the Radial Basis Function (RBF) kernel to compute the distance between a user or item with the cluster i as the probability of assigning the user or item the 1--th cluster, Pyk:
exp (
(-lleg
-cxlocll /2a²)
Cx
(5)
-1 exp e9 Cx
122 /202)
where e is graph node embedding for user or item j. Cg k-th cluster centroid vector for user embeddings item embeddings, is a hyper-paramette adjusted according to the data density.
'We have also experimented with the creation of edges for each observed user-item pair, but observed worse nerformance.
4

---

Under review as a conference paper at ICLR 2023
The cluster-based distribution pa {p is then used to regularise the latent factors for users and
items, as will be shown $ 4.2 Moreover, we have ΣEPýA = 1, that is, the distribution of { p2 resides in hyper spherical cap area on uniform K-ball, weakly approximating
Gaussian
distribution when is large (Diaconis & Freedman] 1987). We therefore obtain a latent vector Z and corresponding geometrie space where the difference each basis follows Gaussian prior.
4.2 A POSTERIOR OVER CONTROLLABLE DISTRIBUTIONS FROM TEXT
In section 4.1 we estimate the distribution of 2 from a user-item bipartite graph. The next step is to optimise the objective of Eq. to infer the latent factors of users and items, 2(4) and 20)) in text modality. In this section, perform the following two steps:(1) minimising the lower bound of OBB a by a RBF-kernel estimated posterior probability; and (2) prior-centralissttoon term which encourages the encoder weight matrix in the variational network to be closer the cluster centroid representations learned the interaction graph, so that the learnt latent variables naturally capture the cluster-based distances.
Optimising Conditional Probability by Kernel Density For the ((a)) Eq. 4 which is de- fined as the conditional probability of loga(z, [1 ))), we use the Nadaraya- Watson estimator
(Hall et 1999):
p( c: xth)
1 N
== Ki 1
16
Q(x")) = llog ((zh)
log
h
(6)
N. Ž== N
where k(-) is a kernel function for density estimation, #t, is the reconstructed z with random noise In practice, we choose the RBF kernel к exp(-llx x' =)( and ignore the denominator since the goal is to optimise the decoding of sh, then we are able to obtain the following bound by the triangle inequality in the learned metric space as ((*%) x log(exp(-1]zz)
xx, )]. To
simplify the computation, we choose natural logarithm function optimisation and obtain a mean squared error based objective function O
1 N Mempte
OBB
ll +111++ (LL
where of E {1 J denote the input user and item representations, and o denote the reconstruction. By minimising the loss, we can preserve the key local similarity of nearby represen- tations while ensuring different representations from different feature characteristics (Czolbe et al.) 2020 Wang et al. 2004). Note that we stop the gradient back pook-propagation
the input x, and optimise the encoder-decoder parameters only, because the updating of input representations might reduce the relevance between modalities, break the assumption of T(X X*) 2 Jax and lead two independent VAFs degeneracy of transferable information bottleneck given by Eq. 2
Minimising Distribution Discrepancy To minimise the second term of KL-divergence in Of we need to project 2 and Z' to metric space with the same size. Inspired by (Van der Maaten & Hinton 2008), a Student t-distribution used mitigate the crowding problem. Here, we use the softmax activation with temperature r to adjust the distribution tail and map its values to the range of [0, 1]. The k-th dimension of latent variable 2f denotes the probability of the corresponding user or item being assigned to the k-th cluster:
1 1+ ((tee))
Trak log This
The
KL(p(Z'
())) (L()(2))ch))))(22))
ΣΣ
ΣR- Ek_ D/7
Puk
k=1
where x0())) is linear transformation of 2 Here minimise the KL-divergence between Pi the user-item interaction graph space) and nfg- Therefore, we rewrite the O/p as:
(in
1 N B 1 = N Bepll 11
The M log
2 2 +1
(9)
P
1=1
'The detailed discussion can be found in Appendix C 'Here, either be index
item index
5

---

Under review as a conference paper at ICLR 2023
Prior-centralisation: Linking Clusters with Encoder Weights In the encoding progress, ob- tain the latent variable 2/1 = f e) by applying the reparameterisation trick (Kingma & Welling 2013). However, due to the randomness in initialisation, the learned distribution of 2, might be chaotic. Therefore, in our one-layer MLP based encoder, we propose a prior-centralisation trick which builds the connection between the pre-trained clusters and the encoder by making the en- coder weights close to the cluster centroids C9 E Rd obtained from the graph:
K
Reentroid
Σ
W en C
(10)
where W
en denotes the k-th column of the encoder weight matrix Wen and Cg denotes the rep- resentation of the lc-th cluster centroid vector. Note that the Reentroid will force the distribution of zn approximate to the prior and ignore the input. Therefore, in practice, this regularisation is only deployed in the initial training epochs.
Rating Prediction and Final Objective Function After obtaining the regularised latent contex- tual representations 2(a) and 2(1), we add their corresponding ID features to obtain the final user and item embeddings Su. and Çi for rating prediction, Çu W. (a) + Eus C W =/( + EG, where the ID features E, and are generated by feeding the user ID and item ID to an embedding layer. Inspired by the latent factor model in recommendation systems, we introduce the global biases for users and items in the final prediction layer as f.i fela(Su. (i) + b, + 6; where fels combines the user and item features into a scalar, by and b, are bias derived from E,, and e;. The regression loss L, of the predicted rating is calculated as the MSE on the given user-item pair. Combining all the components above, we derive the training objective follows:
C = Lr + OiB + Roentroid
(11)
5 EXPERIMENTS
5.1
EXPERIMENTAL SETUP
The evaluation datasets include BeerAdvocate (McAuley et al. 2012) and two amazon review datasets, Digital Music and Office Products (He McAuley, 2016) We use Root Mean Squared Error (RMSE) and Mean Absolute Error (MAH) to evaluate the rating prediction accuracy.
Datasets and Metrics
Baselines We compare with several open-source recommenders, including HPT (McAuley & Leskovee 2013), DeepCoNN (Zheng et al. 2017), NARRE (Chen et al., 2018). HPT combines reviews with ratings and uses exponential transformation function to link review text and the ratings. DeepCoNN uses shared layer for interaction modeling the users and items, which is on top of the two encoders for the users and items, respectively. NARRE introduces review-level atten- tions to select important reviews and incorporates the user and item IDs as discriminative features in rating prediction. Besides, we apply an encoder-decoder AutoEncoder on top of NARRE to learn the latent variable baseline highlight the difference in our proposed regularisation.
Training Procedure We train the model parameters by minimising the objective function defined Eq. (T.. The 3 for the KL divergence term set to be 0.01 and we introduce the L2 regularisation for all the model parameters and the weight is 0.001. To We only train the prior-centralisation term for 0.5 proportion to approximate the cluster centroids for the later training. The KI. term in OιB should be introduced later until the encoder centroid is well trained. To do so, we follow a similar cyclical schedule (Fu et al. 2019) to gradually adjust the anneal factor each epoch. Specifically, we first train the model without the KI. term for 0.5 proportion, then anneal it from 0.5 to for 0.25 proportion, and finally fix 1.
We also experiment with minimising the distance between decoder weight and cluster centroid, as the encoder and decoder are asymmetrical.
'The dataset statistics and pre-proe mreproccsssing details can be found in Appendix A.1
based based evaluation results, ie.. the overlapping between the recommended items and the target items are shown in Appendix D.I
'The ranking-
The hyper-parameter settings are described in Appendix A.2

---

Under review as a conference paper at ICLR 2023
BeerAdvocate
Digital Music
Office Products
Table I:Performance com-
Models
parison in RMSE (%) and NAE (%) for all meth- ods. denotes the statis- tical significance for p <
RMSE()
MAE(U)
RMSE()
MAE(()
RMSE()
MAB(U)
HFT
79.81
62.42
96.42
74.76
89.46
68.57
DeepCoNN
77.28
59.45
94.69
71.03
85.14
64.82
NARRE
76.80
58.94
93.69
69.30
84,40
63.40
AutoEncoder
75.94
58.93
93.97
69.13
85.03
64.32
0.01, compared to the
GIANT
75.36'
57.87
92.87*
68.68"
84.32
62.05'
best CNN-encoder based baseline.
5.2
RATING PREDICTION RESULTS
5.2.1 MAIN RESULTS BY COMPARING TO BASELINES.
The rating prediction results of our model in comparison with baselines on all datasets are given in T, We have the following observations. (1) DeepCoNN built on the stack of non-linear neural networks for review semantic modeling outperforms HFT which leverages an exponential transfor- mation function to link topic distributions review text and latent factors derived from ratings. This shows the superiority of deep learning for feature extraction. (2) AutoEncoder gives slightly better performance than NARRE in BeerAdvocate, which shows its effectiveness of extracting key contex- tual information for rating prediction. (3) Our method consistently outperforms all the baselines and the improvement more predominant on the BeerAdvocate, which has the smallest sparsity. The results demonstrate the effectiveness of our proposed information bottleneck regularisation applied on the latent semantic space. By doing this, the users and items can be grouped into different clusters according to the interaction data, which is not the case in other textual CNN-based recommenders.
5.2.2 PERFORMANCE CONTRIBUTIONS FROM VARIOUS MODULES.
a) Effects of different loss terms We study the effects of the three loss terms in rating prediction accuracy and Diversity of the latent variables. As variational networks could easily collapse into an unconditional generative model, i.c., in the extreme case, all the input will be mapped into the same latent code (Ma et al. 2019b). We use the dimension index whose corresponding latent value is the maximum as the cluster assignment ID. We then derive the cluster assignment results A E R NxX where N is the number of test samples, K is the cluster number. The diversity is calculated based on entropy (X)= = - EveR p(k) log (og()) where p(k) is the fraction of the number of samples falling into the k-th cluster among all the samples. A larger value means a better diversity. We record the largest diversity between user and item latent variables. The results are shown in Table
We observe that the re- moval of Q(x1). leads to the largest performance degra- dation.
BeerAdvocate
Digital Music
Office Products
Variants
RMSB(L)
Div(t)
RMSB(L)
Div(1)
RMSE(L)
Div(1)
After removing Reentroid- the latent variable
Full Model
75.36
1.89
92.87
3.46
84.32
1.58
w/o Reentroid
75.96
0.04
92.62
0.07
84,58
0.02
w/o KL. term
75.64
1.43
93.83
2.86
84,23
1.21
diversity shrinks to near zero,
-w/o (IT..)
76.02
1.04
93.95
3.10
84,94
0.93
indicating nearly all the la-
tent variables fall into the same cluster. This high- lights the capability of our prior-centratisation term in enabling the latent variables 2 effectively reflect the soft cluster assignments. We also found that replacing the MSE in Q(x,,) with the cosine similarity (1 - cosine) reduces the latent variable di- versity significantly. This can be partly explained by the fact that the cosine similarity focuses on measuring the angle between the input vi and the reconstructed output T4, ignoring the magnitude of the vectors which is however important in our case.
((x.. with Cosine
75.35
0.11
92.85
1.76
84.97
0.07
Table 2: RMSE(%) and Diversity among model variants.
b). Effect of our proposed information bottleneck regularisation We compare our approach with StandPrior, Wasserstein VAE and IndivPrior: StandPrior (aka. StandVAE) uses two VAEs for users and items respectively, each with the standard Gaussian distribution (O, 1) as the prior. Wasserstein VAE differs from the StandPrior in using the Wasserstein metric (Tolstikhin et al. 2017) to calculate the distribution discrepancy between posterior and prior p. IndiyPrior assigns a sepa- rate Gaussian prior each user or item with its mean value calculated from the user's or item's 's corre-
7

---

Under review as a conference paper at ICLR 2023
sponding representation derived from the user-item interaction graph, i.e,, p(2(1)) N (W,uu, I), p(z()) (WW,ii I). where ξu E R2 Çi Rd denote the user and item node representations in GCN learning, Wy and W; are learnable parameters in a linear layer of size (d x d), and is the dimension of graph node features.
Figure 2: RMSE(%) with different information bottleneck regularisa- tion. Our method achieves the best, followed by StandPrior and Wasser- stein VAE. IndivPrior is the worst, even worse than AutoEncoder.
77.0
86
76.5
94
75.5
92-
84
75.0
74.3
BeerAdvocate
Digital Music
83
Office Products
The rating prediction results are shown in Figure 2 Wasserstein VAE and StandPrior demonstrate overall better results than IndivPrior and all the baselines in Table [ The improvement can be explained by the nature of VAE, which is particularly beneficial when dealing with sparse data where few observations are available. However, IndivPrior, which imposes a separate Gaussian prior on individual user/item, shows worse performance compared to the other three variational frameworks, and is even worse than AutoEncoder. Our model benefits from a rich prior separated from different user/item clusters, while avoiding using a global normal prior (StandPrior and Wasserstein VAE) or separate instance-level prior for each user/item (IndivPrior).
LATENT VARIABLE INTERPRETABILITY EVALUATION
6.1 CLUSTER SEPARABILITY AND COHERENCE
As each dimension of the latent variables 2f, corresponds to a cluster, for the k-th dimension, we can search for its relevant reviews which have the highest value in 2h, and list the most frequent words in the review set as the representative topic words 11 Representative words in randomly selected three clusters on BeerAdvocate Dataset are shown in Table 3 For our model, the most prominent words different across different clusters (thus being colored), i.e., 'brown roasted", 'pine, caramel' and "good flavour'. We find it hard to see clear topic separations from the Wasserstein VAE results. For example, Cluster and 3 share the same 3 words, 'sweet', "light" and 'malt' out of their top 4 words. addition, 'carbonation" foam. The results show that with the incorporation of the priors derived from user/item clusters, our proposed approach is able to learn latent variables in the review semantic space which can produce better separable topic clusters.
Cluster and 'whitehead" in Cluster 3 are both relating to beer
GIANT
WasserteinVAA
Table 3: The most prominent words (sorted by occurrence frequency) in three randomly selected clusters from GIANT and Wassertein VAE. We highlight the top 3 words that are not found in the other two clusters.
pour, color, brown, foel, roasted, malis, almost, moderate, coflee, dark pine. caramel, lacing, citrus, mouth- feel, hint, ipa, note, hody, strong good, flavour, dark, better, brew, style, bad, Canadian, nose, drinking
light, color, malt, glass, poured, drink, pour, mouthfeel, alcohol sweet, coffee, light, malt, aroma, bonation, pour, mouthfeel, hint, color aroma, light, sweet, malt, white head, mouthfeel, nose, hint, brew,
6.2 COMPREHENSIVENESS EVALUATION BY PERTURBING ON LATENT VARIABLES
To explore the importance of our identified clusters, i.e., the latent dimension with larger value, calculate the performance change before and after removing the specific dimension and define
ΣN (r(=))-(eff/+(t))
Comprehensiveness as:
Where r(.) is the predicted rating, 2f is the latent [| is the identified top k latent dimensions. To remove
N
variable for i-th evaluated user-item pair, 2; the effects from these dimensions, we replace the values in the top i dimensions with the average value of the latent variables according to Fong Vedaldi (2017). In the Figure B line chart, our model demonstrates a larger performance change as more important dimensions are removed and
We exclude the stopwords and most frequent words appeared in all clusters, such as 'beer" which occurs in every cluster in the BeerAdvocate dataset. The results the other datasets, well as comparison with StandPrior shown in Appendix D33 The semantically coherence within cluster are Appendix D.4
8

---

Under review as a conference paper at ICLR 2023
its changes more obvious than the two baselines. We further randomly delete l latent dimen- sions and calculate the relative performance change by subtracting the changes caused by random removal (Table below Figure B). The relative changes in model most predominant, followed by Wasserstein VAE, while relative changes of StandPrior are negative, showing that the random removal of latent dimensions can bring even larger changes to the predominant ones.
Ouro
Basetres
Basciins
Ours
Prodect
Boselhoe
4.2040
Curs
Ourt
93540
4-5-12
100-01
0.6
1-5-37
0.0000
0:0-00
005-00
10
70
Intamuueed
Dimars
Latert
Allumooeed
Brmemitons
Model
2.34
Model
3.18
Stand
WAE
0.96 0.03
0.03
0.05
0.87
0.12
0.59
Stand
WAE
0.49
0.12
0.21
0.37
0.23
0.35
Stand
WAE
0.71 0.31003110
0.54
0.98
0.25
0.23
Wavs:
0.12
1.26
1.68
1.45.00
Wastt
1.63
2.04
10670.67
Wassi
10.71
0.88
Figure 3: Top:
The Comprehensiveness values by removing the top most important latent di- mensions identified, k E (3, 5, 10, 15, 20, 30). Bottom Table: Relative performance changes after subtracting the changes caused by randomly removing & latent dimensions.
6.3 CASE STUDY OF GENERATED INTERPRETATIONS
Interpretations for example user-item pair in Office Products generated by extracting tences most relevant to user/item latent topics from reviews like-minded users and past reviews about the item While existing explainable recommenders are unable to extract information be- yond the current user-item review, GIANT can extract sentences from reviews of like-minded users to explain the current user-item interaction based on the user/item clustering results. Our human evaluation results presented in Table A7 further show that explanations generated by GIANT are better than those from Wasserstein VAE in terms of relevance, faithfulness and informativeness.
Figure Extracted interpreta- tions from GIANT (top) and Wasserstein VAE (bottom). We first select reviews which are relevant the user/item as- signed cluster, and then ex- tract the most relevant sen- tences based their Euclidean distances from the cluster topic representations. Interpretations extracted by GIANT appear to be more faithful with the recommender's decision. while those extracted by Wasserstein- VAE do not support the recom- mendation decision.
The interpretations between 46 U-I pair
Based the reviews from like-miee
(a) great plastic cover thick cardboard with quality paper
part
the double side losse paper holder, it makes things so much easier for you. (b) you something for permanent writing, more expensive journal would be better choice.
on reviews from
(o). took my favorite writing instruments and they all wrote nicely, without bleeding. (b). write smoothly with my
Will recommend
Item
The interpretations between 46 U-I pair
Based the reviews from like-mined
(a). Cheaper binders ofhen inadequate loops just
(b). I would prefer the pocket were in the back so it didn't have be move each time.
Based on the reviews from I:
(a) don't like that becouse can t doodle write It. paper cheap translucent writing on other side doesn't show through.
(b) college
Will recommend iham
7 CONCLUSION
In this paper, we leverage the user/item clusters sharing common interests/characttrriiiss obtained from the user-item interaction graph to refine the review text latent factors via our proposed ge- ometric information bottleneck (GIANT). We empirically show that GIANT better in learning a semantically coherent and interpretable latent space and the generated explanations are more faithful to the model decisions, while achieving comparable rating prediction accuracy on three commonly used datasets.
Detaiis of explanation generation be found in Appendix

---

Under review as a conference paper at ICLR 2023
REFERENCES
Alexander A. Alemi, Ian Fischer, Joshua V. Dillon, and Kevin Murphy, Deep variational information bottleneck. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017. httpttss:
//openzeview.
David M. Blei, Andrew Y. Ng, and Michael I. Jordan. Latent dirichlet allocation. J. Mach. Learn. Res., 3:993-1022, 2003, URL htt.htttp/// /ttt:://iml.:. wttpp/pp...a..o.oo/o/o/. wteaaaprr/////eeee///ee w/3////eeeeaaaaa.m.
Chong Chen, Min Zhang, Yiqun Liu, and Shaoping Ma. Neural attentional rating regression with review-level explanations. In Antroie Champin, Fabien Gandon, Mounia Lalmas, and Panagiotis G. Ipeirotis (eds.), Proceedings of the 2018 World Wide Web Conference on World Wide Web, WWW 2018, Lyon, France, April 23-27, 2018, pp. 1583-1592, ACM, 2018. doi: doi. org. 10 111.1.11111111111111 31888757777777 3186070
10.1145/317887..1 188777 URL
Tong Chen, Hongzhi Yin, Guanhua Ye, Zi Huang. Yang Wang, and Meng Wang. Try this instead: Personalized and interpretable substitute recommendation. In Jimmy Huang. Yi Chang, Xueqi Cheng, Jaap Kamps, Vanessa Murdock, Ji-Rong Wen, and Yiqun Liu (eds.), Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, SIGIR 2020, Virtual Event, China, July 25-30, 2020, pp. 891-900-0000000000 ACM, 2020. doi: 10.1145/ 3397271.340104 URL https: doi org/ 10. 1145/ 335//33977222 3401042
Xu Chen, Zheng Qin, Yongfeng Zhang, and Tao Xu. Learning to rank features for recommendation over multiple categories. In Raffaele Perego, Fabrizio Sebastiani, Javed A. Aslam, Ian Ruthven, and Justin Zobel (eds.), Proceedings the 39th International ACM SIGIR conference on Research and Development in Information Retrieval, SIGIR 2016, Pisa, Italy, July 17-21, 2016, pp. 305- 314. ACM, 2016. doi: 10.1145/2911451 2911549.
URL https: /eoi org
291 1451 2911 1549
Zhongxia Chen, Xiting Wang, Xing Xie, Tong Wu, Guoqing Bu, Yining Wang, and Enhong Chen. Co-attentive multi-task learning for explainable recommendation. In Sarit Kraus (ed.), Proceed- ings of the Twinty-Fighth International Joint Conference Artificial Intelligence, IJCAI 2019, Macao, China, August 10-16, 2019, pp. 2137-2143, ijeai. org, 2019. doi: 10.24963/jjcai.200 296.
/ttp:///o..../.oo or.g 10. 24 9 i jeai 201 9/296
Steffen Czolbe, Oswin Krause, Ingemar J. Cox, and Christian Igel. A loss func- tion for generative neural networks based on watson's perceptual model. Larochelle, Marc' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (eds.), Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020. URL htttps://proceeeiiiiii
Hugo
neurips cc//papp//222//hahh// html
Persi Diaconis and David Freedman. A dozen de Finetti-style results in search of theory. Annales de I'LH, Probabilités et statistiques, (4S2)):377422, 1987. URL http:// //tt::///ww.../// www. numdam.
1987 23 S2 397 0./
Hao Fei, Yafeng Ren, Shengqiong Wu, Bobo Li, and Donghong Ji. Latent target-opinion prior for document-level sentiment classification: A variational approach from fine- aie-ggrined perspective. In Jure Leskovec, Marko Grobelnik, Mare Najork, Jie Tang, and Leila Zia (eds.), WWW '21: The Web Conference 2021, Virtual Event / Ljubljana, Slovenia, April 19-23, 2021, pp. 553-564. URLaattss://eet /ww.///ciiiia
ACM / IW3C2, 2021. doi: 10.1145/3442381.344999 3442381. 3449789
1145/
Ruth Fong and Andrea Vedaldi. Interpretable explanations of black boxes by meaningful perturba- 03296
tion. CORR, abs/1 704.03296, 2017. htt.:htttp://arriw.orr/////
Hao Fu, Chunyuan Li, Xiaodong Liu, Jianfeng Gao, Asli Celikyilmaz, and Lawrence Carin. Cycli- cal annealing schedule: A simple approach to mitigating KL vanishing. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), Proceedings of the 2019 Conference of the North Ameri- can Chapter of the Association for Computational Linguistics: Human Language Technologies, NAT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume I (Long and Short Papers),
10

---

Under review as a conference paper at ICLR 2023
pp. 2
Association for Computational Linguistics, 2019. doi: 1t.118633/1//191 1021. URL
/ /B5. 1021
Azin Ghazimatin, Oana Balalau, Rishiraj Saha Roy, and Gerhard Weikum. Prince: Provider-side
interpretability with counterfactual explanations in recommender systems. In Proceedings of the 13th International Conference on Web Search and Data Mining, pp. 196-204, 2020.
Peter Hall, Rodney C. L. Wolff, and Qiwei Yao. Methods for estimating a conditional distribu- tion function. Journal of the American Statistical Association, 94(445):154-16 1635 1999. ISSN 01621459. URL BRLhttp:: / www. jstor. wttp:o/tttaleeee66655...33
Ruining He and Julian J. McAuley. Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. In Jacqueline Bourdeau, Jim Hendler, Roger Nkambou, Ian Horrocks, and Ben Y. Zhao (eds.), Proceedings of the 25th International Conference on World Wide Web, WWW 2016, Montreal, Canada, April 11 - 15, 2016, pp. 507-517. ACM, 2016. doi: Stthtttp:: doi orgg 10. www.1111122222220 000722288300 2883037
10.1145/2872427 2883337
Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yong-Dong Zhang, and Meng Wang. Lightgon: Simplifying and powering graph convolution network for recommendation. In Jimmy X. Huang, Yi Chang. Xueqi Cheng, Jaap Kamps, Vanessa Murdock, Ji Ji-Rong Wen, and Yiqun Liu (eds.), Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, SIGIR 2020. Virtual Event, China, July 25-30, 2020, pp. 639 648. ACM, URL https: org. 10. 115/339777
2020. doi: 10.1145/3997271.340000
34C1063
Diederik P Kingma and Max Welling. Auto-encoding variational bayes, 2013. URL
https:
/ www.xi.wrr/aaa////
6114
Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. In Yoshua Bengio and Yann LeCun (eds.), 2nd International Conference on Learning Representations, ICLR 2014, Banff. AB, Canada, April 14-16 16, 2014, Conference Track Proceedings, 2014. http:/tt:///att 1es/1312.6111.
Lei Li, Yongfeng Zhang, and Li Chen. Generate neural template explanations for recommenda- tion. In Proceedings of the 29th ACM International Conference on Information amp; Knowl- edge Management, CIKM '20, pp. 755-764, New York. NY. USA, 2020. Association for Com- puting Machinery. ISBN 9781450368599.
doi: 10.1145/3340531.44111
URL https:
//doi.org/10. 10. 1145/ 3340531 3411992
Lei Li, Yongfeng Zhang, and Li Chen. Personalized prompt learning for explainable recommenda- tion. CoRR, abs/2202.07371, 2022. WRLLhttps://arxx http://///er.v..oor///////02// 07371
Piji Li, Zihao Wang, Zhaochun Ren, Lidong Bing, and Wai Lam. Neural rating regression with abstractive tips generation for recommendation. In Noriko Kando, Tetsuya Sakai, Hideo Joho, Hang Li, Arjen P. de Vries, and Ryen W. White (eds.), Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, Shinjuku, Tokyo, doi: 02.14453307713333088833 URL
Japan, August 7-11, 20/7, pp. 345-3335555 354, ACM, 2017.
https: / //o.///o///// o / 00. 1 45 30 76.007773663333 3080882
Jianxin Ma, Peng Cui, Kun Kuang, Xin Wang, and Wenwu Zhu. Disentangled graph convolutional networks. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th Inter- national Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, volume 97 of Proceedings of Machine Learning Research, pp. 4212-4221. PMLR, 2019a. URL htt.httpp/pproceeeiiii
m..r wallr.pressse wwwpprrsssvvv... v ma 9a html
Xuezhe Ma, Chunting Zhou, and Eduard H. Hovy. MAR: mutual posterior-divergence regular- ization for variational autoencoders. In 7th International Conference on Learning Represen- tations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019b. URL atttps://openreviine net. fanett/orrune uorum? 1e= 12ACKQ
Julian McAuley and Jure Leskovec. Hidden factors and hidden topics: Understanding rating di- mensions with review text. In Proceedings of the 7th ACM Conference on Recommender Sys- fems, RecSys 13, pp. 165-172, New York, NY, USA, 2013. Association for Computing Machin- ery. ISBN 9781450324090
doi: 10. 10.1145/2501157.255 1145/2507157 575011171725777
URL ht http::/dooooogoe orrg/10.
11

---

Under review as a conference paper at ICLR 2023
Julian J. McAuley, Jure Leskovec, and Dan Jurafsky. Learning attitudes and attributes from multi- aspect reviews. In Mohammed Javeed Zaki, Arno Siebes. Jeffrey Xu Yu, Bart Goethals, Geof- frey I. Webb, and Xindong Wu (eds.). 12th IEEE International Conference on Data Mining, ICDM 2012, Brussels, Belgium, December 10-13, 2012, pp. 1020-00222200000 IEEE Computer Society, 2012. doi: 10.1109/1CDM.2012 10..
http://0.11199//111 DWW.://IID...000
110
Jianmo Ni. Jiacheng Li, and Julian J. McAuley. Justifying recommendations using ceatantty-labbllee reviews and fine- are-graaned aspects. In Kentaro Inui, Jing Jiang, Vincent Ng. and Xiaojun Wan (eds.), Empirical Methods in Natural Language Processing
Proceedings of the 2019 Conference and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong. China, November 3-7, 2019, pp. 188-197. Association for Computational Linguistics, 2019. doi: 10.18655//1/D19-10111 1018, URL httpt::/tppooooooe http:///./.o....11.11111111..1 D19 018
Juntao Tan, Shuyuan Xu. Yingqiang Ge, Yunqi Li, Xu Chen, and Yongfeng Zhang. Counter- factual Explainable Recommendation, pp. 1784-1793. Association for Computing Machinery, New York, NY, USA, 2021a. ISBN 9781450384469, URL httph::t////d org/10.1145/ 3459637.3482422
Juntao Tan, Shuyuan Xu, Yingqiang Ge, Yunqi Li, Xu Chen, and Yongfeng Zhang. Counter- factual explainable recommendation. In Gianluca Demartini, Guido Zuccon, J. Shane Culpep- per, Zi Huang, and Hanghang Tong (eds.), CIKM '21: The 30th ACM International Con- ference on Information and Knowledge Management, Virtual Event, Queensland, Australia, November 1 5, 2021, pp. 1784-1793, ACM, 2021b. doi: 10.11453459633.34422000 URL 0ttp:////0..000010111111/55/6 /t/.///333355333344220000 3482420
Naftali Tishby and Noga Zaslavsky Deep learning and the information bottleneck principle, 2015.
URL httpttttssssaii...aapa wttp://wwrrabbaab..b..... 02406
Ilya O. Tolstikhin, Olivier Bousquet, Sylvain Gelly, and Bernhard Schölkopf. Wasserstein auto-
encoders. CoRR, abs/171 2017.
Jakub M. Tomezak and Max Welling. VAE with a vampprior. CORR, abs/1 2017. URL
httt:://arxlv.org/abss.
Quoc-Tuan Truong, Aghiles Salah, and Hady W. Lauw. Bilateral variational autoencoder for col- laborative filtering. In Liane Lewin-Eytan, David Carmel, Elad Yom-Tov, Eugene Agichtein, and Evgeniy Gabrilovich (eds.), WSDM '21, The Fourteenth ACM International Conference on Web Search and Data Mining, Virtual Event, Israel, March 8 12, 2021, pp. 192-990-000 ACM, 2021. doi: 10. 10.1145/3433963.3444777
URL http:::/t///t...t. http://///oo..oo///.o..oo 1 410.11113333333444447777
Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(11), 2008
Chong Wang and David M. Blei. Collaborative topic modeling for recommending scientific arti- cles. In Chid Apté. Joydeep Ghosh, and Padhraic Smyth (eds.), Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Diego CA, USA, August 21-24, 2011, pp. 448 448-456. ACM, 2011. doi: 11145/2222008.2222444999 URL 45/ /005/000220...// 2020480
https:://ois .rgg
Nan Wang, Hongning Wang, Yiling Jia, and Yue Yin. Explainable recommendation via multi-task learning in opinionated text data. In Kevyn Collins-Thompson, Qiaozhu Mei, Brian D. Davison, Yiqun Liu, and Emine Yilmaz (eds.). The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, SIGIR 2018, Ann Arbor, MI, USA, July 08- 12, 2018, pp. 1665-174 ACM, 2018. doi: 10. 00.1145/3209978..220001 URL https: //doi http///i.ogg/..... 1145/32099 3210010
Xiang Wang, Hongye Jin, An Zhang, Xiangnan He, Tong Xu, and Tat-Seng Chua. Disentangled graph collaborative filtering. In Jimmy Huang, Yi Chang, Xueqi Cheng, Jaap Kamps, Vanessa Murdock, Ji-Rong Wen, and Yiqun Liu (eds.), Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, SIGIR 2020, Virtual Event, China, July 25-30, 2020, pp. 1001-1010. ACM, 2020. doi: 10.1 10.1145/3397271.34 3,001137. URL
https: 0ttg//...145/39771111 .101711333417777 3401137
12

---

Under review as a conference paper at ICLR 2023
Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process., 13(4):600- 612, 2004. doi: 10.1 10.1109/TIPP2200.888988 1198611
111.9/0011111/111111 2003 8 819861
Yongfeng Zhang and Xu Chen. Explainable recommendation: A survey and new perspectives. Found. Trends Inf. Retr. 14(1):1-101, 2020. doi: 10.1561/1500000066. URL https::
Yongfeng Zhang, Guokun Lai, Min Zhang, Yi Zhang, Yiqun Liu, and Shaoping Ma. Explicit factor models for explainable recommendation based on phrase-level sentiment analysis. In Shlomo Geva, Andrew Trotman, Peter Bruza, Charles L. A. Clarke, and Kalervo Järvelin (eds.), The 37th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '14. Gold Coast QLD, Australia July 06 11, 2014, pp. 83 92. ACM, 2014. doi: 2609579
Yongfeng Zhang. Xu Chen, et al. Explainable recommendation: A survey and new perspectives. Foundations and Trends@ in Information Retrieval, 14(1):1-101) 2020.
Junbo Jake Zhao, Yoon Kim, Kelly Zhang, Alexander M. Rush, and Yann LeCun. Adversarially regularized autoencoders for generating discrete structures. CoRR, Abs/1706.00223, 2017. URL.
http:///axxw..rrg// orgg 7066 04223
Lei Zheng, Vahid Noroozi, and Philip S. Yu. Joint deep modeling of users and items using reviews for recommendation. In Maarten de Rijke, Milad Shokouhi, Andrew Tomkins, and Min Zhang (eds.), Proceedings the Tenth ACM International Conference Web Search and Data Mining, WSDM 2017, Cambridge, United Kingdom, February 6- 10, 2017 pp. 425 434, ACM, 2017. doi: 10.1145/30 8661.300 1,0011111000 8665. CRL.ttps: htte::///oo...rrrr00011145//// 8661 3018665
13

---

Under review as a conference paper at ICLR 2023
A EXPERIMENT SETUP
A.1 DATASET DETAILS AND DATA PROCESSING
We choose three commonly used e-commerce datasets which not only contain the interaction inter- actions between user and item but also the review texts. Amazon review data 13 dataset collection, consisting of 24 kinds of products. As our method derive the user/item mainly based on their reviews, we use the dense subset, 5-core, of the amazon review dataset that extract the user-item pairs, such that each of the remaining users and items have at least 5 reviews each. Considering the training efficiency and topics should be familiar to general readers, we select the sparser
is one of the popular
two datasets with around 50k reviews, Digital Music and Office Products. Besides, dataset BeerAdvocate dataset, i..., the lower rating of observed ratings between total interactions. Table AT summarizes the statistics of the three datasets. We further filter out users items with more than 100 reviews/nncraattoos and truncate the reviews to length of 400,00000000 for the three datasets, respectively. We randomly select 80%, 10%, 10% the vain//aliiatiion/ess sets.
Datasets
# Interactions
# Users
Items
# Reviews Per User
# Reviews Per Item
AsgRating
Sparsity
ReerAdvocate
35450
6939
13122
56
31
3.74
0.00044
Digital Music
51768
5541
3568
20
40
4.22
0.00294
Office Products
35999
4902
2364
86
4.33
0.00349
Table A1:
Datasets statistics. All the datasets have ratings in the range of 1-5. The Aevoaav dataset has the smallest sparsity.
A.2 SAPER-PARRMETER SETTINGS
We use Adam for optimization with the initial learning rate set to 0.001. The batch size is set to 32, 64, and 32, and training epochs are 10, 10, and 15 for the three datasets, respectively. The adja- cency matrix in graph is binary: user-item entries with the interacted ratings higher than the average corpus-wide rating is set to I. and 0 otherwise. We stop training GCN when it reaches the best recall value in the validation set. The cluster centroid vectors are derived by K-means. The graph node feature dimension d is 64, and the number of clusters K are searched in [16, 32, 64, 128, 256, 512]. In CNN review encoding, the word embedding and ID embedding are both initialized by the uni- form distribution (-0.1,0.1) 0.1), their dimensions are set to 300 and 64, respectively. We use two CNN kernels with size and 3, and the dimension of 32 to encode the reviews. As such, the CNN- encoder outputs the review representations with dimension of 64 (i.e., 32x2). To combine multiple reviews for each rived through two consecutive linear transformation layers of 64 - 32 + 1. In GIANT, the latent variable dimension is set to the same the number of clusters K. The softmax temperature is searched in [1, 2, 4, 6, 8). The non-linear activation function in both encoder and decoder is ReL.U. We use linear layer with [K, 64] weights to map the the latent variable 2/x the same space of the ID features. For fets used in rating prediction, we first apply the ReLU function on the result of element-wise multiplication of the user and the item features, then apply linear layer with the shape of (64, 1] to transform the ReLU output to a rating scalar.
item, we use the attention mechanism where the attention weights de-
B TRANSFERABLE INFORMATION BOTTLENECK
Theorem Suppose we use independent well-trained encoder-decoder based architectures to model the features for different modalities (e.g., graph and text), then given input I,, has representations
r X and xQ X under different modalities with the constraint T(X:: X²) )) > I, since X
and X which are from the same input, should be relevant. Also, assuming IZZ X*) < Ic and (Z";x9) & Ic, then the following property holds:
T(X";29 & H(X*) H(X*) - H(X*XX)) + To
Proof: We first apply the chain rule twice in different orders for three sources X X and Z below:
14

---

Under review as a conference paper at ICLR 2023
- + HZ*),
H(X' X*, 29 z + (
Since the right parts the above two equations are the same, we have:
H x", 2 + H(X* (2())= - -X
(9) + H(2))
which can be simplified as: H
H 2Z) H(X ,
Since 2", X and X are derived from the same set of samples but under different modalities or representations, they are obviously not independent. Hence, we have (X | 2 (X* (as X and Z are dependent) and H (X* 2 2 0 (as X and X are dependent). Thus, we can replace H (X* Z9) with (X* and replace [X Z with O, derive the following inequality:
H(X Z (X - H(
By applying the properties conditional differential entropy, which yields (X9 29) H(x)) [Xx,, 29), the above formula can be simplified as:
H(X4129) 2 H(X9)- -
Accordingly, we have:
19) H(X*) H(X1129) H((X*) H(X*) + I(XX; + (XX, H(
$ - - H(X
(12)
Equation 12 shows that the lowest upper bound of the transferable information bottleneck is
[(X;; 2 S Le when the distribution of two modalities are the same, X = =,,, and the conditional
entropy of H (X* is O. Let's take H H[X)) (X') + (XX))x)), then the Equation 22
can be simplified as ((x;; 2t) S H + Ic, where I, is the upper boundary of information bottleneck for both modalities.
Recall the Lagrange multiplier based objective function defined in Eq.
OIB = I[8=1(X;2) 8- T(X;
which aims to optimise:
max (X;2) s.t. I(x:2) $ Ic.
Now, let us focus on the optimisation on the encoder-decoder framework in text by denoting the variables with the superseript. use the latent distribution learned from the graph modality, Z9, to impose constraints on Zt, the objective can be rewritten as:
max n Z²) s.t. 1(X4 H
Since H - + XX* is irrelevant with our optimisation, the Lagrange mul- tiplier based objective function can be the rewritten in the form of Eq. 3
= - 1(x;; 29),
That is, there is need to define the prior for the text modality. Instead, use the posterior distribution of the latent variable from the graph modality as the regularisation to guide the training on the text data.
15

---

Under review as a conference paper at ICLR 2023
C
OPTIMISING OBJECTIVE OF TRANSFERABLE INFORMATION BOTTLENECK
Information Bottleneck: Recall that the IB objective in this work has the form 1(Z'; 8 ; Z where
p(=1(z') d2'
I(Z':X)) = - p(at,z") log
p(x²)
where p(.) is the true distribution which is not observed. According to the proof given by (Alemi et al., 2017), we consider the posterior estimation ((xx(2)) in the decoder. According to Gibbs' inequality, we have:
/ pla", log
p()* g(x')
x(()xx)= 2
dā' dz
f pl ,2) logg(oo dd) de H(X).
Since the entropy of decoding results H(X)) independent of model optimisation, we only need to consider the posterior estimation based lower bound. According to(Alemi al. (017), we have the following empirical approximation by reparameterisation trick:
1 N n== (eep
llogo (2) e)]
For the Lagrange multiplied term I(X*, Z9) which is different from the standard information bottleneck based 3-VAE we do consider the information bottleneck of [(X", Z²) first, because [(X", Z9) S H + I Z") according the proof given in section B Here, we have the following
bound by applying Gibbs' inequality:
f plat, 2² ""ggp'"(<'"ddl jd''ddd dz f p(e')logp((''d'"
(1)
Since estimating the prior distribution of Z, might be difficult, based on the definition of transferable information bottleneck, we apply the Gibbs' inequality and have:
p(21(x)) r(29)
((x:22) S p(= Moo
de' + H,
where r(29) is the posterior distribution from the graph-based latent representation, and decided by the prior of two modalities which is independent of model optimisation. By combining the two results, the above bound can be approximated by the reparameterisation trick (Kingma Welling 2013) with a Gaussian random variable e:
OIB = D. Berpeel lf e)J/N + B- KL(p(Z" (2,, e))r(Z9))
Recons. term: Q(x)))
KL-div. term: KL(ZZ (26)
Further discussion about the reconstruction term Q(xh)
In section 4.2 we propose to use the Nadaraya- Watson estimator(Hall al.) 1999) to approximate the conditional probability If c)]. The idea is to insert the decoding result ø", to the space of training samples {a }. and take the kernel density estimation method approximate Q(x() by Bayesian rule:
Q(x%)) = log (( 2h) = log
K(
((rt))
N Σラ
Since the input feature for 24, is fixed, we only care about the updating of reconstruction through encoder-decoder architecture. Thus have:
16

---

Under review as a conference paper at ICLR 2023
1
N
2
x1
Q(
α
10
N
h
==1
To simplify the above estimation, we apply the RBF-kernel and triangle inequality estimation and have:
1 In
N
- ).
Fy xz x(
Q(2%)) o log
19
N
h
h
/=1
N
1
= log N
Σ
exp( -1111 - =z513)
- -x3113)
N
= log N
exp( 11--|11 || - 2/51 | at,113)
j=1
N
< log
exp( - 1|, - -x 112)
1=1
log[exp(-1112)) 13)]
Thus, we are able to optimise the Qxx() by mean square enror if we choose the natural logarithm function:
-(xhh) 2 -In(exp(- 110",, - =11) = || 2 11
Therefore, we can rewrite the lower boundary well as the learning objective Orn by:
1 D R
OIB N Bespte 1 KT(pp(" (r?)))
D ADDITIONAL EXPERIMENTAL RESULTS
D.1 EVALUATION RESULTS ON RANKING-BASE BASED METRICS
In addition to the accuracy-basee based metrics, i.e., MSE and MAE, also evaluate models using the ranking-basee based metrics, i.e., Precision and Recall (See TA2). We observe that GIANT achieves the best precision results in general across all the datasets, with more prominent improvement compared to baselines on Office Products. In terms of recall, GIANT outperforms the others on Digital Music and Office Products, but it is inferior be the second-best model. The results show that GIANT is more effective on datasets covering products with distinct features, but its advantage over the baselines is less obvious on datasets with similar feature descriptions, such as BeerAdvocate.
BeerAdvocate. Waserstein VAE appears to
#o recommended items @k that are relevant
Precision
#recommended items
Recall of recommended items @k that are relevant
#relevant items
D.2 THE IMPACT OF CLUSTER NUMBER
We show the the impact of different number of clusters K, i.e., the dimensionality of latent variable Zns on review rating prediction results in Table The best rating prediction results obtained for Digital Music and Office Products when the number of clusters is set to 128. But on BeerAdvocate, the optimal cluster number is 256.
17

---

Under review as a conference paper at ICLR 2023
Models
NARRE
AutoEncoder
Wassastein VAE
StandPrior
GIANT
Precision@11
0.24
0.16
0.20
0.20
0.24
BeerAdvocate
1
0.20
0.15
0.17
0.16
0.15
Precision@1
0.20
0.16
0.22
0.23
0.23
Digital Music
Recall
0.21
0.17
0.24
0.23
0.23
Precision
0.24
0.20
0.29
0.28
0.31
Office Products
1
0.22
0.23
0.19
0.23
Table A2:
The rank-based evaluation results, i.e., Precision and Recall for all the baseline and our proposed GIANT.
Dataset
16
32
64
128
256
512
BeerAdvocate
75.65
75.56
75.53
75.46
75.36
75.74
Digital Music
93.51
93.24
92.93
92.87
93.33
93.10
Office Products
84.78
84.53
84.42
84.32
84.35
84.48
Table A3: GINAT rating prediction performance in Mean Square Error (MSE %) with different number of clusters.
D.3 CLUSTER SEPARABILITY
As each dimension of the latent variables corresponds to a cluster, for the k-th dimension, we can search for its relevant reviews which have the highest value in 2І and list the most frequent words in the review set as the representative topic words. We identify 'ABBA' and their popular songs, computer peripherals such 'network cable' and 'router" in Digital Music and Office Products datasets, respectively.
GIANT
Wasserstein VAE
dre, prince. used known, coast, mirrors, hate, westside story. jadekiss, always
quot. album, Ask Rufus, song. funk, voice, music, band, soul, love
Digital Music
bangles, vicki. vocal, place, live, hero, takes, full, liverpool, beatles
song, everglow, album, band, beautiful, track, lyrics, peasion. record, Christian,
mia, abba, greatest hit, money money, gimme gimme, super trouper, knowing knowing, gold
dancing queen, take chance,
record, alhum, song, band, track, time, sound, head heart, folk, live, love
set, computer, wireless, network, cable, router, edit, pencil,
scan, software, pencil, Mae OS, work, printer, small, printed, color,
Office Products
photo, desk, laser. scanner, work, scan, epson. scaning, enough, easy
easy use. dry erase, folder, mouse pad, office, ink car- tridge, works, used, post recommend, printer, folder, paper, easy use, boxes, feature, stapler, pencil, canon, color
pencil, tape. ink, pen, anyway, pen, rubber. clip. pretty.
Table A4: The most prominent words (sorted by occurrence frequency) in three randomly selected clusters from GIANT and Wasserstein VAE on Digital Musie and Office Products datasets. For each dataset, we highlight the top 3 words that are not found in the other two clusters. GIANT generates better separable topics while Wasserstein VAE fail to generate clear topie pattern.
We also compare with the generated clusters using StandPrior. The results are shown in Table AS is difficult to see a clear topic pattern in StandPrior as the top words largely overlap in different clusters. The first several words all 'hop', "malt" and 'good'. The results Wasserstein VAE, which can at least generate diverse clusters, reflected in the distinguished words in each clusters.
worse than
18

---

Under review as a conference paper at ICLR 2023
hop, good, malt, aroma, hint, pour, great, well, mouthfeel, much
BeerAdvocate
hop, aroma, good, sweet, light, malt, finish, brew, poured, glass
hop, malt, light, sweet, much, carbonation, good, well, mouthfeel, pour
great, best, band, love, hit, well, first, make, lyric, come
Digital Music
love, great, best, band, sound, well, first, even, make, lyric great, love, best, well, band, lyric, make, fan, first
use, work, well, binder, great, will, easy, nice, label, ink label, easy, tape, color, great, using, work, make, well, really use, label. product, tape, color, binder, quality, really, scanner, great
Office Products
The most prominent words (sorted by occurrence frequency) in three randomly selected clusters from StandPrior on BeerAdvocte, Digital Music and Office Products datasets. For each dataset, we highlight the top words that are not found in the other two clusters. StandPrior produces topics which contain largely overlapped words.
Table A5:
Table A6: Average cosine similarity of review pairs within each cluster as the coherence mea- sure. A larger similarity value means bet- ter coherence. GIANT generates significantly more coherent clusters than graph clusters.
BeerAdvocate
Digital Music
Office Products
Models
User
Item
User
Item
User
Item
Graph
0.092
0.085
0.155
0.214
0,245
0.200
GIANT
0.490
0.481
0.457
0.471
0,492
0.407
D.4 CLUSTER COHERENCE
We explore the difference between our generated clusters 2nk and the clusters derived in the graph. To verify the capability of creating semantically coherent clusters, we propose to measure the cluster coherence as the average cosine similarity between every review document pair within cluster. We first obtain the document-level review representations by feeding the reviews from the test set to our pre-trained CNN encoder. We then calculate the average cosine similarity between the representa- tions of each review pair in a cluster, and are further averaged across all clusters (in Table A6).
E HUMAN EVALUATION FOR INTERPRETABLITY
We conduct human evaluation to validate the interpretablity of our proposed method. To make it easier for humans understand the rationales behind model decisions, we extract the most relevant sentences from user/item reviews as interpretation for a specific user-item pair (user u and item i). Example generated interpretations are illustrated in Figure 1 and Figure 4.
In particular, our GIANT model infers the latent topic for user u and item i, respectively. Such a topic essentially indicates which cluster user item i belongs to. We can then identify the user candidate reviews as the past reviews on item i written by the users (including u) in the same cluster as Similarly, we identify item candidate reviews from the past reviews on the item i which have their most prominent topic the same as the item latent topic.
Afterwards, we represent each topic by its top-associated 5 words and derive the topic representation by the aggregated word-level GloVe word embeddings. TFIDF scores of all words of reviews in the same topic cluster, with stop words filtered, The review sentence representations are also derived based on the aggregated constituent word GloVe embed- dings. The most relevant sentences from user candidate reviews can then be extracted as a summary of the user u's preferences based on their cosine similarity with the user latent topic representation. Similarly, the most relevant sentences from item candidate reviews are extracted as a summary of item i's characteristics based on their cosine similarity with the item latent topic representation. Apart from the user u's preferences and item i'ss characteristics, we also present to human evaluators the model's recommendation suggestion as will recommend, if the model's predicted rating score is above the average predicted rating; or won't recommend, otherwise.
These words are selected based on the
We also experimented with more representative words, but observed less discriminated topic clusters.
19

---

Under review as a conference paper at ICLR 2023
We propose three evaluation metrics, relevance, faithfulness and informativeness and ask three human evaluators to give 1-5 score to the generated interpretations from Wasser- stein VAE and GIANT on 120 randomly selected user-item pairs from the Office Products'
Relevance. If the extracted sentences from user reviews and item reviews are about the relevant topic/aspect/sujjee? higher score should be given the interpretation with more overlapping aspects.
Faithfulness. Do the extracted sentences from user reviews and item reviews lead to the model's recommendation suggestions?
Informativeness. Do the given interpretations capture the user preferences and item char- acteristics well?
Metric
Relevance
Faithfulness
Informativeness
Wasserstein VAE
3.54
2.84
3.84
GIANT
3.77
3.27
4.17
Table A7: Human evaluation results on salleancee,aitiffuee and informativnnsss for the generated interpretations of randomly selected 120 user-item pairs from the Office Products dataset.
From the results shown in Table A7 we can observe that interpretations generated by our model are better compared Wasserstein VAE across all three aspects.
!5Evllutioon the Beer and Music products appears be difficult as it requires prior knowledge the specific products. We will leave future work.
20


Reviews:
[{"review_id": "wqDpowv3zs", "reviewer": "Reviewer_JQQm", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes an explainable recommender system based on variational autoencoders (VAE). The main idea is to model user/item textual reviews with an autoencoder whose latent space is regularized by user/item cluster-membership distributions. The latter are derived from the user-item interaction data by first clustering users and items, and then estimating a user or item distribution based on its distance to each cluster centroid using a Gaussian Kernel. The authors argue that such regularization would enable the use of collaborative signals from similar users and items while generating explanations for a given user-item pair. Experiments are curried out on three real-world datasets. Various aspects of the proposed method are evaluated, including rating prediction and latent variable interpretability.       ", "summary_of_the_review": "Please refer to the strength and weaknesses section above. ", "strength_and_weaknesses": "Strengths.\n- The paper proposes an interesting idea to improve the interpretability of user/item latent factors in a recommender system by leveraging knowledge from both textual reviews as well as user/item clusters derived from user-item interaction signals.  \n- The proposed method offers competitive performance in terms of rating prediction as measured by MAE and RMSE metrics. Some qualitative/quantitative results also show that the proposed method seems to learn more coherent and meaningful latent representations and clusters. \n\nWeaknesses. \n- Writing requires some important improvements. Section 3 and 4 describing the proposed method are hard to follow. \n(I) The notations are a bit confusing. I would recommend using “t” and “g” to denote text and graph related quantities as superscript instead of subscript.\n(II) The transition from eq. 2 to eq. 3 is not obvious. An alternative is to introduce eq. 4 first, which corresponds to a Beta-VAE with a cluster-membership distribution as a prior, and then provide a connection to the IB objective as a supportive analysis for eq. 4. \n(III) Eq. 8 regarding the KL-term is not obvious either, is p(Z_t|x_n,e) Gaussian? \n- The focus of this work is on explainability, however the results regarding this aspect are weak. I would recommend including some human evaluations (user study) to better assess the quality of the explanations generated by proposed method compared to the baselines. \n- The performance of the proposed recommender system is measured using prediction metrics such MAE and RMSE. In general, MAE and RMSE do not necessarily reflect the quality of item recommendation. Reporting retrieval measures such as Precision and Recall would be more convincing.  \n\nAdditional comments.\n- The clustering component is central in the proposed method. It would be useful to have more experiments regarding this aspect. For instance, what is the impact of number of clusters on the performance of the proposed method. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "veoX4daBpRy", "reviewer": "Reviewer_fUWF", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors propose a new framework called GIANT (for Geometric InformAtioN boTtleneck) to provide results’ interpretations/explanations of a collaborative -filtering-based recommendation system. \n\nThis framework requires first the computation of a user-item interaction graph. \nEach node is represented by LightGCN encoding of:\n- for a user node: all text reviews the user wrote, \n- for an item node: all text reviews posted for this item.\n\nUser and item nodes are then clustered via K-means based on Radial Basis Function kernel for distance. The distance between a user or an item to the cluster centroid vectors is used as the probability to be assigned (soft assignment) to the cluster and the corresponding distribution is used as prior to learn user and item latent vectors from a Variational AutoEncoder (VAE).\n\nFor a pair of user-item taken from the interaction graph, user and item representations are fed to a VAE trained so that obtained latent vectors of the pair are used to predict the rating from that user for this specific item. \n\nFinal loss of the network does not take into account only the regression loss (for rating prediction) but also regularization terms:\n-one for constraining the learning of the latent vector in text modality by imposing the prior knowledge from the graph modality (information bottleneck)\n-one for preventing learned distribution of latent vector to be chaotic (prior-centralisation)\n", "summary_of_the_review": "The approach is showing on the 3 public benchmarks best performance so the method seems promising. Moreover the qualitative evaluation shows indeed interpretation associated is more valuable than the one associated to StandPrior. Why not compare with WassersteinVAE which was giving the second best performance in terms of rating prediction?\nMy score is currently a weak reject, score that can be increased when my questions (in weaknesses part of the review) are answered. \n\n", "strength_and_weaknesses": "Strengths:\n\nThe paper is quite clear and enjoyable to read. \nThe approach is clearly stated with the different steps:\n-interaction graph computation\n-encoding of the nodes\n-clustering\n-VAE: prior / posterior\n-Final objective function\n\nThe framework is supported by multiple experiments on 3 public datasets to evaluate quantitatively accuracy of the recommendation but also importance of each block of the loss by appropriate ablation study. Finally there is also a quantitative analysis of the interpretability of the recommendations based on the cluster analysis: as each dimension of the latent variables from the VAE encoding part are associated to a cluster, we can interpret the recommendation of a given item by looking at the the most frequent words associated to reviews from clusters associated to the highest values of coefficients of the latent vector of the item.\n\nWeaknesses / questions:\n\nI have several questions:\n\nQ1: Why not compare GIANT with Wasserstein VAE and only StandPrior for interpretability evaluation?\n\nQ2: What is the architecture of the predict head? Simply dot product of the latent vectors from the VAE?\n\nQ3: How are the text reviews treated before LightGCN? Is it the same treatment as for the input of the VAE encoder?\n\nQ4: Eq. (1). Which value do you take in practice for $I_c$?\n\nQ5: Is K-means applied on all types of nodes at the same time or two applications on user and items separately? I think there is only one application but p.4 the differentiation of $C^u_k$ and $C^i_k$ is a bit confusing. \n\nQ6: None of the methods described in the Related work section for explainable recommenders have been tested against GIANT. Am I mistaken?\n\nQ7: Did you try the Wassertein metric instead of KL divergence to calculate the distribution discrepancy between posterior and prior? (to make WassersteinVAE benefit from a better prior). \n\nMinor (typos):\n-p.3 text before equation 1, extra comma to remove after decode arrow\n-p.16 missing space between citation and Nadaraya-Watson estimator + bracket typo is conditional probability\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "OLY6VuT4TH", "reviewer": "Reviewer_Scqk", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper presents a review-based explainable recommedation model that predicts user preference and gives explanations based on user/item clusters. The core idea is to learn two sets of clusters, one based on text reviews and the other based on user/item interaction information. This explicitly ensures that information about similar users from the aspect of collaborative filtering can facilitate explanation generation for each other. Mathematically, this is achieved by using information bottleneck: the authors maximize the information between two sets of clusters, while ensuring compression (deriving compact clusters) and reconstruction also through information-based perspective.\nContribution: \n1. The idea to build a connection between multi-modal data (text and interaction) through mutual information is quite interesting and is novel to explainable recommendation according to my knowledge.\n2. The summarization of existing issues in explainable recommendation is good, especially the difficulty in getting ground-truth labels for explanations and the low quality for reviews. ", "summary_of_the_review": "The idea is novel and interesting, and the summarization about current issues in explainable recommendation also provides interesting insights. However, the explanation generation method is not clearly introduced and discussed, and the explanation quality is not well evaluated. Moreover, the related theories and equations can be better introduced to ensure readability. In its current state, it is difficult for me to judge whether the method has issues.", "strength_and_weaknesses": "Strength:\n1. The overall idea (use mutual information to connect multi-modal data) is interesting and novel, and it could be useful for explainable recommendation. \n2. The introduction includes a good summarization of existing issues in explainable recommendation, especially that the reviews usually are not good ground-truth labels for explanations, and that getting the true ground-truth is very difficult.\n\nWeakness:\n1. The major weakness is about the explanation introduction, quality, and evaluation.\n(1) Method: I cannot find a clear and formal introduction about how explanations are generated in the method section. The introduciton seems to talk about explanation generation with one sentence: \"and we extract the representative words in each dimension to offer recommendation explanations.\" According to this sentence and also Fig. 4, I think the explanations are actually many words in some clusters that are activated for a user-item pair. This is different from the example in Fig. 1, which contain only a few words. I suggest the authors more clearly introduce their explanation generation method, since explainability is a core contribution here.\n(2) Explanation quality: When borrowing words from similar items/users, how can you ensure that the selected words are relevant to the item and faithful (not misleading)? Also, the current methods only ensures similarity between input and decoded embeddings. Can this ensure that the decoded words are still relevant with the input words? Even if we can correctly decode a review, how can we ensure that the decoded words serve as good explanations? Can you help explain more about these questions?\n(3) Evaluation: No formal comparison with baselines in terms of explanation quality (e.g., whether the explanations are correct or useful). For example, what are the example explanations that you will show to users, and any comparions with baselines to prove your explanation quality? Also, any ablation study on whether the regularization really helps in improving explanation quality?\n2. Some descriptions about related works are not rigorous\n(1) \"However, the aforementioned approaches suffer from two major\nissues: they rely heavily on the accuracy of sentiment analysis tools (Guan et al., 2018); they tend\nto ignore implicit interactions not expressed directly in the individual review text.\" I do not think all mentioned related works rely on sentiment analysis tools? What do you mean by ignore implicit interactions? I think most methods consider user-item interaction information in the model (e.g., as model outputs)?\n(2) NAREE should be NARRE.\n3. The theory part is unclear to me.\n(1) Why the second term in Eq. (3) is I(X_t, Z_g) rather than I(X_t, Z_t)? Why the first term considers only Z_t but not Z_g? Do you use Eq. (2) to dervie Eq. (3)? Can you give a step-by-step derivation here? For example, can you first replace X, Z in Eq. (1) with Xt, Xg and Zt, Zg and gradually derive each equation?\n(2) The proof in Section B is also difficult to understand. I am an expert in explainable recommendation but I am not an expert in information bottleneck. Every derivation step needs to be clearly written for me to carefully check the correctness of the proof, e.g., what is the chain rule, how you apply it twice. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}]

Author rebuttal:

Decision: Reject