Original: E XPLAINABLE R ECOMMENDER WITH G EOMETRIC I NFORMATION B OTTLENECK
Revised: E XPLAINABLE R ECOMMENDER WITH G EOMETRIC I NFORMATION B OTTLENECK

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Explainable recommender systems have attracted much interest in recent years as they can explain their recommendation decisions, enhancing user trust in the systems.
Revised: Explainable recommender systems have attracted much interest in recent years as they can explain their recommendation decisions, enhancing user trust in the systems.

Original: Most explainable recommender systems rely on human-generated rationales or annotated aspect features from user reviews to train models for rational generation or extraction.
Revised: Most explainable recommender systems rely on human-generated rationales or annotated aspect features from user reviews to train models for rational generation or extraction.

Original: The rationales produced are often confined to a single review.
Revised: The rationales produced are often confined to a single review.

Original: To avoid the expensive human annotation process and to generate explanations beyond individual reviews, we propose an explainable recommender system trained on reviews by developing a transferable G eometric I nform A tio N bo T tleneck ( GIANT ), which leverages the prior knowledge acquired through clustering on a user-item graph built on user-item rating interactions, since graph nodes in the same cluster tend to share common characteristics or preferences.
Revised: To avoid the expensive human annotation process and to generate explanations beyond individual reviews, we propose an explainable recommender system trained on reviews by developing a transferable G eometric I nform A tio N bo T tleneck ( GIANT ), which leverages the prior knowledge acquired through clustering on a user-item graph built on user-item rating interactions, since graph nodes in the same cluster tend to share common characteristics or preferences.

Original: We then feed user reviews and item reviews into a variational network to learn latent topic distributions which are regularised by the distributions of user/item estimated based on their distances to various cluster centroids of the user-item graph.
Revised: We then feed user reviews and item reviews into a variational network to learn latent topic distributions which are regularised by the distributions of user/item estimated based on their distances to various cluster centroids of the user-item graph.

Original: By iteratively refining the instance-level review latent topics with GIANT , our method learns a robust latent space from text for rating prediction and explanation generation.
Revised: By iteratively refining the instance-level review latent topics with GIANT , our method learns a robust latent space from text for rating prediction and explanation generation.

Original: Experimental results on three e-commerce datasets show that our model significantly improves the interpretability of a variational recommender using a standard Gaussian prior, in terms of coherence, diversity and faithfulness, while achieving performance comparable to existing content-based recommender systems in terms of rating prediction accuracy.
Revised: Experimental results on three e-commerce datasets show that our model significantly improves the interpretability of a variational recommender using the Wasserstein distance while achieving performance comparable to existing content-based recommender systems in terms of rating prediction accuracy.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Typically, a recommender system compares users’ preferences with item characteristics (e.g., item descriptions or item-associated reviews) or studies user-item historical interactions (e.g., ratings, purchases or clicking behaviours) in order to identify items that are likely of interest to users.
Revised: Typically, a recommender system compares users’ preferences with item characteristics (e.g., item descriptions or item-associated reviews) or studies user-item historical interactions (e.g., ratings, purchases or clicking behaviours) in order to identify items that are likely of interest to users.

Original: In addition to predictive performance, interpretable recommenders aim to give rationale behind the rating given by a user on an item (Ghazimatin et al., 2020; Zhang et al., 2020).
Revised: In addition to predictive performance, interpretable recommenders aim to give rationale behind the rating given by a user on an item (Ghazimatin et al., 2020; Zhang et al., 2020).

Original: Most existing interpretable recommenders can either generate rationale or extract text spans from a given useritem review as explanations of model decisions.
Revised: Most existing interpretable recommenders can either generate rationale or extract text spans from a given useritem review as explanations of model decisions.

Original: Both rationale generation and extraction require annotated data for training, e.g., short comments provided by users explaining their behaviours of interacting with items, or annotated sentiment-bearings aspect spans in reviews (Zhang et al., 2014;
Revised: Both rationale generation and extraction require annotated data for training, e.g., short comments provided by users explaining their behaviours of interacting with items, or annotated sentiment-bearings aspect spans in reviews (Zhang et al., 2014;

Original: Ni et al., 2019;
Revised: Ni et al., 2019;

Original: Chen et al., 2019; Li et al., 2020; Tan et al., 2021a).
Revised: Chen et al., 2019; Li et al., 2020; Tan et al., 2021a).

Original: We argue that generating explanations based on a specific user-item review document suffers from the following limitations.
Revised: We argue that generating explanations based on a specific user-item review document suffers from the following limitations.

Original: First, some reviews may be too general to explain the rating, rendering them useless for explanation generation.
Revised: First, some reviews may be too general to explain the rating, rendering them useless for explanation generation.

Original: For example, the review ‘ I really like the smartphone, will recommend it to my friends ’ does not provide any clue why the user likes the smartphone.
Revised: For example, the review ‘ I really like the smartphone, will recommend it to my friends ’ does not provide any clue why the user likes the smartphone.

Original: Second, features directly extracted from a review document may fail to reflect some global properties which can only be identified from implicit user-item interactions.
Revised: Second, features directly extracted from a review document may fail to reflect some global properties which can only be identified from implicit user-item interactions.

Original: For example, meaningful insights could still be derived from reviews towards items that are not directly purchased/rated by a user but preferred by other like-minded users.
Revised: For example, meaningful insights could still be derived from reviews towards items that are not directly purchased/rated by a user but preferred by other like-minded users.

Original: Finally,learning an explanation generation model from user/item reviews  often requires text spans annotated as rationales explaining the corresponding review ratings. Such annotations are labour-intensive to obtain in practice.
Revised: Finally, explanation generation model from user/item reviews are often supervised by human-annotated rationales, which are labour-intensive to obtain in practice.

Original: E n c o d e r D e c o d e r Amber coloured with foam Poured a dark brown to black The flavor is hoppy but less fruity
Revised: To address the aforementioned limitations, we propose an AutoEncoder (AE) framework with variational G eometric I nform A tio N bo T tleneck ( GIANT ) to incorporate the prior from user-item interaction graph to refine the induced latent factors of user and item, and generate explanations in an unsupervised manner.

Original: Reviews from item 𝑖 " Reviews from user 𝑢
Revised: For a user-item pair, all reviews written by the user and reviews posted on the

Original: $ 𝑥 "(’)
Revised: [Figure]

Original: 𝑥 $()) Recommendation Rationale from related reviews You may like this beer, as you seem to be interested in roasted, brown, bitter flavours, and good mouthfeel 𝑖 $ 𝑖 " 𝑢 * 𝑖 * Rating PredictHead
Revised: r

Original: D e c o de r
Revised: E n c o d

Original: E n c o d e r 𝑧 $()) 𝑧 "(’) 𝑥̅ $()) 𝑥̅ "(’) Geometric Information Bottleneck
Revised: [Figure]

Original: Regularisation 𝑖 " 𝑖 * 𝑖𝑖 . 𝑢 " 𝑢 $ 𝑢 * 𝑢 - 𝑖 / 𝑖 $
Revised: r 𝑧

Original: U-I Graph
Revised: [Figure]

Original: To address the aforementioned limitations, we propose an AutoEncoder (AE) framework with variational G eometric I nform A tio N bo T tleneck ( GIANT ) to incorporate the distributions of users and items derived from the user-item interaction graph as regularisation to refine the induced latent factors of user and item.

Original: For a user-item pair, all reviews written by the user and reviews posted on the item are fed into two separate encoders to infer latent factors of users/items, and predict rating based on the match of their latent factors.
Revised: item are fed into two separate encoders to infer latent factors, and predict rating based on the match of their latent factors.

Original: The latent variables are supposed to capture the key semantic information to recover the original review text written by the user on the item.
Revised: The latent variables are supposed to capture the key semantic information to recover the original review text written by the user on the item.

Original: Different reviews are assigned to different latent dimensions according to posterior distributions and we extract the representative words in each dimension to offer recommendation explanations.
Revised: Different reviews are assigned to different latent dimensions according to posterior distributions and we extract the representative words in each dimension to summarize the cluster topic.

Revised: The explanations are thus from the reviews related to the topic of the assigned cluster (See in §6.3).

Original: Our proposed framework is illustrated in Figure 1.
Revised: Our proposed framework is illustrated in Figure 1.

Original: The geometric regularisation refers to the clusterbased distance with Gaussian variance, which is derived by firstly clustering the users/items in the user-item interaction graph, and then calculating the distribution of a user/item as its distance to each cluster centroid by a Gaussian kernel (§ 4.1).
Revised: The geometric regularisation refers to the clusterbased distance with Gaussian variance, which is derived by firstly clustering the users/items in the user-item interaction graph, and then calculating the distribution of a user/item as its distance to each cluster centroid by a Gaussian kernel (§ 4.1).

Original: To impose the regularisation on the latent variables of the AE framework taking the set of user reviews and item reviews as input, we apply the KL divergence to minimize the discrepancy between the induced posterior distribution of users z ( u ) , and items z ( i ) , and the geometric regularisation as the prior (§4.2), after linking the cluster to the review text encoder via prior-centralisation (§4.2).
Revised: To impose the regularisation on the latent variables of the AE framework taking the set of user reviews and item reviews as input, we apply the KL divergence to minimize the discrepancy between the induced posterior distribution of users z ( u ) , and items z ( i ) , and the geometric regularisation as the prior (§4.2), after linking the cluster to the review text encoder via prior-centralisation (§4.2).

Original: Experimental results on the three commonly used benchmarking datasets show that the proposed method achieves performance comparable with several strong baselines in recommendation.
Revised: Experimental results on the three commonly used benchmarking datasets show that the proposed method achieves performance comparable with several strong baselines in recommendation.

Original: Moreover, the quantity and the quality analysis in interpretability show that our method can generate coherent, diverse and faithful explanations.
Revised: Moreover, the quantity and the quality analysis in interpretability show that our method can generate coherent, diverse and faithful explanations.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: We review recommender systems, with particular attention to those built on Variational Autoencoder (VAE) (Kingma & Welling, 2014) or offering explanations of recommendation decisions.
Revised: We review recommender systems, with particular attention to those built on Variational Autoencoder (VAE) (Kingma & Welling, 2014) or offering explanations of recommendation decisions.

Original: Recommender with variational autoencoder Text data such as user reviews, item or brand descriptions could be important for developing a high-quality recommender system as they can be exploited to address the sparsity issue in user-item interactions.
Revised: Recommender with variational autoencoder Text data such as user reviews, item or brand descriptions could be important for developing a high-quality recommender system as they can be exploited to address the sparsity issue in user-item interactions.

Original: HFT (McAuley & Leskovec, 2013) and CTR (Wang & Blei, 2011) adopted Latent Dirichlet Allocation (LDA) (Blei et al., 2003) to extract latent topics from review text.
Revised: HFT (McAuley & Leskovec, 2013) and CTR (Wang & Blei, 2011) adopted Latent Dirichlet Allocation (LDA) (Blei et al., 2003) to extract latent topics from review text.

Original: VAE can also be used for text modeling due to its ability in extracting latent and interpretable features (Fei et al., 2021; Truong et al., 2021; Wang et al., 2020).
Revised: VAE can also be used for text modeling due to its ability in extracting latent and interpretable features (Fei et al., 2021; Truong et al., 2021; Wang et al., 2020).

Original: Truong et al.
Revised: Truong et al.

Original: (2021) argued that the commonly used isotropic Gaussian in VAE is over-simplified and proposed BiVAE by introducing constrained adaptive prior (CAP) for learning user- and itemdependent prior distributions.
Revised: (2021) argued that the commonly used isotropic Gaussian in VAE is over-simplified and proposed BiVAE by introducing constrained adaptive prior (CAP) for learning user- and itemdependent prior distributions.

Original: More recently, review information is used to disentangle the latent user intents at the finer granularity, for example, DisenGCN (Ma et al., 2019a) and DisenHan (Wang et al., 2020) used the graph attention mechanism to differentiate multiple relations and features.
Revised: More recently, review information is used to disentangle the latent user intents at the finer granularity, for example, DisenGCN (Ma et al., 2019a) and DisenHan (Wang et al., 2020) used the graph attention mechanism to differentiate multiple relations and features.

Original: Explainable Recommender System One popular interpretable method is to train a language generation model with the ground-truth explanations supplied, which can be the first sentence of a given review or human-annotated text spans in the review text (Li et al., 2017; Chen et al., 2019;
Revised: Explainable Recommender System One popular interpretable method is to train a language generation model with the ground-truth explanations supplied, which can be the first sentence of a given review or human-annotated text spans in the review text (Li et al., 2017; Chen et al., 2019;

Original: Ni et al., 2019; Li et al., 2022).
Revised: Ni et al., ; Li et al., 2022).

Original: Feature-level explanations aim to provide the important item features the users care about when making decisions (Zhang & Chen, 2020).
Revised: Feature-level explanations aim to provide the important item features the users care about when making decisions (Zhang & Chen, 2020).

Original: Explicit Factor Model adopted
Revised: Explicit Factor Model adopted

Original: Matrix Factorization (MF) to predict review ratings and aligned each dimension of latent factors of the MF results with some item features, thus explaining the recommendation decision by the corresponding features (Zhang et al., 2014).
Revised: Matrix Factorization (MF) to predict review ratings and aligned each dimension of latent factors of the MF results with some item features, thus explaining the recommendation decision by the corresponding features (Zhang et al., 2014).

Original: Other approaches firstly processed the review text to extract user-aspect-opinion and item-aspect-opinion tuples.
Revised: Other approaches firstly processed the review text to extract user-aspect-opinion and item-aspect-opinion tuples.

Original: They then modeled the interactions between the two tuples to derive the rating and the indicative features (Chen et al., 2016; 
Revised: They then modeled the interactions between the two tuples to derive the rating and the indicative features (Chen et al., 2016; Wang et al., 2018;

Original: Wang et al., 2018; Tan et al., 2021b; Chen et al., 2020).
Revised: Tan et al., 2021b; Chen et al., 2020).

Original: However, the aforementioned approaches suffer from two major issues: they rely heavily  on the accuracy of sentiment analysis tools (Guan et al., 2018); they tend to ignore implicit interactions not expressed directly in the individual review text.
Revised: However, the aforementioned approaches suffer from two major issues. Some approaches require aspects to be extracted first in order to provide aspect-related predictive features as interpretations, thus heavily dependent on the quality of the aspect extractors employed; while others rely on human-annotated rationales to train supervised language generators for explanation generation. Our proposed approach can instead generate explanations with the only supervision signals coming from review ratings.

Original:  G EOMETRIC I NFORMATION B OTTLENECK
Revised:  G EOMETRIC I NFORMATION B OTTLENECK

Original: To consider implicit user-item interactions for developing an interpretable recommender system, we propose to incorporate the geometric regularisation derived from user-item interaction graphs to learn the latent factors of users and items from review text in a variational network.
Revised: To consider implicit user-item interactions for developing an interpretable recommender system, we propose to incorporate the geometric regularisation derived from user-item interaction graphs to learn the latent factors of users and items from review text in a variational network.

Original: Although various work has been proposed to adopt richer priors to infer a more complex and realistic posterior in standard VAEs (Tomczak & Welling, 2017; Zhao et al., 2017), our solution is different from existing approaches as we need to impose the prior knowledge learned from the graph modality to constrain the learning of latent variables in the text modality.
Revised: Although various work has been proposed to adopt richer priors to infer a more complex and realistic posterior in standard VAEs (Tomczak & Welling, 2017; Zhao et al., 2017), our solution is different from existing approaches as we need to impose the prior knowledge learned from the graph modality to constrain the learning of latent variables in the text modality.

Original: In what follows, we show how this can be done under the theory of information bottleneck.
Revised: In what follows, we show how this can be done under the theory of information bottleneck.

Original: Based on the Information Bottleneck ( IB ) theory (Tishby & Zaslavsky, 2015), we can train an encoder-decoder architecture x encode −−−−→ z decode −−−−→, ¯ x , where x ∈ X , z ∈ Z , ¯ x ∈ ¯ X are the input,  hidden, and  reconstructed representations respectively,  by optimising
Revised: Based on the Information Bottleneck ( IB ) theory (Tishby & Zaslavsky, 2015), we can train an encoder-decoder architecture x encode −−−−→ z decode −−−−→ ¯ x , where x ∈ X , z ∈ Z , ¯ x ∈ ¯ X are the input, the hidden, and the reconstructed representations, respectively, in order to preserve the meaningful information about ¯ X in Z while maximally compress the information of X . We achieve it by maximising the following equation:

Original: [Equation]
Revised: [Equation]

Original: where I ( · ) is the mutual information and θ is the parameter in the encoder-decoder.
Revised: where I ( · ) denotes the mutual information, β is a Lagrange multiplier.

Original: The goal of maximising the mutual information in Eq.
Revised: The second term in Eq.

Original: (1) is tolearn an encoding Z that can maximally reconstruct ¯ X while maximally compress X under a constraint I c .
Revised: (1) is to maximally compress X , equivalent to minimize the mutual information between X and Z , denoted as I ( X ; Z ) ≤ I c , I c is the upper bound.

Original: In this paper, the goal is to build a mechanism which is able to transfer the constraint from graph to text, i.e., across different modalities.
Revised: In this paper, the goal is to build a mechanism which is able to compress the information across different modalities, i.e., the text and graph, while keep the mutual information between the input X and compressed representation Z bounded by an upper bound as well.

Original: Intuitively, we assume the  input x  has representations x t ∈ X t and x g ∈ X g under different modalities 1 satisfying the constraint, I ( X t , X g ) ≥ I x , since X t and X g are derived from the same input, they should be relevant.
Revised: Intuitively, we assume the n -th input x n has representations x tn ∈ X t and x gn ∈ X g under different modalities 1 satisfying the constraint, I ( X t ; X g ) ≥ I x , since X t and X g are derived from the same input, they should be relevant.

Original: Then,for two independent  encoder-decoder architectures  optimised by Eq.
Revised: Then, two independent well-trained encoder-decoder architectures in the two modalities optimised by Eq.

Original: (1) with the constraints, I ( X t , Z t ) ≤ I c
Revised: (1) are constrained by I ( X t ; Z t ) ≤ I c

Original: and I ( X g , Z g ) ≤ I c , the following property holds 2 :
Revised: and I ( X g ; Z g ) ≤ I c , respectively. The following property holds 2 :

Original: [Equation]
Revised: [Equation]

Original: where H ( · ) and H ( ·|· ) are entropy and conditional entropy, respectively.
Revised: where H ( · ) and H ( ·|· ) are entropy and conditional entropy, respectively.

Original: Equivalently, with the introduction of a Lagrange multiplier β , we can maximise the objective function:
Revised: Therefore, we can maximise the objective function using the input from the text modality, with a regularisation from another modality such as a user-item interaction graph, but still maintain a new upper bound:

Original: [Equation]
Revised: [Equation]

Original: According to (Alemi et al., 2017), the above lower bound can be approximated by applying the reparameterisation trick (Kingma & Welling, 2013) with a random Gaussian noise ϵ :
Revised: According to (Alemi et al., 2017), the above lower bound can be approximated by applying the reparameterisation trick (Kingma & Welling, 2013) with a random Gaussian noise ϵ :

Original: [Equation]
Revised: [Equation]

Original: wherethe x n is the training sample from one modality X t , ¯ x n is the VAE-based reconstruction result, q ( · ) and p ( · ) are the posterior probabilities estimated by decoder and encoder respectively, r ( Z g ) is the estimated distribution from other modality.
Revised: where x tn is the training sample from one modality X t (e.g., text), ¯ x tn is the VAE-based reconstruction output, q ( · ) and p ( · ) are the posterior probabilities estimated by decoder and the encoder, respectively, r ( Z g ) is the estimated distribution from another modality (e.g., user-item interaction graph).

Original: Here, Eq.
Revised: Here, Eq.

Original: (4) has a similar form to β -VAE which contains a reconstruction term Q ( x n ) defined by  posterior probability and a KL-divergence term KL(Z t | Z g ) regularised by the prior from other modality Z g .
Revised: (4) has a similar form as β -VAE which contains a reconstruction term Q ( x tn ) defined by the posterior probability and a KL-divergence term KL( Z t | Z g ) regularised by the prior from another modality Z g .

Original: By this equation, we are able to guarantee that the information bottleneck between two modalities are:
Revised: By this equation, we are able to guarantee that the information bottleneck between two modalities are:

Original: The proof is shown in the Appendix B.

Original: • Transferable −
Revised: • Transferable −

Original: According to Eq.
Revised: According to Eq.

Original: (2), the information bottleneck between modalities is still constrained by the global information bottleneck, which means that we can take the bound from one modality to guide the training on the other.
Revised: (2), the information bottleneck between modalities is still constrained by the global information bottleneck, i e., we can take the bound from one modality to guide the training on the other.

Original: • Practicable − One challenge in the training of variational models is the choice of the prior distribution.
Revised: • Practicable − One challenge in the training of variational models is the choice of the prior distribution.

Original: By Eq.
Revised: By Eq.

Original: (2), we do not have to utilise a pre-defined prior but instead use the posterior constraints from other modalities, which are more flexible and efficient.
Revised: (2), we do not have to utilise a pre-defined prior but instead use the posterior constraints from other modalities, which are more flexible and efficient.

Original: • Interpretable −
Revised: • Interpretable −

Original: The alignment between representations from different modalities and  corresponding probabilities defined on geometric space makes it possible to use modalities offering better interpretability, such as review text, to explain the black-box features from other modalities.
Revised: The alignment between representations from different modalities and their corresponding probabilities defined in the geometric space makes it possible to use modalities offering better interpretability, such as review text, to explain the black-box features from other modalities.

Original:  GIANT F RAMEWORK
Revised:  GIANT F RAMEWORK

Original: We propose a novel explainable recommender built on VAE with the geometric information bottleneck.
Revised: We propose a novel explainable recommender built on VAE with the geometric information bottleneck.

Original: For a given ( user, item ) pair, the input to our recommender is a set of reviews written by the user and a set of reviews on the item.
Revised: For a given ( user, item ) pair, the input to our recommender is a set of reviews written by the user and a set of reviews on the item.

Original: As each user or item is associated with multiple reviews, and some reviews might be very long, the Convolutional Neural Network (CNN) is used to encode reviews due to its efficiency in encoding long sequences, as have been previously studied in Chen et al.
Revised: As each user or item is associated with multiple reviews, and some reviews might be very long, the Convolutional Neural Network (CNN) is used to encode reviews due to its efficiency in encoding long sequences, as have been previously studied in Chen et al.

Original: ; Zheng et al.
Revised: ; Zheng et al.

Original: We then generate the user or item representation by aggregating their associated multiple reviews by the attention mechanism.
Revised: We then generate the user or item representation by aggregating their associated multiple reviews by the attention mechanism.

Original: The final contextual vector for user u and item i is denoted as x ( u ) and x ( i ) , respectively.
Revised: The final contextual vector for user u and item i is denoted as x ( u ) and x ( i ) , respectively.

Original: Given the contextual vectors, we employ two geometric regularised variational networks to infer their latent factors, in order to better capture their latent semantic topics.
Revised: Given the contextual vectors, we employ two geometric regularised variational networks to infer their latent factors, in order to better capture their latent semantic topics.

Original: According to Eq.
Revised: According to Eq.

Original: (2), we use graph-based Z g as the transferable information bottleneck to regularise the optimisation of the text-based Z t .
Revised: (2), we use graph-based Z g as the transferable information bottleneck to regularise the optimisation of the text-based Z t .

Original: To ensure feasibility and robustness, we propose an initialisation trick called prior-centralisation which uses the Gaussian kernel based geometric estimation to regularise the initialisation and optimisation of the encoders based on the transferable information bottleneck.
Revised: To ensure feasibility and robustness, we propose an initialisation trick called prior-centralisation which uses the Gaussian kernel based geometric estimation to regularise the initialisation and optimisation of the encoders based on the transferable information bottleneck.

Original: In the experiments section, we verify the interpretability of our proposed method by the alignment between the two latent feature spaces from graph and text modalities.
Revised: In the experiments section, we verify the interpretability of our proposed method by the alignment between the two latent feature spaces from graph and text modalities.

Original: 4. D ERIVING P RIORS FROM U SER /I TEM C LUSTERS IN I NTERACTION G RAPH
Revised: 4. D ERIVING P RIORS FROM U SER /I TEM C LUSTERS IN I NTERACTION G RAPH

Original: In the user-item bipartite graph G , a node can be either a user or an item.
Revised: In the user-item bipartite graph G , a node can be either a user or an item.

Original: For each user-item pair, an edge is created if the user has previously rated on the item and the rating score is higher than the average rating score calculated across all user-item ratings.
Revised: For each user-item pair, an edge is created if the user has previously rated on the item and the rating score is higher than the average rating score calculated across all user-item ratings.

Original: 3 We use the LightGCN (He et al., 2020) to encode our interaction graph and obtain the user and item embeddings.
Revised: 4 We use the LightGCN (He et al., 2020) to encode our interaction graph and obtain the user and item embeddings.

Original: Once user/item embeddings are learnt, we apply K-means on the learned node embeddings to derive the cluster centroid vectors, denoted as C uk for users and C ik for items, k ∈ { 1 , 2 , · · · , K } .
Revised: Once user/item embeddings are learnt, we apply K-means on the learned node embeddings to derive the cluster centroid vectors, denoted as C ( u ) k for users and C ( i ) k for items, k ∈ { 1 , 2 , · · · , K } .

Original: We use the Radial Basis Function (RBF) kernel to compute the distance between a user or item with the cluster k as the probability of assigning the user or item j to the k -th cluster, ρ jk :
Revised: We use the Radial Basis Function (RBF) kernel to compute the distance between a user or item with the cluster k as the probability of assigning the user or item j to the k -th cluster, ρ jk :

Original: [Equation]
Revised: [Equation]

Original: where e j is graph node embedding for user or item j , C k is k -th cluster centroid vector for user embeddings or item embeddings, α is a hyper-parameter adjusted according to the data density.
Revised: where e gj is graph node embedding for user or item j , C gk is k -th cluster centroid vector for user embeddings or item embeddings, α is a hyper-parameter adjusted according to the data density.

Original: The cluster-based distribution ρ  = { ρ jk } is then used to regularise the latent factors for users and items, as will be shown in § 4.2.
Revised: The cluster-based distribution ρ g = { ρ gjk } is then used to regularise the latent factors for users and items, as will be shown in § 4.2.

Original: According to definition given by Eq. (5), we have Σ k ρ jk = 1 ,which yields that the graph-feature based latent representation Z g = {√ ρ jk } are distributed on a S K − 1 hyper-sphere. By the theorem given by (Diaconis & Freedman, 1987), the distribution of {√ ρ jk } , which yields a hyper spherical cap area on an uniform K -ball, weakly approximates to a Gaussian distribution with the increasing value of K .
Revised: Moreover, we have Σ k ρ gjk = 1 , that is, the distribution of { (cid:113) ρ gjk } resides in a hyper spherical cap area on an uniform K -ball, weakly approximating to a Gaussian distribution when K is large (Diaconis & Freedman, 1987).

Original: Therefore, through the clustering, we obtain a latent vector Z g and a corresponding geometric space where the difference on each basis follows a Gaussian prior.
Revised: We therefore obtain a latent vector Z g and a corresponding geometric space where the difference on each basis follows a Gaussian prior.

Original: 4. A P OSTERIOR OVER C ONTROLLABLE D ISTRIBUTIONS FROM T EXT
Revised: A P OSTERIOR OVER C ONTROLLABLE D ISTRIBUTIONS FROM T EXT

Original: In section 4.1, we estimate the distribution of Z g from a user-item bipartite graph.
Revised: In section 4.1, we estimate the distribution of Z g from a user-item bipartite graph.

Original: The next step is to optimise the objective of Eq.
Revised: The next step is to optimise the objective of Eq.

Original: 4 to infer the latent factors of users and items, z ( u ) and z ( i ) in text modality.
Revised: 4 to infer the latent factors of users and items, z ( u ) and z ( i ) in text modality.

Original: In this section, we perform the following two steps:(1) minimising the lower bound of O IB by a RBF-kernel estimated posterior probability; and (2) a prior-centralisation term which encourages the encoder weight matrix in the variational network to be closer to the cluster centroid representations learned in the interaction graph, so that the learnt latent variables naturally capture the cluster-based distances.
Revised: In this section, we perform the following two steps:(1) minimising the lower bound of O tIB by a RBF-kernel estimated posterior probability; and (2) a prior-centralisation term which encourages the encoder weight matrix in the variational network to be closer to the cluster centroid representations learned in the interaction graph, so that the learnt latent variables naturally capture the cluster-based distances.

Original: Optimising Conditional Probability by Kernel Density For the Q ( x  n ) in Eq.
Revised: Optimising Conditional Probability by Kernel Density For the Q ( x t n ) in Eq.

Original: (4) which is defined as the conditional probability of log q (¯ x n | f ( x n , ϵ )) , we use the Nadaraya-Watson estimator (Hall et al., 1999):
Revised: (4) which is defined as the conditional probability of log q (¯ x tn | f ( x tn , ϵ )) , we use the Nadaraya-Watson estimator (Hall et al., 1999):

Original: [Equation]
Revised: [Equation]

Original: where κ ( · ) is a kernel function for density estimation, ¯ x n,ϵ is the reconstructed x n with random noise ϵ .
Revised: where κ ( · ) is a kernel function for density estimation, ¯ x tn,ϵ is the reconstructed x tn with random noise ϵ .

Original: In practice, we choose the RBF kernel κ = exp( −|| x − x ′ || 2 )) and ignore the denominator since the goal is to optimise the decoding of ¯ x n , then we are able to obtain the following bound by the triangle inequality in the learned metric space as Q ( x n )∝ log(exp
Revised: In practice, we choose the RBF kernel κ = exp( −|| x − x ′ || 2 )) and ignore the denominator since the goal is to optimise the decoding of ¯ x tn , then we are able to obtain the following bound by the triangle inequality in the learned metric space as Q ( x tn )

Original: ( −|| ¯ x n,ϵ − x n || 2 )) .
Revised: ∝ log(exp( −|| ¯ x tn,ϵ − x tn || 2 )) .

Original: To simplify the computation, we choose natural logarithm function in optimisation and obtain a mean squared error based objective function O IB 4 :
Revised: To simplify the computation, we choose natural logarithm function in optimisation and obtain a mean squared error based objective function O tIB 5 :

Original: [Equation]
Revised: [Equation]

Original: where x n ∈ { x ( u ) } (cid:83) { x ( i ) } denote the input user and item representations, and ¯ x n,ϵ denote the reconstruction.
Revised: where x tn ∈ { x ( u ) } (cid:83) { x ( i ) } denote the input user and item representations, and ¯ x tn,ϵ denote the reconstruction.

Original: By minimising the loss, we can preserve the key local similarity of nearby representations while ensuring different representations from different feature characteristics (Czolbe et al., 2020; Wang et al., 2004).
Revised: By minimising the loss, we can preserve the key local similarity of nearby representations while ensuring different representations from different feature characteristics (Czolbe et al., 2020; Wang et al., 2004).

Original: Note that we stop the gradient back-propagation of the input x n and optimise the encoder-decoder parameters only, because the updating of input representations might reduce the relevance between modalities, break the assumption of I ( X t , X g ) ≥ I x , and lead to two independent VAEs as a degeneracy of transferable information bottleneck given by Eq. 2.
Revised: Note that we stop the gradient back-propagation of the input x tn and optimise the encoder-decoder parameters only, because the updating of input representations might reduce the relevance between modalities, break the assumption of I ( X t , X g ) ≥ I x , and lead to two independent VAEs as a degeneracy of transferable information bottleneck given by Eq. 2.

Original: Minimising Distribution Discrepancy To minimise the second term of KL-divergence in O IB , we need to project Z g and Z t to a metric space with the same size.
Revised: Minimising Distribution Discrepancy To minimise the second term of KL-divergence in O tIB , we need to project Z g and Z t to a metric space with the same size.

Original: Inspired by (Van der Maaten &
Revised: Inspired by (Van der Maaten &

Original: Hinton, 2008), a Student t -distribution is used to mitigate the crowding problem.
Revised: Hinton, 2008), a Student t -distribution is used to mitigate the crowding problem.

Original: Here, we use the softmax activation with temperature τ to adjust the distribution tail and map its values to the range of [0 , 1] .
Revised: Here, we use the softmax activation with temperature τ to adjust the distribution tail and map its values to the range of [0 , 1] .

Original: The k -th dimension of a latent variable z nk 5 denotes the probability of the corresponding user or item being assigned to the k -th cluster:
Revised: The k -th dimension of a latent variable z tnk 6 denotes the probability of the corresponding user or item being assigned to the k -th cluster:

Original: [Equation]
Revised: [Equation]

Original: where φ ( z n ) is a linear transformation of z n .Here we minimise the KL-divergence between ρ jk (in the user-item interaction graph space) and η  jk .
Revised: where φ ( z tn ) is a linear transformation of z tn .Here we minimise the KL-divergence between ρ gjk (in the user-item interaction graph space) and η t jk .

Original: Therefore, we rewrite the O  IB as:
Revised: Therefore, we rewrite the O t IB as:

Original: [Equation]
Revised: [Equation]

Original: Prior-centralisation: Linking Clusters with Encoder Weights In the encoding progress, we obtain the latent variable z n = f ( x n , ϵ ) by applying the reparameterisation trick (Kingma & Welling, 2013).
Revised: Prior-centralisation: Linking Clusters with Encoder Weights In the encoding progress, we obtain the latent variable z tn = f ( x tn , ϵ ) by applying the reparameterisation trick (Kingma & Welling, 2013).

Original: However, due to the randomness in initialisation, the learned distribution of z n might be chaotic.
Revised: However, due to the randomness in initialisation, the learned distribution of z tn might be chaotic.

Original: Therefore, in our one-layer MLP based encoder, we propose a prior-centralisation trick which links the pre-trained clusters to encoder by approximating the encoder weights  to the cluster centroids C  d× K obtained on the graph:
Revised: Therefore, in our one-layer MLP based encoder, we propose a prior-centralisation trick which builds the connection between the pre-trained clusters and the encoder by making the encoder weights close to the cluster centroids C g ∈ R d obtained from the graph:

Original: [Equation]
Revised: [Equation]

Original: where W ken denotes the k -th column of the encoder weight matrix W en 6 and C k denotes the representation of the k -th cluster centroid vector.
Revised: where W ken denotes the k -th column of the encoder weight matrix W en 7 and C gk denotes the representation of the k -th cluster centroid vector.

Original: Note that the R centroid will force the distribution of z n approximate to the prior and ignore the input.
Revised: Note that the R centroid will force the distribution of z tn approximate to the prior and ignore the input.

Original: Therefore, in practise, this regularisation  only deployed in several beginning epochs.
Revised: Therefore, in practice, this regularisation is only deployed in the initial training epochs.

Original: Rating Prediction and Final Objective Function After obtaining the regularised latent contextual representations z ( u ) and z ( i ) , we add their corresponding ID features to obtain the final user and item embeddings ζ u and ζ i for rating prediction, ζ u = W u z ( u ) + ϵ u , ζ i
Revised: Rating Prediction and Final Objective Function After obtaining the regularised latent contextual representations z ( u ) and z ( i ) , we add their corresponding ID features to obtain the final user and item embeddings ζ u and ζ i for rating prediction, ζ u = W u z ( u ) + ϵ u , ζ i

Original: =
Revised: =

Original: W i
Revised: W i

Original: z ( u )
Revised: z ( u )

Original: + ϵ i , where the ID features ϵ u and ϵ i are generated by feeding the user ID and item ID to an embedding layer.
Revised: + ϵ i , where the ID features ϵ u and ϵ i are generated by feeding the user ID and item ID to an embedding layer.

Original: Inspired by the latent factor model in recommendation systems, we introduce the global biases for users and items in the final prediction layer as ˆ r ui =  ζ Tu ζ 
Revised: Inspired by the latent factor model in recommendation systems, we introduce the global biases for users and items in the final prediction layer as ˆ r ui = f cls ( ζ u , ζ i ) +

Original: i + b u + b i , where  b u and b i are bias derived from ϵ u and ϵ i .
Revised: b u + b i , where f cls combines the user and item features into a scalar, b u and b i are bias derived from ϵ u and ϵ i .

Original: The regression loss L r of the predicted rating is calculated as the MSE on the given user-item pair.
Revised: The regression loss L r of the predicted rating is calculated as the MSE on the given user-item pair.

Original: Combining all the components above, we derive the training objective as follows:
Revised: Combining all the components above, we derive the training objective as follows:

Original: [Equation]
Revised: [Equation]

Original:  E  XPERIMENTAL S ETUP
Revised:  E XPERIMENTS 5. E XPERIMENTAL S ETUP

Original: Datasets and Metrics The evaluation datasets include BeerAdvocate (McAuley et al., 2012) and two amazon review datasets, Digital Music and Office Products (He & McAuley, 2016) 7 .
Revised: Datasets and Metrics The evaluation datasets include BeerAdvocate (McAuley et al., 2012) and two amazon review datasets, Digital Music and Office Products (He & McAuley, 2016) 8 .

Original: We calculate Root Mean Squared Error (RMSE) and Mean Absolute Error (MAE) to evaluate the rating prediction accuracy.
Revised: We use Root Mean Squared Error (RMSE) and Mean Absolute Error (MAE) to evaluate the rating prediction accuracy.

Original: Baselines We compare with several open-source recommenders, including HFT (McAuley &
Revised: Baselines We compare with several open-source recommenders, including HFT (McAuley &

Original: Leskovec, 2013), DeepCoNN (Zheng et al., 2017), NAREE (Chen et al., 2018).
Revised: Leskovec, 2013), DeepCoNN (Zheng et al., 2017), NARRE (Chen et al., 2018).

Original: HFT combines reviews with ratings and uses an exponential transformation function to link review text and the ratings.
Revised: HFT combines reviews with ratings and uses an exponential transformation function to link review text and the ratings.

Original: DeepCoNN uses a shared layer for interaction modeling the users and items, which is on top of the two encoders for the users and items, respectively.
Revised: DeepCoNN uses a shared layer for interaction modeling the users and items, which is on top of the two encoders for the users and items, respectively.

Original: NAREE introduces review-level attentions to select important reviews and incorporates the user and item IDs as discriminative features in rating prediction.
Revised: NARRE introduces review-level attentions to select important reviews and incorporates the user and item IDs as discriminative features in rating prediction.

Original: Besides, we apply an encoder-decoder AutoEncoder on top of NAREE to learn the latent variable as a baseline to highlight the difference in our proposed regularisation.
Revised: Besides, we apply an encoder-decoder AutoEncoder on top of NARRE to learn the latent variable as a baseline to highlight the difference in our proposed regularisation.

Original: Training Procedure We train the model parameters by minimising the objective function defined in Eq.
Revised: Training Procedure We train the model parameters by minimising the objective function defined in Eq.

Original: The β for the KL divergence term is set to be 0.01 and we introduce the L 2 regularisation for all the model parameters and the weight is 0.001.
Revised: The β for the KL divergence term is set to be 0.01 and we introduce the L 2 regularisation for all the model parameters and the weight is 0.001.

Original: We only train the prior-centralisation term for 0.5 proportion to approximate the cluster centroids for the later training.
Revised: We only train the prior-centralisation term for 0.5 proportion to approximate the cluster centroids for the later training.

Original: The KL term in O IB should be introduced later until the encoder centroid is well trained.
Revised: The KL term in O IB should be introduced later until the encoder centroid is well trained.

Original: To do so, we follow a similar cyclical schedule (Fu et al., 2019) to gradually adjust the anneal factor λ in each epoch.
Revised: To do so, we follow a similar cyclical schedule (Fu et al., 2019) to gradually adjust the anneal factor λ in each epoch.

Original: Specifically, we first train the model without the KL term for 0.5 proportion, then anneal it from 0.5 to 1 for 0.
Revised: Specifically, we first train the model without the KL term for 0.5 proportion, then anneal it from 0.5 to 1 for 0.

Original: proportion, and finally fix λ = 1 .
Revised: proportion, and finally fix λ = 1 .

Original: E XPERIMENTAL R ESULTS

Original: We first show that the proposed model performs better in rating prediction compared to existing

Original: CNN-based recommenders.

Original: We then examine the contribution of each training objective and the geometric prior.

Original: We finally show that the learnt latent variables are disentangled, semantic coherent and informative, which benefit the explainable recommendation system.

Original: [Table]
Revised: [Table]

Original: 6. R ATING P REDICTION R ESULTS 
Revised: 5. R ATING P REDICTION R ESULTS 5.2. M AIN R ESULTS BY C OMPARING TO B ASELINES .

Original: The rating prediction results of our model in comparison with baselines on all datasets are given in
Revised: The rating prediction results of our model in comparison with baselines on all datasets are given in

Original: Table 1.
Revised: Table 1.

Original: We have the following observations.
Revised: We have the following observations.

Original: (1) DeepCoNN built on the stack of non-linear neural networks for review semantic modeling outperforms HFT which leverages an exponential transformation function to link topic distributions in review text and latent factors derived from ratings.
Revised: (1) DeepCoNN built on the stack of non-linear neural networks for review semantic modeling outperforms HFT which leverages an exponential transformation function to link topic distributions in review text and latent factors derived from ratings.

Original: This shows the superiority of deep learning for feature extraction.
Revised: This shows the superiority of deep learning for feature extraction.

Original: (2) AutoEncoder gives slightly better performance than NAREE in BeerAdvocate , which shows its effectiveness of extracting key contextual information for rating prediction.
Revised: (2) AutoEncoder gives slightly better performance than NARRE in BeerAdvocate , which shows its effectiveness of extracting key contextual information for rating prediction.

Original: (3) Our method consistently outperforms all the baselines and the improvement is more predominant on the BeerAdvocate , which has the smallest sparsity.
Revised: (3) Our method consistently outperforms all the baselines and the improvement is more predominant on the BeerAdvocate , which has the smallest sparsity.

Original: The results demonstrate the effectiveness of our proposed information bottleneck regularisation applied on the latent semantic space.
Revised: The results demonstrate the effectiveness of our proposed information bottleneck regularisation applied on the latent semantic space.

Original: By doing this, the users and items can be grouped into different clusters according to the interaction data, which is not the case in other textual CNN-based recommenders.
Revised: By doing this, the users and items can be grouped into different clusters according to the interaction data, which is not the case in other textual CNN-based recommenders.

Original: 6. P ERFORMANCE CONTRIBUTIONS FROM VARIOUS MODULES 
Revised: 5.2. P ERFORMANCE CONTRIBUTIONS FROM VARIOUS MODULES . a)

Original: Effects of different loss terms We study the effects of the three loss terms in rating prediction accuracy and Diversity of the latent variables.
Revised: Effects of different loss terms We study the effects of the three loss terms in rating prediction accuracy and Diversity of the latent variables.

Original: As variational networks could easily collapse into an unconditional generative model, i.e., in the extreme case, all the input will be mapped into the same latent code (Ma et al., 2019b).
Revised: As variational networks could easily collapse into an unconditional generative model, i.e., in the extreme case, all the input will be mapped into the same latent code (Ma et al., 2019b).

Original: We use the dimension index whose corresponding latent value is the maximum as the cluster assignment ID.
Revised: We use the dimension index whose corresponding latent value is the maximum as the cluster assignment ID.

Original: We then derive the cluster assignment results A ∈ R N × K , where N is the number of test samples, K is the cluster number.
Revised: We then derive the cluster assignment results A ∈ R N × K , where N is the number of test samples, K is the cluster number.

Original: The diversity is calculated based on entropy H ( X ) =
Revised: The diversity is calculated based on entropy H ( X ) =

Original: − (cid:80) k ∈ K p ( k ) log ( p ( k ))) , where p ( k ) is the fraction of the number of samples falling into the k -th cluster among all the samples.
Revised: − (cid:80) k ∈ K p ( k ) log( p ( k ))) , where p ( k ) is the fraction of the number of samples falling into the k -th cluster among all the samples.

Original: A larger value means a better diversity.
Revised: A larger value means a better diversity.

Original: We record the larger diversity between user and item latent variables.
Revised: We record the largest diversity between user and item latent variables.

Original: The results are shown in Table 2.
Revised: The results are shown in Table 2.

Original: [Table]
Revised: [Table]

Original: We firstly remove one of the three losses, i.e., R centroid ,

Original: KL term and Q ( x n ) to study their contributions. We observe that theoverall model performance drops, with the removal of Q ( x n ) leading to the largest performance degradation.
Revised: We observe that the removal of Q ( x tn ) leads to the largest performance degradation.

Original: After removing
Revised: After removing

Original: R centroid , the latent variable diversity shrinks to near zero, indicating nearly all the latent variables fall into the same cluster.
Revised: R centroid , the latent variable diversity shrinks to near zero, indicating nearly all the latent variables fall into the same cluster.

Original: This highlights the capability of our prior-centralisation term in enabling the latent variables z n effectively reflect the soft cluster assignments.
Revised: This highlights the capability of our prior-centralisation term in enabling the latent variables z tn effectively reflect the soft cluster assignments.

Original: We also found that replacing the MSE in Q ( x n ) with the cosine similarity ( 1 − cosine ) reduces the latent variable diversity significantly.
Revised: We also found that replacing the MSE in Q ( x tn ) with the cosine similarity ( 1 − cosine ) reduces the latent variable diversity significantly.

Original: This can be partly explained by the fact that the cosine similarity focuses on measuring the angle between the input x n and the reconstructed output ¯ x n , ignoring the magnitude of the vectors which is however important in our case. 
Revised: This can be partly explained by the fact that the cosine similarity focuses on measuring the angle between the input x tn and the reconstructed output ¯ x tn , ignoring the magnitude of the vectors which is however important in our case. b).

Original: Effect of our proposed information bottleneck regularisation We compare our approach with StandPrior , WassersteinVAE and IndivPrior : StandPrior (aka.
Revised: Effect of our proposed information bottleneck regularisation We compare our approach with StandPrior , WassersteinVAE and IndivPrior : StandPrior (aka.

Original: StandVAE ) uses two VAEs for users and items respectively, each with the standard Gaussian distribution N (0 , 1) as the prior.
Revised: StandVAE ) uses two VAEs for users and items respectively, each with the standard Gaussian distribution N (0 , 1) as the prior.

Original: WassersteinVAE differs from the StandPrior in using the Wasserstein metric (Tolstikhin et al., 2017) to calculate the distribution discrepancy between posterior η and prior ρ .
Revised: WassersteinVAE differs from the StandPrior in using the Wasserstein metric (Tolstikhin et al., 2017) to calculate the distribution discrepancy between posterior η and prior ρ .

Original: IndivPrior assigns a separate Gaussian prior to each user or item with its mean value calculated from the user’s or item’s corresponding representation derived from the user-item interaction graph, i.e., p ( z ( u ) )
Revised: IndivPrior assigns a separate Gaussian prior to each user or item with its mean value calculated from the user’s or item’s corre- sponding representation derived from the user-item interaction graph, i.e., p ( z ( u ) )

Original: =
Revised: =

Original: N ( W u ξu, I ) , p ( z ( i ) )
Revised: N ( W u ξu, I ) , p ( z ( i ) )

Original: =
Revised: =

Original: N ( W i ξi, I ) , where ξu ∈ R d , ξi ∈ R
Revised: N ( W i ξi, I ) , where ξu ∈ R d , ξi ∈ R

Original: d denote the user and item node representations in GCN learning, W u and W i are learnable parameters in a linear layer of size ( d × d ) , and d is the dimension of graph node features.
Revised: d denote the user and item node representations in GCN learning, W u and W i are learnable parameters in a linear layer of size ( d × d ) , and d is the dimension of graph node features.

Original: Our method achieves the best, followed by StandPrior and WassersteinVAE .

Original: IndivPrior is the worst, even worse than AutoEncoder .

Original: The rating prediction results are shown in Figure 2.
Revised: The rating prediction results are shown in Figure 2.

Original: WassersteinVAE and StandPrior demonstrate overall better results than IndivPrior and all the baselines in Table 1.
Revised: WassersteinVAE and StandPrior demonstrate overall better results than IndivPrior and all the baselines in Table 1.

Original: The improvement can be explained by the nature of VAE, which is particularly beneficial when dealing with sparse data where few observations are available.
Revised: The improvement can be explained by the nature of VAE, which is particularly beneficial when dealing with sparse data where few observations are available.

Original: However, IndivPrior , which imposes a separate Gaussian prior on individual user/item, shows worse performance compared to the other three variational frameworks, and is even worse than AutoEncoder .
Revised: However, IndivPrior , which imposes a separate Gaussian prior on individual user/item, shows worse performance compared to the other three variational frameworks, and is even worse than AutoEncoder .

Original: Our model benefits from a rich prior separated from different user/item clusters, while avoiding using a global normal prior ( StandPrior and WassersteinVAE ) or a separate instance-level prior for each user/item ( IndivPrior ).
Revised: Our model benefits from a rich prior separated from different user/item clusters, while avoiding using a global normal prior ( StandPrior and WassersteinVAE ) or a separate instance-level prior for each user/item ( IndivPrior ).

Original: 6.  L ATENT V ARIABLE I NTERPRETABILITY E VALUATION 
Revised:  L ATENT V ARIABLE I NTERPRETABILITY E VALUATION 6. C LUSTER S EPARABILITY AND C OHERENCE

Original: Cluster Separability and Coherence

Original: As each dimension of the latent variables z n corresponds to a cluster, for the k -th dimension, we can search for its relevant reviews which have the highest value in z nk and list the most frequent words in the review set as the representative topic words 9 .
Revised: As each dimension of the latent variables z tn corresponds to a cluster, for the k -th dimension, we can search for its relevant reviews which have the highest value in z tnk and list the most frequent words in the review set as the representative topic words 11 .

Original: Representative words in randomly selected three clusters on BeerAdvocate Dataset are shown in Table 3.
Revised: Representative words in randomly selected three clusters on BeerAdvocate Dataset are shown in Table 3.

Original: For our model, the most prominent words are different across different clusters (thus being colored), i.e., ‘ brown roasted ’, ‘ pine, caramel ’ and ‘ good flavour ’.
Revised: For our model, the most prominent words are different across different clusters (thus being colored), i.e., ‘ brown roasted ’, ‘ pine, caramel ’ and ‘ good flavour ’.

Original: We fail to seea clear topic pattern in StandPrior as the top words largely overlap in different clusters.
Revised: We find it hard to see clear topic separations from the WassersteinVAE results.

Original: The first several words are all ‘ hop ’, ‘ malt ’ and ‘ good ’.
Revised: For example, Cluster 2 and 3 share the same 3 words, ‘ sweet ’, ‘ light ’ and ‘ malt ’ out of their topwords.

Revised: In addition, ‘ carbonation ’ in Cluster 2 and ‘ whitehead ’ in Cluster 3 are both relating to beer foam.

Original: The results show that with the incorporation of the priors derived from user/item clusters, our proposed approach is able to learn latent variables in the review semantic space which can produce better separable topic clusters.
Revised: The results show that with the incorporation of the priors derived from user/item clusters, our proposed approach is able to learn latent variables in the review semantic space which can produce better separable topic clusters.

Original: [Table]
Revised: [Table]

Revised: 6. C OMPREHENSIVENESS E VALUATION BY PERTURBING ON LATENT VARIABLES

Revised: To explore the importance of our identified clusters

Original: [Table]
Revised: [Table]

Original: We explore the difference between our generated clusters z nk and the clusters derived in the graph. To verify the capability of creating semantically coherent clusters , we propose to measure the cluster coherence as the average cosine similarity between every review document pair within a cluster. We first obtain the document-level review representations by feeding the reviews from the test set to our pre-trained CNN encoder.
Revised: i

Original: We then calculate the average cosine similarity between the representations of each review pair in a cluster, and are further averaged across all clusters (in Table 4).
Revised: [Table]

Original: Comprehensiveness Evaluation by perturbing on latent variables To explore the faithfulness of our generated explanations, i.e., the identified predominant cluster in the latent dimension, we calculate the performance change before and after removing the specific dimension and define Comprehensiveness as:Comprehensiveness = (cid:80) Ni ( r(z i ) − r(z
Revised: e., the latent dimension with larger value [Table] we calculate the performance change before and after removing the specific dimension and define Comprehensiveness as:

Original: i / e i ) ) 2 N , Where r ( · ) is the predicted rating, z i is the latent variable for i -th evaluated user-item pair, e i is thegenerated explanations, i.e., the identified top k latent dimensions.
Revised: [Equation] N , Where r ( · ) is the predicted rating, z ti is the latent variable for i -th evaluated user-item pair, z t [ k ] i is the identified top k latent dimensions.

Original: To delete the associated indicative features, we replace the values in the top k dimensions with the average value of the latent variables according to Fong & Vedaldi (2017).
Revised: To remove the effects from these dimensions, we replace the values in the top k dimensions with the average value of the latent variables according to Fong & Vedaldi (2017).

Original: In the Figure 3 line chart,our model demonstrates a larger performance change as more important dimensions are removed and its changes are more obvious than the StandVAE .
Revised: In the Figure 3 line chart, model demonstrates a larger performance change as more important dimensions are removed and its changes are more obvious than the two baselines.

Original: To avoid the effects from performance fluctuation caused by the model itself, we randomly delete k latent dimensions and calculate the relative performance change by subtracting the changes caused by random removal (Table below Figure 3). 
Revised: We further randomly delete k latent dimensions and calculate the relative performance change by subtracting the changes caused by random removal (Table below Figure 3). The relative changes in our model are most predominant, followed by WassersteinVAE [Figure]

Original: Some relative changes of StandVAE are negative, showing that the  removal ofrandom latent dimensions can bring even larger changes to the predominant ones. 
Revised: while relative changes of StandPrior are negative, showing that the random removal of latent dimensions can bring even larger changes to the predominant ones. [Figure] [Table] [Figure]

Original:  [Figure]
Revised: [Table] [Figure] [Table] [Figure]

Revised: 6. C ASE STUDY OF G ENERATED I NTERPRETATIONS

Original: [Figure]
Revised: Interpretations for an example user-item pair in Office Products are generated by extracting sentences most relevant to user/item latent topics from reviews of like-minded users and past reviews about the item. 12 While existing explainable recommenders are unable to extract information beyond the current user-item review, GIANT can extract sentences from reviews of like-minded users to explain the current user-item interaction based on the user/item clustering results. Our human evaluation results presented in Table A7 further show that explanations generated by GIANT are better than those from WassersteinVAE in terms of relevance, faithfulness and informativeness. [Table]

Original: to derive the latent variables for rating prediction. We identify the top 3 prominent clusters which has the larger latent values (illustrated by size) to show the fine-grained explanations.
Revised: [Figure]

Original: Fine-grained Interpretations We illustrate how our approach could explain the recommender behaviour by providing fine-grained interpretations in Figure 4.
Revised: [Table] [Figure] [Table] [Figure]

Original: The two users are mapped into the same top 3 clusters ( z ( u ) 105 , z ( u ) 73 , z ( u ) 155 ), showing interests about the ‘ rock ’ music, ‘ rap ’ and ‘ lyrics ’, respectively.
Revised: [Table] [Figure] [Table] [Figure]

Original: The first item (I156) is mapped to three clusters with main features about the rock group ‘ ABBA ’ and another features in ‘ lyrics ’, ‘ performance’ , while the second item(I262) is assigned to ‘ classic lyric ’, ‘ classic vocal ’ and ‘ folk ’, respectively.
Revised: [Table] Will recommend [Table] item

Original: There is a good match between the user primary interest ( rock music ) and the item main characteristics ( ABBA band ) and hence explains a higher rating of 4.91.
Revised: [Table]

Original: Our model outputs also explain that the user ‘U1759’, who is also featured as a rock music fan, expresses little interests (2.01 rating score) in item‘I262’ about classic style music .
Revised: [Figure] recommend [Figure] item

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this paper, we leverage the user/item clusters sharing common interests/characteristics obtained on the  interaction graph to refine the review text latent factor via our proposed geometric information bottleneck ( GIANT ).
Revised: In this paper, we leverage the user/item clusters sharing common interests/characteristics obtained from the user-item interaction graph to refine the review text latent factors via our proposed geometric information bottleneck ( GIANT ).

Original: We use the information bottleneck theory to elaborate its transferability and feasibility across modalities and design a variational framework to impose the prior derived from the graph to review latent space.

Original: We empirically show that GIANT is better in learning a semantically coherent and interpretable latent space and the generated explanations are  faithful to the model, while achieving comparable rating prediction accuracy on three commonly used datasets.
Revised: We empirically show that GIANT is better in learning a semantically coherent and interpretable latent space and the generated explanations are more faithful to the model decisions, while achieving comparable rating prediction accuracy on three commonly used datasets.

Original: A E XPERIMENT S ET U P A. D ATASET D ETAILS AND D ATA P ROCESSING
Revised: A E XPERIMENT S ET U P A. D ATASET D ETAILS AND D ATA P ROCESSING

Original: We choose three commonly used e-commerce datasets which not only contain the interaction interactions between user and item but also the review texts.
Revised: We choose three commonly used e-commerce datasets which not only contain the interaction interactions between user and item but also the review texts.

Original: Amazon review data 10 is one of the popular dataset collection, consisting of 24 kinds of products.
Revised: Amazon review data 13 is one of the popular dataset collection, consisting of 24 kinds of products.

Original: As our method derive the user/item mainly based on their reviews, we use the dense subset, 5-core, of the amazon review dataset that extract the user-item pairs, such that each of the remaining users and items have at least 5 reviews each.
Revised: As our method derive the user/item mainly based on their reviews, we use the dense subset, 5-core, of the amazon review dataset that extract the user-item pairs, such that each of the remaining users and items have at least 5 reviews each.

Original: Considering the training efficiency and topics should be familiar to general readers, we select the two datasets with around 50k reviews, Digital Music and Office Products .
Revised: Considering the training efficiency and topics should be familiar to general readers, we select the two datasets with around 50k reviews, Digital Music and Office Products .

Original: Besides, we use a sparser dataset BeerAdvocate dataset, i.e., the lower rating of observed ratings between total interactions.
Revised: Besides, we use a sparser dataset BeerAdvocate dataset, i.e., the lower rating of observed ratings between total interactions.

Original: Table A1 summarizes the statistics of the three datasets.
Revised: Table A1 summarizes the statistics of the three datasets.

Original: We further filter out users or items with more than 100 reviews/interactions and truncate the reviews to length of [200,400,300] for the three datasets, respectively.
Revised: We further filter out users or items with more than 100 reviews/interactions and truncate the reviews to length of [200,400,300] for the three datasets, respectively.

Original: We randomly select 80%, 10%, 10% as the train/validation/test sets.
Revised: We randomly select 80%, 10%, 10% as the train/validation/test sets.

Original: [Table]
Revised: [Table]

Original: A. H YPER -P ARAMETER S ETTINGS
Revised: A. H YPER -P ARAMETER S ETTINGS

Original: We use Adam for optimization with the initial learning rate set to 0.001.
Revised: We use Adam for optimization with the initial learning rate set to 0.001.

Original: The batch size is set to 32, 64, and 32, and training epochs are 10, 10, and 15 for the three datasets, respectively.
Revised: The batch size is set to 32, 64, and 32, and training epochs are 10, 10, and 15 for the three datasets, respectively.

Original: The adjacency matrix in graph is binary: user-item entries with the interacted ratings higher than the average corpus-wide rating is set to 1, and 0 otherwise.
Revised: The adjacency matrix in graph is binary: user-item entries with the interacted ratings higher than the average corpus-wide rating is set to 1, and 0 otherwise.

Original: We stop training GCN when it reaches the best recall value in the validation set.
Revised: We stop training GCN when it reaches the best recall value in the validation set.

Original: The cluster centroid vectors are derived by K-means.
Revised: The cluster centroid vectors are derived by K-means.

Original: The graph node feature dimension d is 64, and the number of clusters K are searched in [16 , 32 , 64 , 128 , 256 , 512] .
Revised: The graph node feature dimension d is 64, and the number of clusters K are searched in [16 , 32 , 64 , 128 , 256 , 512] .

Original: In CNN review encoding, the word embedding and ID embedding are both initialized by the uniform distribution ( − 0 .
Revised: In CNN review encoding, the word embedding and ID embedding are both initialized by the uniform distribution ( − 0 .

Original: 1 , 0 . 1) , their dimensions are set to 300 and 64, respectively.
Revised: 1 , 0 . 1) , their dimensions are set to 300 and 64, respectively.

Original: We use two
Revised: We use two

Original: CNN kernels with size 2 and 3, and the dimension of 32 to encode the reviews.
Revised: CNN kernels with size 2 and 3, and the dimension of 32 to encode the reviews.

Original: As such, the CNNencoder outputs the review representations with dimension of 64 (i.e., 32 × 2).
Revised: As such, the CNNencoder outputs the review representations with dimension of 64 (i.e., 32 × 2).

Original: To combine multiple reviews for each user or item, we use the attention mechanism where the attention weights are derived through two consecutive linear transformation layers of 64 → 32 → 1 .
Revised: To combine multiple reviews for each user or item, we use the attention mechanism where the attention weights are derived through two consecutive linear transformation layers of 64 → 32 → 1 .

Original: Inthe GIANT , the latent variable dimension is set to the same as the number of clusters K .
Revised: In GIANT , the latent variable dimension is set to the same as the number of clusters K .

Original: The softmax temperature τ is searched in [1 , 2 , 4 , 6 , 8] .
Revised: The softmax temperature τ is searched in [1 , 2 , 4 , 6 , 8] .

Original: The non-linear activation function in both encoder and decoder is ReLU.
Revised: The non-linear activation function in both encoder and decoder is ReLU.

Original: We use a linear layer with [ K, 64] weights to map the the latent variable z n to the same space of the 
Revised: We use a linear layer with [ K, 64] weights to map the the latent variable z tn to the same space of the ID features.

Original: ID features, and the  linear rating prediction head is [64 , 1] .
Revised: For f cls used in rating prediction, we first apply the ReLU function on the result of element-wise multiplication of the user and the item features, then apply a linear layer with the shape of [64 , 1] to transform the ReLU output to a rating scalar.

Original: B T RANSFERABLE I NFORMATION B OTTLENECK
Revised: B T RANSFERABLE I NFORMATION B OTTLENECK

Original:  Suppose we use independent  encoder-decoder based architectures to model the features for different modalities (e.g., graph and text), then a given input x  has representations x t ∈ X t and x g ∈ X g under different modalities with the constraint I ( X t , X g ) ≥ I x since X t and X g which are from the same input, should be relevant.
Revised: Theorem Suppose we use independent well-trained encoder-decoder based architectures to model the features for different modalities (e.g., graph and text), then a given input x n has representations x tn ∈ X t and x gn ∈ X g under different modalities with the constraint I ( X t ; X g ) ≥ I x since X t and X g which are from the same input, should be relevant.

Original: Also, assuming I ( Z t , X t ) ≤ I c 
Revised: Also, assuming I ( Z t ; X t ) ≤ I c and

Original: and I ( Z g , X g ) ≤ I c , then the following property holds:
Revised: I ( Z g ; X g ) ≤ I c , then the following property holds:

Original: [Equation]
Revised: [Equation]

Original: Proof: We first apply the chain rule twice  for three sources X t , X g and Z g , it can be written:
Revised: Proof: We first apply the chain rule twice in different orders for three sources X t , X g and Z g below:

Original: [Equation]
Revised: [Equation]

Original: Thus following inequality must be true:
Revised: Since the right parts of the above two equations are the same, we have:

Original: [Equation]
Revised: [Equation]

Original: By applying the above equation to the mutual information between X t and Z g , where I ( X  t , Z g ) =
Revised: which can be simplified as: [Equation] Since Z g , X t and X g are derived from the same set of samples but under different modalities or representations, they are obviously not independent. Hence, we have H ( X g | X t , Z g ) ≤

Original: H ( X  t ) − H ( X t |  Z g ) , we have:
Revised: H ( X g | X t ) (as X g and Z g are dependent) and H ( X t | X g , Z g ) ≥ 0 (as X t and X g are dependent). Thus, we can replace H ( X g | X t , Z g ) with H ( X g | X t ) , and replace H ( X t | X g , Z g ) with 0, to derive the following inequality:

Original: [Equation]
Revised: [Equation]

Original: Here, the lowest upper bound of the transferable information bottleneck is I ( X t , Z g )≤ I c when
Revised: By applying the properties of conditional differential entropy, which yields H ( X g | Z g )

Original: X t =X g . It means that the distribution of two modalities are the same and the conditional entropy of H ( X t |
Revised: =

Original:  X g ) is 0 .
Revised: H ( X g ) −

Original: Based on a similar proof, we can obtain another results holds by I ( X t , Z t ) :
Revised: I ( X g , Z g ) , the above formula can be simplified as: [Equation] Accordingly, we have: [Equation]

Original:  I ( X t , Z g ) ≤H + I ( X t  , Z t ) where H = H
Revised: □ Equation 12 shows that the lowest upper bound of the transferable information bottleneck is I ( X t ; Z g ) ≤ I c when the distribution of two modalities are the same, X t = X g , and the conditional entropy of H ( X g | X t ) is 0 . Let’s take H

Original:  ( X g ) − H ( X t ) 
Revised: = H ( X g ) − H ( X t ) + H ( X t | X g ) , then the Equationcan be simplified as I ( X g ; Z t ) ≤ H + I c , where I c is the upper boundary of information bottleneck for both modalities.

Revised: Recall the Lagrange multiplier based objective function defined in Eq. 1,

Revised: [Equation]

Revised: which aims to optimise:

Original: + H
Revised: [Equation] Now, let us focus on the optimisation on the encoder-decoder framework in text by denoting the variables with the t superscript. If we use the latent distribution learned from the graph modality, Z g , to impose constraints on Z t , the objective can be rewritten as:

Revised: [Equation]

Original:  ( X g | X t ) .
Revised: Since H = H ( X g ) − H ( X t ) + H ( X t

Revised: | X g ) is irrelevant with our optimisation, the Lagrange multiplier based objective function can be the rewritten in the form of Eq. 3:

Revised: [Equation]

Revised: That is, there is no need to define the prior for the text modality.

Revised: Instead, we can use the posterior distribution of the latent variable from the graph modality as the regularisation to guide the training on the text data.

Original: C O PTIMISING O BJECTIVE OF T RANSFERABLE I NFORMATION B OTTLENECK
Revised: C O PTIMISING O BJECTIVE OF T RANSFERABLE I NFORMATION B OTTLENECK

Original: Information Bottleneck: Recall that the IB objective in this work has the form I ( Z t , ¯ X t ) − β ·
Revised: Information Bottleneck: Recall that the IB objective in this work has the form I ( Z t ; ¯ X t ) − β ·

Original: I (Z g , X t  ) , where
Revised: I ( X t ; Z g ) , where

Original: [Equation]
Revised: [Equation]

Original: where p ( · ) is the true distribution which is not observed.
Revised: where p ( · ) is the true distribution which is not observed.

Original: According to the proof given by (Alemi et al., 2017), we consider the posterior estimation q (¯ x t | z t ) in the decoder.
Revised: According to the proof given by (Alemi et al., 2017), we consider the posterior estimation q (¯ x t | z t ) in the decoder.

Original: According to Gibbs’ inequality, we have:
Revised: According to Gibbs’ inequality, we have:

Original: [Equation]
Revised: [Equation]

Original: Since the entropy of the decoding results H ( ¯ X t ) is independent of model optimisation, we only need to consider the posterior estimation based lower bound.
Revised: Since the entropy of the decoding results H ( ¯ X t ) is independent of model optimisation, we only need to consider the posterior estimation based lower bound.

Original: According to(Alemi et al., 2017), we have the following empirical approximation by reparameterisation trick:
Revised: According to(Alemi et al., 2017), we have the following empirical approximation by reparameterisation trick:

Original: [Equation]
Revised: [Equation]

Original: For the Lagrange multiplied term I ( X t , Z g ) which is different from the standard information bottleneck based β -VAE, we do consider the information bottleneck of I ( X t , Z t ) first, because
Revised: For the Lagrange multiplied term I ( X t , Z g ) which is different from the standard information bottleneck based β -VAE, we do consider the information bottleneck of I ( X t , Z t ) first, because

Original: I ( X t , Z g ) ≤ H + I ( X t , Z t ) according to the proof given in section B.
Revised: I ( X t , Z g ) ≤ H + I ( X t , Z t ) according to the proof given in section B.

Original: Here, we have the following bound by applying Gibbs’ inequality:
Revised: Here, we have the following bound by applying Gibbs’ inequality:

Original: [Equation]
Revised: [Equation]

Original: Since estimating the prior distribution of Z t might be difficult, based on the definition of transferable information bottleneck, we apply the Gibbs’ inequality and have:
Revised: Since estimating the prior distribution of Z t might be difficult, based on the definition of transferable information bottleneck, we apply the Gibbs’ inequality and have:

Original: [Equation]
Revised: [Equation]

Original: where r ( z g ) is the posterior distribution from the graph-based latent representation, and H is decided by the prior of two modalities which is independent of model optimisation.
Revised: where r ( z g ) is the posterior distribution from the graph-based latent representation, and H is decided by the prior of two modalities which is independent of model optimisation.

Original: By combining the two results, the above bound can be approximated by the reparameterisation trick (Kingma & Welling, 2013) with a Gaussian random variable ϵ :
Revised: By combining the two results, the above bound can be approximated by the reparameterisation trick (Kingma & Welling, 2013) with a Gaussian random variable ϵ :

Original: [Equation]
Revised: [Equation]

Original: Further discussion about the reconstruction term Q ( x n )
Revised: Further discussion about the reconstruction term Q ( x tn )

Original: In section 4. [Equation] we propose to use the Nadaraya-Watson estimator(Hall et al., 1999) to approximate the conditional probability q (¯ x n | f ( x n , ϵ )] .
Revised: In section 4.2, we propose to use the Nadaraya-Watson estimator(Hall et al., 1999) to approximate the conditional probability q (¯ x tn | f ( x tn , ϵ )] .

Original: The idea is to insert the decoding result ¯ x n to the space of training samples { x n } , and take the kernel density estimation method to approximate Q ( x n ) by
Revised: The idea is to insert the decoding result ¯ x tn to the space of training samples { x tn } , and take the kernel density estimation method to approximate Q ( x tn ) by

Original: Bayesian rule:
Revised: Bayesian rule:

Original: [Equation]
Revised: [Equation]

Original: Since the input feature for x n is fixed [Equation] we only care about the updating of reconstruction through encoder-decoder architecture.
Revised: Since the input feature for x tn is fixed, we only care about the updating of reconstruction through encoder-decoder architecture.

Original: Thus we have:
Revised: Thus we have:

Original: [Equation]
Revised: [Equation]

Original: To simplify the above estimation
Revised: To simplify the above estimation

Original: [Equation]
Revised: [Equation]

Original: we apply the RBF-kernel and triangle inequality in estimation and have:
Revised: we apply the RBF-kernel and triangle inequality in estimation and have:

Original: [Equation]
Revised: [Equation]

Original: Thus, we are able to optimise the Q ( x n ) by mean square error if we choose the natural logarithm function:
Revised: Thus [Equation] we are able to optimise the Q ( x tn ) by mean square error if we choose the natural logarithm function:

Original: [Equation]
Revised: [Equation]

Original: Therefore, we can rewrite the lower boundary as well as the learning objective O IB by:
Revised: Therefore [Equation] we can rewrite the lower boundary as well as the learning objective O IB by:

Original: [Equation]
Revised: [Equation]

Original: D A DDITIONAL E XPERIMENT R ESULTS D. C LUSTER S EPARABILITY
Revised: D A DDITIONAL E XPERIMENTAL R ESULTS D. E VALUATION RESULTS ON RANKING - BASED METRICS

Revised: In addition to the accuracy-based metrics, i [Equation]

Revised: e., MSE and MAE, we also evaluate models using the ranking-based metrics, i.e., Precision and Recall (See Table A2).

Revised: We observe that GIANT achieves the best precision results in general across all the datasets, with a more prominent improvement compared to baselines on Office Products .

Revised: In terms of recall, GIANT outperforms the others on

Revised: Digital Music and Office Products , but it is inferior on BeerAdvocate .

Revised: WasersteinVAE appears to be the second-best model.

Revised: The results show that GIANT is more effective on datasets covering products with distinct features, but its advantage over the baselines is less obvious on datasets with similar feature descriptions, such as BeerAdvocate .

Revised: [Equation]

Revised: D. T HE IMPACT OF CLUSTER NUMBER

Revised: We show the the impact of different number of clusters K , i.e., the dimensionality of latent variable z n , on review rating prediction results in Table A3.

Revised: The best rating prediction results are obtained for

Revised: Digital Music and Office Products when the number of clusters is set to 128.

Revised: But on BeerAdvocate , the optimal cluster number is 256.

Revised: [Table]

Revised: AutoEncoder

Revised: WassasteinVAE

Revised: StandPrior

Revised: GIANT

Original:  As each dimension of the latent variables z n corresponds to a cluster, for the k -th dimension, we can search for its relevant reviews which have the highest value in z k and list the most frequent words in the review set as the representative topic words.
Revised: [Table] [Table] D. C LUSTER S EPARABILITY As each dimension of the latent variables z n corresponds to a cluster, for the k -th dimension, we can search for its relevant reviews which have the highest value in z k and list the most frequent words in the review set as the representative topic words.

Original: We identify ‘ ABBA ’ and their popular songs, computer peripherals such as ‘ network cable ’ and ‘ router ’ in Digital Music and Office Products datasets, respectively.
Revised: We identify ‘ ABBA ’ and their popular songs, computer peripherals such as ‘ network cable ’ and ‘ router ’ in Digital Music and Office Products datasets, respectively [Table] [Figure] [Table]

Revised: [Figure]

Original: [Table]
Revised: [Table]

Revised: We also compare with the generated clusters using StandPrior .

Revised: The results are shown in Table A5.

Revised: It is difficult to see a clear topic pattern in StandPrior as the top words largely overlap in different clusters.

Revised: The first several words are all ‘ hop ’, ‘ malt ’ and ‘ good ’.

Revised: The results are even worse than

Revised: WassersteinVAE , which can at least generate diverse clusters, reflected in the distinguished words in each clusters.

Revised: [Table]

Revised: [Table]

Revised: D. C LUSTER C OHERENCE

Revised: We explore the difference between our generated clusters z nk and the clusters derived in the graph.

Revised: To verify the capability of creating semantically coherent clusters , we propose to measure the cluster coherence as the average cosine similarity between every review document pair within a cluster.

Revised: We first obtain the document-level review representations by feeding the reviews from the test set to our pre-trained CNN encoder.

Revised: We then calculate the average cosine similarity between the representations of each review pair in a cluster, and are further averaged across all clusters (in Table A6).

Revised: E H UMAN E VALUATION FOR I NTERPRETABLITY

Revised: We conduct human evaluation to validate the interpretablity of our proposed method.

Revised: To make it easier for humans to understand the rationales behind model decisions, we extract the most relevant sentences from user/item reviews as interpretation for a specific user-item pair (user u and item i ).

Revised: Example generated interpretations are illustrated in Figure 1 and Figure 4.

Revised: In particular, our GIANT model infers the latent topic for user u and item i , respectively.

Revised: Such a topic essentially indicates which cluster user u or item i belongs to.

Revised: We can then identify the user candidate reviews as the past reviews on item i written by the users (including u ) in the same cluster as u .

Revised: Similarly, we identify item candidate reviews from the past reviews on the item i which have their most prominent topic the same as the item latent topic.

Revised: Afterwards, we represent each topic by its top-associated 5 words and derive the topic representation by the aggregated word-level GloVe word embeddings.

Revised: 14 These words are selected based on the

Revised: TFIDF scores of all words of reviews in the same topic cluster, with stop words filtered.

Revised: The review sentence representations are also derived based on the aggregated constituent word GloVe embeddings.

Revised: The most relevant sentences from user candidate reviews can then be extracted as a summary of the user u ’s preferences based on their cosine similarity with the user latent topic representation.

Revised: Similarly, the most relevant sentences from item candidate reviews are extracted as a summary of item i ’s characteristics based on their cosine similarity with the item latent topic representation.

Revised: Apart from the user u ’s preferences and item i ’s characteristics, we also present to human evaluators the model’s recommendation suggestion as will recommend , if the model’s predicted rating score is above the average predicted rating; or won’t recommend , otherwise.

Revised: We propose three evaluation metrics, relevance , faithfulness and informativeness and ask three

Revised: English-proficient human evaluators to give 1-5 score to the generated interpretations from WassersteinVAE and GIANT on 120 randomly selected user-item pairs from the Office Products 15 .

Revised: • Relevance .

Revised: If the extracted sentences from user reviews and item reviews are about the relevant topic/aspect/subject?

Revised: A higher score should be given to the interpretation with more overlapping aspects.

Revised: • Faithfulness .

Revised: Do the extracted sentences from user reviews and item reviews lead to the model’s recommendation suggestions?

Revised: • Informativeness .

Revised: Do the given interpretations capture the user preferences and item characteristics well?

Revised: [Table]

Revised: From the results shown in Table A7, we can observe that interpretations generated by our model are better compared to WassersteinVAE across all three aspects.