Enhancing Explainable Rating Prediction through Annotated Macro Concepts (2024)

Abstract: Generating recommendation reasons for recommendation results is a long-standing problem because it is challenging to explain the underlying reasons for recommending an item based on user and item IDs. Existing models usually learn semantic embeddings for each user and item, and generate the reasons according to the embeddings of the user-item pair. However, user and item IDs do not carry inherent semantic meaning, thus the limited number of reviews cannot model users’ preferences and item characteristics effectively, negatively affecting the model generalization for unseen user-item pairs. To tackle the problem, we propose the Concept Enhanced Explainable Recommendation framework (CEER), which utilizes macro concepts as the intermediary to bridge the gap between the user/item embeddings and the recommendation reasons. Specifically, we maximize the information bottleneck to extract macro concepts from user-item reviews. Then, for recommended user-item pairs, we jointly train the concept embeddings with the user and item embeddings, and generate the explanation according to the concepts. Extensive experiments on three datasets verify the superiority of our CEER model.

heading: 1 Introduction
text: In recommender systems, rating prediction is a crucial component for filtering user interest items from a vast pool of candidates. However, many recommender systems only offer rating prediction results without providing any explanations (Pan et al., 2022; Zhang et al., 2022). Personalized and appropriate explanations can greatly arouse user interest and enhance the overall user experience. Therefore, there is potential to deliver accurate rating predictions while generating personalized and suitable reasons for recommended items (Balog et al., 2019; Wang et al., 2022a).
∗Xiao Huang is the corresponding author.
Current explainable rating prediction models typically involve learning user and item embeddings and then generating explanations based on these embeddings (Chen et al., 2019, 2020b). For instance, they attribute semantic meanings to each user and item ID by reconstructing the associated review sentence using user and item embeddings (Wang et al., 2023; Gao et al., 2019). CETP (Li et al., 2021a) goes a step further by incorporating knowledge graphs as auxiliary information and leveraging graph neural networks to enhance the item embeddings. Similarly, MMCT utilizes multi-modal information to aid in training the user and item embeddings (Liu et al., 2023).
However, existing models still highly rely on the embeddings of user and item. Nevertheless, training user and item embeddings of explainable rate prediction face the following limitations: (i). training sparsity: The observed user-item pairs are sparse in comparison to all possible combinations. This sparsity issue prevents users and item embeddings from capturing sufficient semantic information by solely reconstructing the available reviews, thus negatively affecting the model’s generalization in generating explanations for unseen user-item pairs. (ii). Inference sparsity: The absence of available information in the inference stage makes it challenging to generate appropriate explanations. There are some efforts that explicitly put the collected item features as prompting information to alleviate this issue (Li et al., 2020b; Cheng et al., 2023). However, it is hard to predict in advance which specific features will appear in the explanations.
It is challenging to solve the sparsity problems due to two reasons. (i) ID insufficient semantics: The initial embeddings for users and items lack semantic meanings, and are not effectively recognized by language models. The insufficient initial semantics exacerbate the learning of a detailed and accurate mapping to the semantic space.
11736
(ii) Mixed high-level semantics: The user reviews are complicated and may involve multiple highlevel semantics. For example, a review that “I hate this guitar since the quality of the guitar is bad." could be identified by three high-level semantics: “negative emotion”, “instrument”, and “quality”. It is difficult to assist language models in recognizing user focus by providing an item feature.
To this end, we propose a framework named Concept Enhanced Explanation Recommendation (CEER). Our framework explicitly annotates highlevel semantics, i.e., macro concepts that abstract item characteristics with similar semantics in reviews and embeds the potentially matched macro concepts into the user and item embeddings. The mined macro concepts enrich the semantic meaning of user and item embeddings without requiring additional data to learn. Our key contributions are summarized below:
• To improve the explanation generation, we leverage the macro concepts to address the issue of inadequate semantic information in user and item embeddings.
• To build macro concepts, we discover the micro characteristics of the item attended by the user from the reviews by maximizing the information bottleneck.
• We devise three tasks to enrich user and item embeddings motivated by integrating the macro concept into representations space.
• Extensive experiments on three real-world datasets prove the superiority of the proposed framework CEER.

heading: 2 Preliminary
text: Problem Formulation. We denote the user set and item set as U = {u1, u2, ..., u|U|} and V = {v1, v2, ..., v|V|}, respectively. The rating score rij assigned towards the interaction (ui, vj) characterizes the degree of user preference. The observed reviews are organized within a matrix X, where each element in the matrix represents a review and Xij represents the review for the rating rij . Xij is defined as a sentence denoted as [x1..., xm, ..., xM ], where xm is the mth word in Xij . During inference, the available information for each interaction is limited to the user and item ID. In this work,
we define explanation generation as a sequence-tosequence prediction task. Specifically, given a useritem pair, we apply the user and item embeddings as soft prompts, asking the transformer to generate explanations, where each word in the explanation is predicted given the preceding words and soft prompt. For example, given the soft prompt [ui, vj ], the transformer is trained to predict the next word x1. After multiple generation steps, the output from the transformer forms the sequence [x1..., xm, ..., xM ]. The notations are put into Appendix A.1

heading: 3 Methodology
text: This section introduces two primary components of our framework: a macro concept annotator and an explanation generator. The first component involves two steps: identifying the informative micro characteristics of items in the reviews and using LLMs to annotate macro concepts. The second component utilizes three tasks based on the annotated concepts to enhance user and item embeddings and facilitate explanation generation. The architecture sketch is provided in Figure 1.

heading: 3.1 Macro Concept Annotation
text: Micro Characteristics Identification. We represent each macro concept as a group of semantically similar micro characteristics of items that are carefully curated from the reviews. To serve as the micro characteristics, the selected words are required to be informative and capable of justifying the rationale behind recommendations.
To measure the informativeness of the words, we need to first quantify the informative level of the words in the corpus. Pre-trained language models (PLMs) are trained on vast amounts of text data and can recognize which words are more informative in conveying the explanation. To learn the informative level quantification, we maximize the following objective, i.e., information bottleneck (TISHBY, 2000) M(Xij , rij):
max M(Xij , rij) = max I(Tij , rij)−τ ·I(Tij ,Xij), (1) where I(·, ·) represents mutual information between two variables; τ is a trade-off hyperparameter; Tij ∈ RM×d signifies the intermediate word representation from PLMs whose informative levels can be measured. Here, d denotes the word dimension in the frozen PLM, which, for instance, is 768 when using BERT (Devlin et al.,
2019) as the frozen PLM. In Eq. (1), the first term maximally preserves the information related to rating prediction while the second term (Alemi et al., 2016) serves as the regularization term that constrains the amount of information in intermediate representations encoded from Xij .
The variational approximation of the information bottleneck of traditional approaches applies restrictive constraints to the pre-trained language models as the second term (Tu and Li, 2022). The posterior distribution of the intermediate representations may not be standard Gaussian distribution (Tu et al., 2022). Some recent efforts have tried more flexible distributions, e.g., sample-based representations of variational distributions (Fang et al., 2019, 2022). They could learn expressive intermediate representations but are unable to explicitly quantify the informative level of the words. To achieve this goal, we inject noise into word representations (Guan et al., 2019). The extent to which these representations tolerate noise can serve as a reflection of their informativeness, as informative words play an important role in understanding the corpus and are sensitive to the introduced noise.
To facilitate the second-term calculation, we synthesize the word representations with noise as Tij :
Tij = (1− σ)⊙ PLM(Xij) + σ ⊙ ϵ, (2)
where PLM(·) provides the original word representations from PLMs, and ϵ ∈ RM×d denotes the random noise independently sampled from standard Gaussian distribution while the learnable vector σ ∈ RM[0,1] controls the magnitude of noise. The minimization of I(Tij ,Xij) is calculated as:
min H(Xij)−H(Xij |Tij) ≈ max log σ, (3)
where we assume P (Xij |Tij) ≈ P (Tij) and follows the multivariate Gaussian distribution with zero covariance. Next we could use 1 − σxm to identify the informative level of word xm in Xij . Then the informative level for each word in Xij is:
[1− σx1 , ..., 1− σxm , ..., 1− σxM ] . (4)
To calculate the first term, we transform word representations into ratings and minimize the difference between predictions and labels. In this process, we employ a rating-specific message-passing encoder for transformation (Shuai et al., 2022).
After obtaining σ, we exploit the LLMs to perform the macro concept annotation for top k important micro characteristics. Here we use LLM ChatGPT, i.e., gpt-3.5-turbo and take one demonstration following (Lou et al., 2023; Zhang et al., 2023) as an example:
[Task Description] User: Organize the following keywords into groups based on their semantic relationships, and give a concept name to each group. [Demonstrations] Input: [Expensive, Cheap]. Output: {Economy: [Expensive, Cheap]} [Test Instance] Input: [Guitar, Hate, Quality]. Output: {Instrument: [Guitar], Negative Emotion: [Hate], Quality: [Quality]}.
In this way, we obtain the macro concept labels for k informative micro characteristics from the historical reviews and create a padded concept label for the remaining words. The words in Xij are labeled as {cx1 , ..., cxm , ..., cxM }, where cxm is the macro concept ID for word xm.

heading: 3.2 Explanation Generator
text: In this component, our goal is to utilize the annotated macro concepts to enrich user/item embeddings. Specifically, we embed the relationships between users/items and concepts, as well as concepts and reviews, in representations through three tasks. These three tasks collaborate to utilize macro concepts as an intermediary to align user and item embeddings with the review embeddings.

heading: 3.2.1 User/item-concept Relationship Modeling
text: The first task user/item-concept relationship modeling utilizes user and item embeddings to aid in predicting macro concepts in the associated review. To train the prediction of macro concept distribution, we need to craft macro concept labels Wij = [ w1, ..., wq, ..., w|C| ] ∈ R|C| for review Xij . The macro concept label wq is determined by the cumulative informative level of the words in Xij associated with that concept cq:
M∑
m=1
I(cxm = cq)(1− σxm), (5)
where the indicator I(·) take value 1 if the condition holds otherwise 0.
After obtaining the macro concept labels for review Xij , we come to train the user/item embeddings to predict the macro concept distribution Ŵij in the associated review, defined as:
Ŵij = MLP (eui , evj ), (6)
where eui , evj denotes the user and item embeddings respectively. Next, we use Kullback-Leibler divergence loss to penalize the divergence between the predicted macro concept distribution and the labels, defined as:
Lp = LKL(Ŵij ||Wij) = |C|∑
q=1
wq · log wq ŵq . (7)
After training, the user and item embeddings could be utilized to recognize the potential macro concepts in the inference. Then, we obtain a composite concept embedding by the weighted sum of all macro concept embeddings with the predicted importance: ∑|C| q=1 ŵqecq/ ∑|C| q=1 ŵq. The composite macro concept embedding is appended behind the user and item embedding to instruct the transformer to generate relevant explanations.

heading: 3.2.2 Concept-review Relationship Modeling
text: The second task concept-review relationship modeling brings the words in reviews and associated macro concept embeddings closer together. Through this task, we proceed to enhance the words in reviews’ awareness of the associated macro concept within the representation space. To reach this objective, we reduce the uncertainty about each word mapping to its associated macro concept. For simplicity, we use the inner product to compute the uncertainty and optimize it with the cross-entropy loss, driving the movement of word embeddings to the macro concept embedding:
Lm = − M∑
m=1
|C|∑
q=1
I(cxm = cq) · log(pxm,cq), (8)
where log(pxm,cq) is the uncertainty between the word xm and the macro concept cq, computed by inner product of embedding: pxm,cq = e ⊤ xm · ecq . It encourages words and associated concepts to exhibit proximate representations.

heading: 3.2.3 User/item-review Relationship Modeling
text: The third task user/item-review relationship modeling assists the transformer in considering words belonging to the same macro concepts when generating the next word. Traditional approaches employ the maximum likelihood function to reconstruct the original reviews. However, it ignores the diversity of the language expression. In practice, each position in the sentence could have multiple word choices, e.g., semantically similar words. To achieve this, we substitute the word in the review with multiple alternatives in the same macro concept. The loss can be reformulated as:
Ls = − 1
M
M∑
m=1
(log p(xm) + α log p(x ′ m)), (9)
where x′m denotes a randomly selected word within the same macro concept as xm and α denotes a hyper-parameter to control the loss term. Here, we only incorporate one substitution. More substitutions may take the risk of introducing noise and generating contextually inappropriate explanations.

heading: 3.2.4 Training Objective
text: Besides the above tasks, we jointly perform rating prediction following previous efforts (Li et al., 2023b) using the loss function defined as:
Lr = ( MLP (eui , evj )− rij )2 . (10)
Finally, we unify these tasks in a linear combination form to train the transformer. The joint learning objective is indicated by:
L = Ls + βLm + γLp + Lr, (11)
where β, γ are trade-off hyper-parameters to balance the effect of different objectives of the model.

heading: 4 Experiments
text: In this section, we perform comprehensive experiments to validate the effectiveness of CEER and gain insights into its behavior. We aim to answer the following research questions: RQ1: How well do the explanations and ratings generated by CEER and baselines align with the actual labels in terms of text quality? RQ2: How does the quality of explanations generated by all models in terms of interpretability? RQ3: How do different tasks contribute to the whole model performance? RQ4: What impact do different hyper-parameter settings of the proposed tasks have on CEER? RQ5: How does information bottleneck perform compared with other micro characteristics identification methods for macro concept annotation?

heading: 4.1 Dataset Processing
text: In our experiments, we use three real-world Amazon datasets (He and McAuley, 2016), i.e., Musical Instruments, Home and Kitchen, and Automotive, to evaluate the overall performance. The dataset statistics are shown in Table 1. The detailed description of these datasets is illustrated in Appendix A.2.

heading: 4.2 Experimental Setup
text: We implement our framework by PyTorch using NLG4RS ecosystem (Li et al., 2020a) and run it on NVIDIA Tesla V100S PCIe, 32GB memory. The learning rate is set as 0.001. All models are optimized by the Adam optimizer and the embedding dimension is set as 64 for a fair comparison. The batch size equals 256. We truncate explanations with a maximum length of 20 and the budget k is set as 5000 for each dataset. We search the number of transformer layers and attention heads in each layer for the base explanation generator from {1, 2, 3, 4, 5}. And we also search the
trade-off factor α, β, γ from {1e− 5, 1e− 4, 1e− 3, 1e − 2, 1e − 1, 1} and the dropout rate from {0, 0.2, 0.4, 0.6, 0.8}. The search strategy is grid search. We also set the patience threshold on the validation set to stop model training earlier and the optimal performance on the validation set is selected to report performance on the test set. We run each model five times and report the average performance.

heading: 4.3 Evaluation Metrics
text: To evaluate the explanations, we categorize all the adopted metrics into two groups, which evaluate the text quality and the quality of interpretation, respectively. In the first text quality evaluation group, we have BLEU-1, BLEU-4, MAE, and RMSE. In the second interpretability evaluation group, we have USR, FCR, FMR, Entail, and Consistency. The detailed introduction of these metrics is illustrated in Appendix A.3.

heading: 4.4 Baseline Methods
text: We integrate the following baselines to conduct the comparison: RGCL (Shuai et al., 2022) applies a graph contrastive learning framework to learn useritem rating prediction; NRT (Li et al., 2017) learns explanation generation via gated recurrent units; NETE (Li et al., 2020b) generates explanations with controlled neural templates; PETER (Li et al., 2021b) introduces contextualization loss and item features to enhance the quality of the explanation; SAER (Yang et al., 2022) adopts an extract-andrefine architecture to effectively generate explanations; PEPLER (Li et al., 2022) fuses user/item IDs into soft prompt and fine-tunes GPT-2.
To demonstrate the effectiveness of different components, we also introduce three variants of the CEER: 1) P-CEER: it removes all the proposed tasks and adopts the pure transformer to generate explanations. 2) C-CEER: it adds the conceptreview relationship modeling task to the P-CEER. 3) CW-CEER: it further adds the user/item-review relationship modeling task to the C-CEER.
To examine the usefulness of the information bottleneck in micro characteristics identification, we incorporate the following variants: (1) GPT3.5: Directly annotate the macro concepts in the historical reviews with ChatGPT and remove the information bottleneck. (2) β-VAE: Replace the information bottleneck with the β-VAE (Higgins et al., 2017) and use the learned variance as the informative level.
NETE (2020) 0.9400 0.7139 1.1207 0.8691 1.0261 0.7740 SAER (2022) 1.0970 0.9156 1.1731 0.9409 1.0429 0.8131
CEER 0.9210 0.6680 1.0789 0.7490 0.9693 0.6771

heading: 4.5 Text Quality Analysis (RQ1)

heading: 4.5.1 Word-level Relevance
text: First, we evaluate the text quality of explanations in terms of BLEU-1 and BLEU-4 and exhibit the results in Table 2. We have several observations.
1) The performance of baselines does not consistently improve with increasing complexity across the three datasets. The NRT model is simple and designed for generating abstractive tips, while the PETER, NETE, and SAER have more complicated architectures. Surprisingly, we find that the complex models do not consistently outperform the simpler NRT model in all cases. Sparse data during inference and training hinders the effective learning of user and item embeddings and limits the ability of advanced language models to generate accurate explanations. Although PEPLER, which fine-tunes GPT-2, fails to yield satisfactory results. This may be because they do not have harvest knowledge about ID during pre-training. 2) The proposed CEER outperforms all baselines across the three datasets. Compared to the baseline methods, the CEER framework consistently produces better results. We infer that this improve-
ment is due to the inclusion of macro concepts, which offer higher-level semantics that enhance the learning of ID embeddings and provide additional information for generating explanations.

heading: 4.5.2 Rating Prediction
text: We also examine the recommendation performance and present the results in Table 3. We find that the proposed CEER performs consistently better than the explainable recommendation baselines. And it performs even better than the dedicated rating prediction baseline RGCL in most cases. We infer that incorporating the high-level semantics, i.e., macro concepts refines the user and item representations, leading to improved recommendation performance. As a result, our model exhibits strong potential for real-world applications, as it not only offers explanations with high text quality but also delivers accurate rating predictions, addressing the specific requirements of the system.

heading: 4.6 Interpretability Quality Analysis (RQ2)

heading: 4.6.1 Personalization
text: We also evaluate the degree of informativeness and diversity, i.e., personalization of explanations, and report the results in Figure 2. Notably, in Subsection 4.6, we only select the explainable baselines with superior performance for the RQ1 (i.e., PEPLER, PETER, NETE, and NRT) for comparison. From the results, we have several observations. First, the higher diversity of explanations does not indicate that the features are accurately
included in each explanation. It indicates that improving informativeness and diversity simultaneously is a challenging task. Second, the proposed model generally outperforms the selected baselines in most cases, especially on the Instrument and Automotive datasets. It indicates that the proposed model can not only generate more diverse explanations but also match the corresponding features for the explanations.

heading: 4.6.2 Entail & Consistency
text: We evaluate the interpretability from the sentiment perspective and display the results in Figure 3. We notice that the CEER achieves relatively better performance in producing emotionally coherent and factually accurate explanations in most cases. During training, words become associated with the relevant concepts they belong to. This alignment between positive or negative concepts and the corresponding words enables CEER to consistently generate emotionally coherent explanations. Additionally, user and item representations tend to align with the representations of related concepts. It facilitates the incorporation of related facts into the explanation generation process.

heading: 4.7 Ablation Study (RQ3)
text: To explore the individual contributions of different tasks to the overall model performance, we analyze three variants and report the results in Table 4. Notably, in Subsections 4.7 and 4.8, we select a subset of metrics for evaluating text quality and interpretability rather than including all available metrics to save layout space. We have several observations: 1) The P-CEER has the worst performance. It does not include any of the designed tasks. The bad performance underscores the necessity of integrating the designated task to enhance model performance. 2) By taking each designed task gradually, CEER, CWCEER, and C-CEER demonstrate performance improvements compared with its predecessor CW-CEER, and C-CEER, P-CEER correspondingly. It demonstrates the effectiveness of each task in addressing the issue of limited generalization caused by sparsity. The overall results validate the effectiveness of the proposed model.

heading: 4.8 Hyper-parameter Sensitivity (RQ4)
text: We further analyze the hyper-parameters α, β, γ in the proposed regularization terms. We conduct experiments on the Automotive dataset and present
the results in Figure 4. We have several observations. 1) Across all selected metrics, the model is sensitive to extreme hyper-parameter values. Hyper-parameters that are too large, such as 1, or too small, such as 1e − 5, do not lead to the best performance of the model. 2) Different evaluation metrics require different hyper-parameters to reach their optimal values. This observation implies that it is difficult to have a hyper-parameter setting that could achieve optimal results considering multiple evaluation metrics together for the whole model. 3) Compared with β and γ, α generally prefers relatively small values. We infer that the user/item-review relationship modeling task may act as noise and hinder the modeling of the language pattern in the explanations when α is large. All the results imply that the CEER is sensitive to the hyper-parameter values and it is important to choose appropriate values.

heading: 4.9 Characteristics Identification Study (RQ5)
text: In this subsection, we first conduct the comparison between different micro characteristics identification methods and display the results in Table 5. Then we present an example from the Amazon Instrument dataset to intuitively show the identified micro characteristics, annotated macro concept, and the generated explanation in Figure 5. Performance Comparison. We compare the information bottleneck method with two other choices using the same annotation prompt, and explanation generator. The two choices are compared on the Amazon Instrument dataset and the Amazon Home dataset, respectively. From the results, we find that direct annotation without information bottleneck negatively affects the model performance. We infer that although LLMs are proficient at text comprehension, they are not trained to assign consistent macro concepts to similar words, which leads to poor performance. And the β-VAE also performs worse than the information bottleneck in most cases. β-VAE does not quantitatively define the importance of the words. And the learned variance may not be a good indicator to differentiate word importance although the small variance may mean that the meaning of the word does not change a lot across contexts, such as stop words. These results jointly demonstrate the effectiveness of the information bottleneck method. Case Study. We showcase the informative level of each word in the historical reviews associated with
the given user-item pair. Then, we demonstrate the annotated concepts that drive the model to generate the explanation involved by the user and item in this example. First, we visualize the informative level of each word representation of two historical reviews associated with user 543 and item 504, respectively. We observe that those keywords that explain the reason for the rating are assigned significantly lower amounts of noise. For example, in the first review of subfigure (a), the words “costly” and “price” describe the reason for user dissatisfaction and their informative levels are very high. It demonstrates that the first component could select the informative words. That facilitates the cost reduction for LLMs to annotate concepts.
Second, we compare the explanations generated
by the CEER and the P-CEER, i.e., the transformer model for the interaction between user 543 and item 504, and show it in subfigure (b). The target rating is 5. We observe that the P-CEER only describes the user’s positive sentiment towards the item and does not contain any useful information. The generated explanation from CEER reveals the reason for the high rating is due to the price and the positive quality. We also exhibit the top two prioritized concepts and words associated with them in the subfigure (c). It shows to a certain extent that the high-level concept does drive the transformer to generate semantically relevant sentences. And the associated words for the concept demonstrate that the LLMs could appropriately assign a concept label to semantic similar words.

heading: 5 Related Work

heading: 5.1 Sentence-based Explanation Generation
text: Compared with examining which feature plays a more important role in driving the interaction (Wang et al., 2018a, 2022b), sentence-based explanations for rating prediction offer natural language explainable information about why the user assigns particular ratings (Hua et al., 2023; Wu et al., 2024). Some efforts only manipulate the historical reviews to generate the explanations (Le and
Lauw, 2020; Pugoy and Kao, 2021), other efforts attempt to integrate varied auxiliary information into explanation generation. Knowledge graph provides an organized way to access facts (Chen et al., 2024; Shengyuan et al., 2024). CETP (Li et al., 2021a) selects the triples from the knowledge graph and incorporates them into explanation generation. Further, METER (Geng et al., 2022; Liu et al., 2023) incorporates multi-modality information to encourage the model to write more faithful and diverse explanations. LLMs have strong text manipulation abilities (Dong et al., 2024; Zhang et al., 2024). PRAG (Xie et al., 2023) leverages LLMs and reformulates the explanation generation as a questionanswering task. Additionally, the tips written by users show how they feel about the interaction and can also be utilized to generate a comprehensive explanation (Zhu et al., 2023). To generate coherent explanations, AESG tries to incorporate the syntax graph dependency tree to improve the quality of explanations (Hu et al., 2023).

heading: 5.2 Review-based Rating Prediction
text: Review-based rating prediction involves predicting a numerical rating for a user-item pair by modeling the text of a review provided by a user (Harrag et al., 2019). Early effort introduces the reviewlevel attention mechanism to model the review for rating prediction (Chen et al., 2018). It ignores the fine-grained information in the review. NPA (Wu et al., 2019) leverages user ID to incorporate wordlevel and review-level information to improve performance attentively. Further, DAML (Liu et al., 2019) exploits the mutual information between the user and the item extracted from information that the convolutional neural network attends to. Similarly, CARP (Li et al., 2019) models mutual information and uses the capsule network to extract high-level information. EDMF (Liu et al., 2022) extracts the useful feature from the review text to further improve performance. Graph neural networks (GNNs) capture global relationships between nodes that are indirectly connected (Dong et al., 2023; Chen et al., 2020a). Efforts adopt GNNs to model user and item relationships and the review features can be regarded as edge features (Qiao et al., 2022; Shuai et al., 2022).

heading: 6 Conclusion and Future Work
text: In this paper, we present the CEER framework, designed to enrich user and item embeddings with
high-level semantics, i.e., macro concepts to improve the explanation generation. The framework identifies micro characteristics of items from associated reviews and utilizes LLMs to annotate macro concepts. These macro concepts act as intermediaries to align the user and item embeddings with review embeddings to obtain more semantic information. And we achieve this through three tasks that focus on establishing relationships between users/items, macro concepts, and reviews. We conduct extensive experiments, and the results confirm the effectiveness of CEER. Our future work will involve adapting our method to annotate concepts from various data sources, such as images, and enhancing the explanation generation process (Li et al., 2023a).

heading: Acknowledgments
text: The work described in this paper was fully supported by a grant from the Research Grants Council of the Hong Kong Special Administrative Region, China (Project No. PolyU 25208322).
Limitations
In this study, we annotate the macro concept from the reviews to assist the explanation generation. The existing framework is constrained to annotating macro concepts solely from text and lacks the capability to expand to additional data sources like images. And we annotate macro concepts from a series of micro item characteristics, i.e., words. However, the semantic meaning of words differs under different contexts. Moreover, we employ LLMs for annotating macro concepts and training a small language model for explanation generation instead of fine-tuning LLMs. While fine-tuned GPT-2 may outperform the suggested framework when available reviews are rich, our framework demonstrates relatively better effectiveness in situations with extremely sparse reviews and does not incur significant computational costs.
Ethics Statement
In this study, all the Amazon datasets are publicly available and have been extensively used in research related to recommendation systems. All the baseline codes are open-sourced. And we adhere to the ACL Code of Ethics1.
1https://www.acm.org/code-of-ethics

heading: A Appendix
text: A.1 Notations In this work, matrices are symbolized by uppercase bold letters (e.g., E), vectors are represented by lowercase bold letters (e.g., e), and scalars are represented by lowercase letters (e.g., rij). We denote sets with calligraphic letters (e.g., U) and denote their size using cardinality (e.g., |U|). The ith row of matrix E is denoted as ei, whereas the element locating in the ith row and jth column is denoted as Eij .
A.2 Dataset Details These datasets consist of customer reviews and ratings specifically related to items. We regard the sentence in the review that can explain the purchase
motivation as the explanation following the previous practice (Li et al., 2020b; Ni et al., 2019). We extract explanations and item features (Yang et al., 2021) from the given reviews using ChatGPT. Both of them will be included in the evaluation metrics, which will be introduced later. To ensure that users and items under evaluation receive training, we reserve users and items that have at least one record in the training set (Li et al., 2021b). The interactions are initially sorted based on their timestamps and then divided into training, validation, and test sets with a proportion of 8 : 1 : 1. The rating score range in three datasets based on the sentiment from negative to positive is {1, 2, 3, 4, 5}.
A.3 Evaluation Metrics
Here, the symbol ↑ followed by the metric means that the higher the score, the better the performance, while ↓ represents the lower the score, the better the performance. In the first group, we view the explanations as purely plain text and evaluate the text quality. (i) We adopt the widely used BLEU1↑ and BLEU-4↑ metrics to measure the wordlevel relevance of the generated explanations to the ground-truth explanations. (ii) User and item serve as the soft prompt to generate explanations, with their representations containing textual information. We adopt two classic metrics, MAE↓ and RMSE↓ to evaluate the predicted ratings using user and item representations with Eq. (10) (Li et al., 2017, 2020b, 2021b).
In the second group, we examine the interpretability quality of generated explanations from two aspects. (i) To measure the degree of personalization, we adopt Unique Sentence Ratio (USR)↑, Feature Coverage Ratio (FCR)↑, and Feature Matching Ratio (FMR)↑ by following previous efforts (Li et al., 2020b, 2021b). Specifically, USR assesses the diversity of generated interpretations, while FCR and FMR measure the diversity and accuracy of generated features. (ii) To examine the semantic-level relevance, we adopt Entail↑ (Xie et al., 2023) to estimate the extent of whether the generated explanation entails the label explanation. To measure the emotional consistency between the predicted rating and the generated explanations, we adopt Consistency↑ (Wang et al., 2018b).
Consistent with prior practices (Xie et al., 2023), we employ a pre-trained entailment model to assess whether the generated explanations entail the ground truths. Similarly, following previous
work (Wang et al., 2018b), we employ a sentiment classification model to estimate sentiment scores based on the generated explanations and calculate the correlation with the predicted rating.

heading: Future Work
text: Our future work will involve adapting our method to annotate concepts from various data sources, such as images, and enhancing the explanation generation process (Li et al. , 2023a)