RDRec: Rationale Distillation for LLM-based Recommendation (2024)

Abstract: Large language model (LLM)-based recommender models that bridge users and items through textual prompts for effective semantic reasoning have gained considerable attention. However, few methods consider the underlying rationales behind interactions, such as user preferences and item attributes, limiting the reasoning capability of LLMs for recommendations. This paper proposes a rationale distillation recommender (RDRec), a compact model designed to learn rationales generated by a larger language model (LM). By leveraging rationales from reviews related to users and items, RDRec remarkably specifies their profiles for recommendations. Experiments show that RDRec achieves state-of-the-art (SOTA) performance in both top-N and sequential recommendations. Our source code is released at https://github.com/WangXFng/RDRec.

heading: 1 Introduction
text: Large language models (LLMs) with powerful reasoning capabilities have been extensively studied for recommendations, including news and item recommendations (Li et al., 2022; Wei et al., 2023; Huang et al., 2023), explainable recommendations (Yang et al., 2023; Cheng et al., 2023), and zero-/few-shot and cold-start recommendations (He et al., 2023; Sanner et al., 2023). Several attempts have leveraged knowledge of LLMs to improve recommendation performance, such as enhancing embedding initialization (Harte et al., 2023), reranking candidates (Yue et al., 2023), and learning representation (Ren et al., 2023; Lin et al., 2023; Lei et al., 2023; Viswanathan et al., 2023). A straightforward approach is to integrate user and item IDs into LMs through prompt learning (Liu et al., 2023), including discrete prompts to find alternative words to represent IDs, continuous prompts to directly feed ID vectors into a pre-trained model (Sun et al., 2019), and hybrid prompts (Li et al., 2023a; Zhang and Wang, 2023). Recently, Geng et al. (2022)
present a P5 paradigm to transform user–item interactions, user sequential behaviors, and reviews into text-to-text prompts for LLMs. This enables P5 to capture deeper semantics for LLM-based recommendations. Li et al. (2023b) enhance P5 by a prompt distillation, resulting in significant improvement and reductions in inference time.
However, they pay no attention to mining the rationale behind each interaction, such as user preferences and item attributes, which hampers the reasoning capabilities of LLMs. As an example, in Fig. 1 (a), a user review for an item says: “It was pretty fun and cool since we had to change our strategy (user preference) to try to prevent her from playing intrigue cards (item attributes).” The user prefers strategic thinking in the game, and intrigue cards symbolize item characteristics. This introduces noise into the user’s profile, as the user leans towards a strategic game rather than merely cards. This suggests that the original review without intermediate prompts prevents the model from learning to understand the rationale behind the interaction.
The Chain-of-Thought (CoT) prompting (Wei et al., 2022; Wang et al., 2023a) that promises
65
LLMs to decompose intermediate rationales, has been widely applied for rationale extraction (Wang et al., 2023b; Zhang et al., 2023b; McKee et al., 2023; Zhu et al., 2023). More recently, Hsieh et al. (2023) utilize the CoT prompting to distill rationales via LLMs to train smaller models. Inspired by this, we propose a compact recommender model to learn the interaction rationales, i.e., user preferences and item attributes, distilled from reviews using a larger LM. In this way, the model acquires clear textual knowledge with less noise (e.g., “intrigue cards” that may hinder understanding the user’s preference for “strategic games” in Fig. 1 (b)). This enables the model to derive more specified user and item profiles from all reviews given by the user or regarding the item for recommendations, as illustrated in Fig. 1 (cd).
The main contributions of this paper can be summarized as follows. (1) We propose a compact RDRec model that effectively specifies user and item profiles by distilling interaction rationales from relevant reviews using a larger LM, and (2) RDRec consistently outperforms SOTA baselines on three real-world datasets in both sequential and top-N recommendations.

heading: 2 RDRec Framework
text: We present an RDRec model consisting of two stages, an interaction rationale distillation and a rationale-aware recommendation.

heading: 2.1 Interaction Rationale Distillation
text: Inspired by the recent works (Hsieh et al., 2023; Miao et al., 2023) that employ LLMs to produce training data for smaller models, we distill user preferences and item attributes from reviews by using the following prompt template: “A user bought an item and said ‘{review}’. Use two sentences to explain the user’s preference and the item’s at-
tributes, respectively.” As illustrated in Fig. 2, a review feeds into LLMs with the prompt template. The output is user preferences and item attributes.
Formally, given a user–item interaction triplet (u, i, ru,i) where u, i, and ru,i indicate a user, an item, and a review, respectively, we generate a quadruplet (u, i, pu,i, au,i) through rationale distillation. Here, pu,i and au,i refer to the distilled user preference and item attribute, respectively.

heading: 2.2 Rationale-aware Recommendation
text: The RDRec uses PrOmpt Distillation (POD) (Li et al., 2023b) as its backbone. POD converts three recommendation tasks into LLM-based text generation tasks, and then distills continuous prompt vectors from task templates. These tasks are (i) sequential recommendations, predicting the next item through the user’s ordered interactions, (ii) top-N recommendations, recommending the top N items not yet engaged with by the user, and (iii) explanation generation for a user’s interactions.
In contrast to POD, we incorporate an additional rationale generation task, consisting of a user preference generation and an item attribute generation. Specifically, following POD, we first distill prompt vectors (“<P4>” and “<P5>” in Fig. 3) from the templates of “Generate user_{#u}’s preference” and “Generate item_{#i}’s attribute”, where #u and #i denote the user and item IDs. Then, we concatenate prompt vectors with user and item IDs as the input, and the generated preference pu,i and attribute au,i as the output to train the model. To address the token composing issue (i.e., the token of “user_123” is often tokenized by LLMs as a sequence of [“user”, “_”, “12” and “3”]), we use the whole-word embedding (Geng et al., 2022) to treat each sequence of ID tokens as a complete unit, making it distinguishable as a word.
Fig. 3 illustrates the input and output example of the four tasks. We define a pair of input-output
words as X = [x1, ..., x|X|] and Y = [y1, ..., y|Y |], respectively. We then concatenate the tokens of the input with prompt vectors and obtain [x1, ..., x|X|, p1, ..., p|P |]. After adding the whole-word representation [w1, ..., w|X|+|P |], we feed them into the smaller model in RDRec to obtain a probability distribution p(y|Y<t, X) over a vocabulary at each step t, where Y<t denotes the tokens generated before step t. We adopt a log-likelihood loss function to optimize the model parameters Θ:
LΘ = 1 |D| ∑
(X,Y )∈D
1
|Y |
|Y |∑
t=1
− log p(y|Y<t, X),
(1) where D denotes the training set consisting of all input-output pairs for four tasks. |D| and |Y | denote the amount of training samples and the number of tokens in the output sequence, respectively.

heading: 2.3 Model Optimization and Inference
text: Following POD, we shuffle the input-output pairs of four tasks and randomly select samples from each task in a specified proportion. We thereafter mixed these samples to train the RDRec model. During inference, we employ a beam search algorithm to generate results by selecting the word with the highest likelihood from the vocabulary.

heading: 3 Experiment

heading: 3.1 Experimental Setup
text: Datasets and Metrics. Consistent with POD, we performed experiments on three public datasets, i.e., Sports & Outdoors, Beauty, and Toys & Games, which are collected from the Amazon dataset1. Each record in the dataset contains a user ID, an
1https://www.amazon.com/
item ID, a rating, a textual review, and a timestamp. We split each dataset into training, validation, and test sets with a ratio of 8:1:1. The statistics of datasets are provided in Table 1. To evaluate the recommendation performance, we utilized the evaluation metrics of hit rate (HR)@k (H@k) and normalized discounted cumulative gain (NDCG)@k (N@k) with k ∈ {1, 5, 10}. Baselines. We compared RDRec with ten baselines for sequential recommendations: CASER (Tang and Wang, 2018), HGN (Ma et al., 2019), GRU4Rec (Hidasi et al., 2015), BERT4Rec (Sun et al., 2019), FDSA (Zhang et al., 2019), SASRec (Kang and McAuley, 2018), S3-Rec (Zhou et al., 2020), P5 (Geng et al., 2022), RLS (Chu et al., 2023) and POD (Li et al., 2023b). We compared RDRec with five baselines for top-N recommendations: MF (Koren et al., 2009), MLP (Cheng et al., 2016), P5 (Geng et al., 2022), RLS (Chu et al., 2023) and POD (Li et al., 2023b).
Implementation. For a fair comparison, RDRec used T5-small (Raffel et al., 2020) as the smaller model, aligning with the baselines P5 and POD. We used Llama-2-7b (Touvron et al., 2023) as the larger LM. We reported a 10-trial T-test to show the robustness of RDRec. Our RDRec was implemented and experimented with Pytorch on Nvidia GeForce RTX 3090 (24GB memory). The Appendix A.1 provides further details.

heading: 3.2 Experimental Results
text: Tables 2 and 3 show comparative results between RDRec and baselines. We can see that the RDRec consistently surpasses the runner-ups, POD and RSL, with the improvement of 0.5 ∼ 9.8% in H@k and N@k for sequential recommendations, and 12.1 ∼ 42.2% in H@k and N@k for top-N recommendations, where k ∈ {1, 5, 10}. This highlights the effectiveness of learning interaction rationales to improve both recommendation tasks.
We also observed that RDRec exhibits greater improvement in top-N recommendations compared to sequential recommendations. This indicates that specifying user preferences and item attributes is more beneficial to recommending top-N unknown candidates, whereas sequential recommenders rely more on capturing correct behavioral patterns for predicting the user’s next choice.
We conducted an ablation experiment to examine the rationale distillation. The result in Table. 4 shows that distilling user preferences and item attributes from reviews is advantageous for both sequential and top-N recommendations. We can see that specifying item profiles is generally more effective for top-N recommendation, whereas specifying user profiles is more effective for sequential recommendation on the Sports and Beauty datasets.

heading: 3.3 Error Analysis of Sequential Recommendation
text: We conducted an error analysis to examine the sequential recommendations by RDRec. We identified two noteworthy error cases: Case (i). RDRec may prioritize the next item based on a user’s earlier interactions rather than recent ones. One reason is that the Transformer (Vaswani et al., 2017) in T5 excels in capturing long-term dependencies, while it may cause RDRec to pay less attention to recent interactions. This suggests to enhance its self-attention (Fan et al., 2022) or develop short-term prompt-aware templates for LLM-based sequential recommendations.
Case (ii). RDRec often disregards popular items for users because they do not align with their sequential patterns. One possible reason for this is that, during training, RDRec selects random subsequences from the user interaction sequence and predicts the last item of each subsequence. This process emphasizes sequential patterns but possibly sacrifices the model’s capability to identify popular items. This suggests introducing a popularitybased interaction graph to help the model be aware of popular high-order neighbors.

heading: 3.4 In-Depth Analysis of RDRec
text: To better understand the RDRec, we conducted indepth experiments and analysis. The Appendix A.2 provides further analyses. Effect of sample ratios. We observe from Table 5 that on the Toys dataset, increasing the ratio of topN samples for training RDRec improves sequential recommendations, while in major cases a higher ratio of sequential samples always harms top-N recommendations. One reason is that the training strategy of sequential tasks prioritizes sequential patterns while compromising its ability to detect unknown items. Computational complexity. Both Llama2 and T5 are Transformer-based models, with computational complexity of O(L2), where L is the number of word tokens. Consequently, RDRec’s computational complexity relies on user interaction count rather than the number of users and items. Compared with other complex ID-based methods, such as graph convolution network-based approaches with O((M + N)2) (He et al., 2020; Yu et al., 2022; Wang et al., 2023c, 2024), where M and N are the numbers of users and items, respectively, and (M+N ) ≫ L in Table 1, RDRec exhibits reduced computational demands, thereby rendering it suitable for deployment in large-scale applications.
Study of rationale distillation. We investigated the rationale distillation and obtained two findings. One is that, even when a user negatively reviews an item, the LLM objectively specifies user requirements and item attributes. For instance, in the following input, the customer advises not buying the book unless the kids are truly interested in it. However, many others provide positive comments, such as “The toy was really nice.” and “Fun little toy to match the book.”
Input: My Nephew is all about trucks and machines it’s cute for him but unless the kid’s really into the book or just general construction I wouldn’t bother.
This indicates that objective profiles (e.g., a book and its content) are more crucial than users’ subjective opinions in real-world recommendations. We found that the generated item attributes by the LLM are relatively objective which is shown as follows:
Output: The user prefers items that are cute and appealing to children, but not necessarily related to construction or machines.
The item’s attributes include being a colorful and engaging picture book that teaches children about different construction vehicles.
This could be a reason for the noticeable improvement in performance by learning rationales.
The other observation is that, when a review is extremely short, the prompt could urge the LLM to produce hallucinations during rationale distillations. Recently, Zhang et al. (2023a) have proposed to mitigate hallucinations of LLM-based recommender to enhance its performance. This is a rich space for further exploration (Liu et al., 2022; Gao et al., 2023; Peng et al., 2023).

heading: 4 Conclusion
text: We proposed a compact RDRec model to learn the underlying rationales for interactions generated by a larger LM. By learning rationales from all related reviews, RDRec effectively specifies user and item profiles for recommendations. Experimental results showed the effectiveness of our RDRec. Future work involves (i) exploring better prompts for sequential recommendations, and (ii) enhancing explanation generation in RDRec.

heading: Acknowledgements
text: We would like to thank anonymous reviewers for their thorough comments and suggestions. This work is supported by the China Scholarship Council (No.202208330093) and JKA (No.2023M-401).

heading: Ethics Statement
text: This paper does not involve the presentation of a new dataset, an NLP application, and the utilization of demographic or identity characteristics information.

heading: Limitation
text: The hallucination issue during rationale distillation remains unsolved. Additionally, RDRec faces an unfaithful reasoning problem, misinterpreting user opinions about candidate items despite delivering correct recommendations.

heading: A Appendix

heading: A.1 Experimental Details
text: This section provides further experimental results and implementation setup. We focus on the fourth task as Geng et al. (2022) have conducted a thorough study for the others.

heading: A.1.1 Effect of Various Sample Ratios
text: Table 6 shows the effect of various sample ratios on the recommendation performance. We can see that the sample ratio of various tasks for pretraining the model would influence the recommendation performance. Specifically, on the Toys dataset, increasing the ratio of top-N samples sometimes improves both sequential and top-N recommendations. In contrast, a higher ratio of sequential samples often negatively affects the performance of top-N recommendations across all datasets. The reason is that during training, RDRec selects random user interaction subsequences and predicts the last item of each subsequence. This process emphasizes sequential patterns, although possibly sacrifices the model’s capability to identify popular items.

heading: A.1.2 Execution Time
text: Table 7 shows the execution time in various stages by RDRec on three datasets. These results were obtained through Nvidia GeForce RTX 3090 (24GB memory). We can see that RDRec efficiently makes inferences for recommendations with a small backbone, while the interaction rationale distillation and pre-training are time-consuming. Fortunately, these processes are only required once.
A.1.3 Implementation Details For a fair comparison, all the hyperparameters of RDRec are in the same setting as POD. Specifically, both the encoder and decoder consist of 6 layers with each layer comprising an 8-headed attention layer. The vocabulary of T5 contains a total number of 32,100 tokens, with an embedding dimensionality of 512. We iteratively and randomly sampled a segment from a user’s item sequence for training the sequential recommendation task. The number of negative items for top-N recommendation is set to 99 for both training and evaluation. We used the AdamW optimizer (Loshchilov and Hutter, 2017). We set the number of prompt vectors to 3 for all tasks, the batch size for training all three tasks to 64, and the learning rate to 0.001 for the Sports dataset and 0.0005 for both the Beauty and Toys datasets.
We exploit the discrete prompt templates for different tasks from (Geng et al., 2022). During training, we save a checkpoint if the total validation loss of the model in all tasks is the lowest for the current epoch. If this doesn’t occur 5 times, we terminate the training process and load the best checkpoint for evaluation. At the inference stage, we set the number of beams at 20 for sequential and top-N recommendations. For generation tasks, we apply group beam search with the number of beams and beam groups set to 21 and 3, respectively.

heading: A.1.4 Baselines
text: To evaluate the performance of sequential and topN recommendations, we compared our RDRec with twelve baselines: • MF (Koren et al., 2009) accesses the inner prod-
uct between user and item latent factors for predicting users’ preference for candidates. • GRU4Rec (Hidasi et al., 2015) regards the entire item sequence of each user as the user’s session to recommend. • MLP (Cheng et al., 2016) exploits a stack of nonlinear layers to learn user and item embeddings for making recommendations. • CASER (Tang and Wang, 2018) treats user interactions as images and employs 2-dimensional convolutions to capture sequential patterns. • SASRec (Kang and McAuley, 2018) exploits Markov Chains to excavate short-term semantics in users’ sequential patterns. • HGN (Ma et al., 2019) exploits a novel gating strategy to model users’ long- and short-term interests in candidate items. • BERT4Rec (Sun et al., 2019) proposes to leverage the BERT-style cloze task for the sequential recommender algorithm. • FDSA (Zhang et al., 2019) incorporates item features with item sequences of users to perform recommendations. • S3-Rec (Zhou et al., 2020) learns users’ latent behavioral features via employing a self-supervised learning paradigm. • P5 (Geng et al., 2022) converts three different recommendation tasks into textual generation tasks using LLMs for recommendations. • RSL (Chu et al., 2023) adopts novel training and inference strategies to deliver LLM-based recommendations. • POD (Li et al., 2023b) refines P5 through prompt distillation to make efficient and precise recommendations.

heading: A.2 Further Analyses

heading: A.2.1 Hallucination by LLMs
text: We observed that when a review is too short, the LLM might produce hallucinations. The following example illustrates a rationale with hallucinations generated by the LLM:
Input: This is a fantastic game.
Output: The user prefers games with engaging storylines .
The item’s attributes include a realistic game world ,
immersive sound effects , and smooth gameplay .
The contents of “engaging storylines”, “immersive sound effects” and “smooth gameplay” marked by gray are hallucinations overly inferred by the LLM. Toward this, mitigating hallucinations of LLM-based recommender is a rich space for future exploration (Liu et al., 2022; Gao et al., 2023; Peng et al., 2023; Zhang et al., 2023a).

heading: A.2.2 Effect of Explanation Generation
text: We observed that RDRec can generate correct explanations in many cases, such as the explanation
“This is a great product for the price,” for the provided review “very good quality for the price.”
However, RDRec sometimes recommends candidates correctly but provides explanations that completely differ from the user’s review. For instance, the generated explanation is, “Absolutely great product,” whereas the user’s actual review is, “I wouldn’t recommend this for painting your full nail.” One possible reason is that RDRec has learned to prioritize predicting user-item interaction over considering the rationale for making recommendations. This is a challenging yet intriguing path to further improve RDRec.

heading: A.2.3 The Whole-Word Embedding
text: To address the token composing issue (i.e., the token of “user_1234” is often tokenized by the tokenizer of LLMs as a sequence of [“user”, “_”, “12” and “34”]), we employed the whole-word embedding (Geng et al., 2022) to ensure that each sequence of ID tokens is a complete unit and can be distinguished from a word.
It is noteworthy that the whole-word embedding will not cause scalability issues because we only need to identify which tokens represent the same user (or item). For instance, given a token list [“P1”, “P2”, “P3”, “user”, “_”, “12”, “34”, “item”, “_”, “98”, “76”], the index list over the whole-word embedding vocabulary is [0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 2]. Since the number of negative samples is set to 99 and the average user interaction is less than 9 in our datasets, an embedding matrix (512 * 512) with a maximum incremental number of 512 is sufficient. Even if a user’s interaction count exceeds 512, we only need to expand the wholeword embedding matrix, which is acceptable for a real-world deployment.

heading: Future Work
text: Toward this, mitigating hallucinations of LLM-based recommender is a rich space for future exploration (Liu et al. , 2022; Gao et al. , 2023; Peng et al. , 2023; Zhang et al. , 2023a)