Leveraging LLM Reasoning Enhances Personalized Recommender Systems (2024)

Abstract: Recent advancements have showcased the potential of Large Language Models (LLMs) in executing reasoning tasks, particularly facilitated by Chain-of-Thought (CoT) prompting. While tasks like arithmetic reasoning involve clear, definitive answers and logical chains of thought, the application of LLM reasoning in recommendation systems (RecSys) presents a distinct challenge. RecSys tasks revolve around subjectivity and personalized preferences, an under-explored domain in utilizing LLMs’ reasoning capabilities. Our study explores several aspects to better understand reasoning for RecSys and demonstrate how task quality improves by utilizing LLM reasoning in both zero-shot and finetuning settings. Additionally, we propose RecSAVER (Recommender Systems Automatic Verification and Evaluation of Reasoning) to automatically assess the quality of LLM reasoning responses without the requirement of curated gold references or human raters. We show that our framework aligns with real human judgment on the coherence and faithfulness of reasoning responses. Overall, our work shows that incorporating reasoning into RecSys can improve personalized tasks, paving the way for further advancements in recommender system methodologies.

text: Findings of the Association for Computational Linguistics ACL 2024, pages 13176–13188 August 11-16, 2024 ©2024 Association for Computational Linguistics
Recent advancements have showcased the potential of Large Language Models (LLMs) in executing reasoning tasks, particularly facilitated by Chain-of-Thought (CoT) prompting. While tasks like arithmetic reasoning involve clear, definitive answers and logical chains of thought, the application of LLM reasoning in recommendation systems (RecSys) presents a distinct challenge. RecSys tasks revolve around subjectivity and personalized preferences, an under-explored domain in utilizing LLMs’ reasoning capabilities. Our study explores several aspects to better understand reasoning for RecSys and demonstrate how task quality improves by utilizing LLM reasoning in both zero-shot and finetuning settings. Additionally, we propose RecSAVER (Recommender Systems Automatic Verification and Evaluation of Reasoning) to automatically assess the quality of LLM reasoning responses without the requirement of curated gold references or human raters. We show that our framework aligns with real human judgment on the coherence and faithfulness of reasoning responses. Overall, our work shows that incorporating reasoning into RecSys can improve personalized tasks, paving the way for further advancements in recommender system methodologies.

heading: 1 Introduction
text: The rapid advancement of Large Language Models (LLMs) has ushered in a new era of transformative capabilities, demonstrating the potential across a spectrum of applications. Recent progress has showcased their ability in reasoning tasks. Particularly, the advent of Chainof-Thought (CoT) [30] and zero-shot CoT prompting [19] has provided a pathway for these models to engage in multi-step reasoning. Tasks examined in these studies, ranging from arithmetic reasoning [1, 5, 16] to common-
*Equal contribution. Contact: aliciatsai@berkeley.edu, adamkraft@google.com.
†Work done while at Google DeepMind.
sense question answering [14, 25], typically demand clear, definitive answers and logical chains of thought. In contrast, the landscape of recommender systems (RecSys), outlined in Figure 1, introduces a nuanced challenge, where reasoning extends beyond objective criteria to encompass subjectivity and personalized user preferences. This aspect remains an under-explored domain in leveraging the reasoning capabilities of LLMs. Prior works have utilized LLMs in recommender systems, employing techniques such as in-context learning and instruction tuning [12, 13, 18, 32]. However, a comprehensive understanding of how LLMs execute reasoning in the context of personalized preferences remains elusive. Our work fills this gap by investigating how the reasoning capabilities of LLMs can enhance personalized recommendations in zero-shot and fine-tuning, resulting in improved task performance.
In contrast to the prediction task performance, objectively assessing the quality of reasoning presents challenges in the absence of curated gold standard references or human raters. To surmount this, we propose RecSAVER. This framework provides an efficient means of assessing the quality of LLM outputs, contributing to our understanding of LLM reasoning dynamics in personalized recommendation scenarios. We gauge human assessments on coherence, faithfulness, and insightfulness. Our observations suggest that syntactic metrics such as BLEU and ROUGE, demonstrate suitability in evaluating the output faithfulness of LLMs. On the other hand, metrics like METEOR and BERTScore prove to be more adept at measuring coherence in the generated outputs. To the best of our knowledge, this is the first study that comprehensively examines the effects and quality of LLM reasoning for personalized RecSys tasks. In summary, our contributions are as follows:
1. We explore the utilization of LLMs for reasoning in personalized recommendations, showcasing notable task performance improvements in both zero-shot and fine-tuning scenarios. 2. We demonstrate the effectiveness of using larger models to generate reasoning data, enhancing the performance and reasoning abilities of smaller finetuned models. 3. We introduce Rec-SAVER, an automatic reasoning evaluation framework that doesn’t require curated gold references. It offers meaningful insights
13176
into LLMs’ reasoning capabilities, aligning with human judgment while providing cost and efficiency benefits.

heading: 2 Methodology

heading: 2.1 Problem Setting
text: The landscape of RecSys tasks, segmented along two axes, is illustrated in Figure 1. In our experiments, we focus on the user rating prediction task, which involves a high degree of both user decision making effort and collected user feedback, making this task well suited for exploring the extent of LLM reasoning for RecSys.
Let R represent a collection of user ratings and D denote a collection of user-written reviews for a set of items I belonging to a specific category (e.g., books), provided by users in U . Each rating ru,i ∈ R is paired with a corresponding written review du,i ∈ D, reflecting the user’s u ∈ U overall satisfaction with item i ∈ I. Each item i is associated with metadata Mi, comprising details such as title, description, category, brand, and price. The user’s purchase history Hu = (hu,1, hu,2, . . . , hu,t) constitutes a chronologically ordered collection of past purchases. Each past purchase hu,j = (Mj , ru,j ,du,j) represents a triplet comprising the metadata for the purchased item j, the user’s rating for the purchase item, and the user’s review for that item. The primary objective of the rating prediction task is to forecast the unknown ratings for items that users have not yet reviewed. The objective can be formalized as follows:
r̂u,i = argmax k
P(ru,i = k | Hu,Mi),
where i /∈ Hu, k ∈ {1, 2, 3, 4, 5}. (1)
In this equation, r̂u,i represents the predicted rating for user u on item i, chosen from the set of possible ratings 1 to 5. This prediction is based on the user’s purchase
historyHu and the item’s metadataMi, where the item i has not been previously reviewed by user u. Recent advancements in recommendation systems utilize LLMs to model the rating prediction task described by equation (1), denoted as r̂u,i = LLM(Hu,Mi).
2.2 Zero-shot Learning with Reasoning
As shown in Figure 2, we employ zero-shot CoT prompting strategies [19] to guide the LLM in generating a reasoning response, denoted as ŝu,i, alongside a rating prediction r̂u,i for a given user u and recommended item i:
〈 ŝu,i, r̂u,i 〉 = LLM(Hu,Mi). (2)
Our prompt consists of four key elements: a preamble, the user history Hu, the new item metadataMi, and a task description. The preamble provides context for the subsequent information and establishes the rating scale ranging from 1 to 5. Following the preamble, the user historyHu is presented sequentially, detailing the user’s past interactions. A new item i is then introduced along with it’s metadataMi before the task description, prompting the model to make predictions. The task description also delineates the output requirements for the model responses. An abstract prompt template is illustrated in Table 1, while more prompt details are provided in Appendix A. Unlike traditional RecSys modeling techniques, our approach leverages natural language presentation for all information. This enables a more intuitive representation of rich content as natural language, as opposed to numerical IDs, enabling a more encompassing understanding of information [13].

heading: 2.3 Fine-tuning with Reasoning
text: Zero-shot learning with CoT prompting can be computationally intensive. Hence, fine-tuning with domainspecific datasets has emerged as a pragmatic strategy, especially when leveraging smaller pre-trained models [4]. Our interest lies in investigating whether training with reasoning outputs can further enhance task performance. Building on the prompting methods outlined in
Section 2.2, we collect reasoning outputs generated by a larger language model to serve as training data for finetuning smaller models. For each user-item pair (u, i) with input (Hu,Mi), we gather multiple reasoning responses and rating predictions by adjusting a decoding temperature parameter T > 0 during generation [17]:
〈 ŝmu,i, r̂ m u,i 〉 = LLM(Hu,Mi), (3)
where m = 1, 2, . . . ,M indexes the M candidate output pairs sampled from the decoder. This process yields a diverse set of reasoning paths, which is particularly advantageous for personalized recommendations, recognizing that the same rating can stem from various personal preferences and reasons. Optionally, we can use different methods to filter out reasoning responses r̂mu,i that do not align with the ground truth ru,i. The fine-tuned model is then trained using the reasoning responses ŝmu,i and the real ground truth rating label ru,i as targets. The overall method is illustrated in Figure 3.

heading: 3 Rec-SAVER: Evaluation of Reasoning
text: In contrast to reasoning processes for solving mathematical problems or general question answering tasks, reasoning in RecSys rating prediction is highly subjective and personalized for individual users. Unlike in other domains where humans can provide reasoning steps and verify their validity, resulting in curated gold references, such references are challenging to obtain in RecSys due to the subjective nature of user preferences. To address this challenge, we propose a system called Rec-SAVER: Recommender Systems Automatic Verification and Evaluation of Reasoning. Rec-SAVER aims to automatically generate good reasoning references specifically tailored for RecSys tasks. These references can then be utilized to quantitatively evaluate the quality of reasoning responses generated by LLMs. Additionally, we conduct a human study to demonstrate the alignment of our method with real human judgment, thus providing validation for the effectiveness of RecSAVER in assessing reasoning quality in RecSys applications.

heading: 3.1 Reference Generation with Self-Verification
text: The core concept of Rec-SAVER involves a two-step process leveraging LLM-generated explanations and LLM self-verification. As illustrated in Figure 4, we present the LLM with a user-item pair (Hu,Mi). Additionally the target user rating ru,i is also provided as input. The model is instructed to provide a post hoc explanation, describing why the user assigned such a rating based on the given user history and new item information. We denote this post hoc explanation generated by LLM as ĝu,i. Note that this is different from the aforementioned reasoning ŝu,i, where the ground truth rating ru,i is not included as an input. Similar to the approach in Section 2.3 where multiple responses are sampled, we generate N different explanations ĝnu,i where n = 1, 2, . . . , N . These post hoc explanations are then passed onto a verification process.
To ensure the credibility and consistency of these LLM-generated explanations ĝnu,i, we implement a selfverification step atop the previous explanation generation process. The self-verification step involves making a second call to the same LLM, inputting the user-item information (Hu,Mi), and the explanations ĝnu,i generated from the previous call. The model is then tasked with making a rating prediction based on the user history, new item information, and the post hoc explanation, formally defined as r̃nu,i = LLM(Hu,Mi, ĝnu,i). However, in practice, we have observed that many explanations ĝnu,i contain text snippets such as “the user gave a rating of 5 because ...”, which can lead to information leakage. To prevent ĝnu,i from directly leaking the ground truth, we employ a simple post-processing step by removing sentences that mention “a rating of ”, “stars”, or “scores” before performing the prediction. In future work we aim to improve upon this manual process to fully ensure the removal of information leakage.
We then validate whether the new rating r̃nu,i matches the original ground truth ru,i. Explanations ĝnu,i that pass the self-verification step are retained as the final verified references, constituting a diverse pool of LLM-generated references Ĝ. This two-step process follows the intuition that good explanations based on the given information and the ground truth should enable the model to make a correct prediction. By validating the predictions based on the generated explanations against the ground truth ratings, we ensure that only high-quality explanations are retained to serve as the final references. These verified references then serve as proxies for the unknown set of gold references G. Since the self-verification may result in different verified references per sample, we may have varying numbers of final references per sample. The full reference generation process is summarized in Algorithm 1.
Algorithm 1 Reference generation with self-verification 1: Inputs: N 2: Ĝ ← ∅ . verified references 3: for (Hu,Mi, ru,i) in dataset do 4: for n = 1 . . . N do 5: ĝnu,i ← LLM(Hu,Mi, ru,i) 6: ĝnu,i ← post-process(ĝnu,i) 7: r̃nu,i ← LLM(Hu,Mi, ĝnu,i) 8: if r̃nu,i = ru,i then 9: Ĝ ← Ĝ ∪ {ĝnu,i} 10: end if 11: end for 12: end for

heading: 3.2 Human Judgment Alignment Study
text: To gauge the effectiveness of Rec-SAVER, we conduct a study to evaluate how our proposed method aligns with human judgment regarding the candidate reasons generated by LLMs. This study aims to provide insight into the reliability and validity of our proposed method. During the study, human raters are presented with sample input prompts and the reasoning outputs ŝu,i generated by LLMs. It is important to note that these reasoning outputs ŝu,i are produced using only user history Hu and item metadataMi as inputs; no ground truth rating ru,i is provided to the LLM. No ratings (ground truth or LLM predicted) are shown to the human raters. Human raters are asked to assess the reasoning outputs based on the following dimensions:
• Coherence (5-point Likert): Evaluate whether the generated reasoning makes sense and follows a clear and coherent logical flow that reflects the reasons behind the user’s preference. • Faithfulness (Binary): Examine the presence of hallucination in the generated reasoning and whether it contains fabricated information.
• Insightfulness (5-point Likert): Assess the degree to which the generated reasoning delivers valuable, informative, interesting or delightful insights into the user’s preferences and purchasing patterns.

heading: 4 Experiments

heading: 4.1 Data Preparation and Tasks Setup
text: Experiments are conducted on the Amazon product review dataset*, which is a widely recognized benchmark in RecSys. This dataset offers comprehensive user feedback, including ratings and review text, as well as detailed product metadata such as descriptions, category information, price, and brand [22]. We focus our experiments on the rating prediction task, conducting evaluations in two distinct domains: BEAUTY and MOVIES/TV. To understand the extent that LLMs can understand user preferences, we filter examples where 4 ≤ |Hu| ≤ 10. The lower threshold ensures we have enough past purchases to see trends and patterns while the higher threshold prevents inputs from exceeding the LLM context window. The original label distribution is heavily skewed towards positive ratings, with a rating of 5 accounting for over 60 % of the data. We perform random subsampling to create a fully balanced dataset with an even label distribution, resulting in 4,000 training examples (800 per label) and 500 test examples (100 per label). The training split is used to test out different prompts for zero-shot learning and as training examples for the fine-tuning experiments. The user sets are mutually exclusive between training and test. This setup allows us to better test and understand the capabilities of LLM reasoning, while we acknowledge this may not reflect a full real world scenario.
In following sections, we report the rating task predic-
*https://cseweb.ucsd.edu/~jmcauley/datasets/ amazon_v2/
tion metrics, including multi-class and binary metrics. Specifically, we compare the model’s output r̂u,i against the ground truth rating ru,i. The binary metrics were calculated using a cutoff threshold of r̂u,i > 3. The quality of the reasoning outputs will be analyzed in more detail later on in Section 5.

heading: 4.2 Zero-shot Learning Improves with Reasoning
text: In the following zero-shot experiments, we utilize the PaLM 2-M LLM [2], a highly capable model trained on a broad set of languages and tasks, including reasoning tasks. Initially, we investigate the impact of prompting the model to engage in reasoning prior to prediction (zero-shot chain-of-thought) compared to direct prediction (zero-shot). Table 2 shows the differences between the final task description of the input prompt for zero-shot CoT and direct prediction. Subsequently, Table 3 outlines the outcomes of different zero-shot ablation studies. We also compare against a naive baseline, where we use the historical rating average of the user’s history as a prediction for the future item. We observe a notable performance improvement across both product domains when the model is guided to output reasoning alongside the prediction (“Our Method (zero-shot CoT)” vs. “No Reasoning Outputs”). This suggests that personalized tasks are inherently difficult for LLMs to solve without further guidance such as engaging in an intermediate reasoning step.
Impact of explicit user feedback. A user’s past purchase, denoted as ht = (Mj , ru,j ,du,j) ∈ Hu, encapsulates the user-item relations, providing explicit user feedback. To understand the helpfulness of this feedback, we conduct ablation studies. In the first case, we eliminate the written reviews du,j from the purchase history where ht = (Mj , ru,j). In the second case, we further eliminate the numerical ratings ru,j from ht, resulting in ht = (Mj). The first case mirrors a common scenario in RecSys where only numerical rating information is available, while the second case simulates scenarios where only implicit feedback from a user, in the form of past purchases, is accessible. Table 3 presents the ablation results, highlighting a significant performance drop when the review text is excluded from ht (“No Review”). The performance declines further when both reviews and ratings are excluded (“No
Review, No Rating”). When only the written review text is removed, the results are similar to direct predictions made without reasoning (“No Review” vs “No Reasoning Outputs) and sometimes even worse than naive average baseline. This indicates that review text is essential for utilizing the reasoning capabilities of LLMs. Without user reviews, the model lacks detailed insights into past user interactions and can only rely on numerical ratings, resulting in performance similar to or worse than “No Reasoning Outputs” and the naive average baseline.
Furthermore, when both reviews and ratings are unavailable, the outcomes are akin to random guessing, as evidenced by the multi-class accuracy hovering around 0.2. This performance is strictly worse than the naive baseline and direct prediction without the reasoning outputs. Our ablation studies suggest that while the LLM can estimate some user preference information simply from the numerical user rating, it benefits even more when we have the full written user reviews and a guided reasoning step. Review texts help discern nuanced details about a user’s specific preferences. For instance, a user might rate the movie “Top Gun” as 5 stars for various reasons, such as their interest in airplanes, admiration for Tom Cruise, or a fondness for action movies in general. These preferences may be explicitly stated in the review text, enabling the LLM to make more informed decisions.
Effect of Pre-trained Knowledge. In addition to analyzing the impact of excluding explicit user feedback, we also investigate the removal of item descriptions. We found that while performance decreases in both domains when item descriptions are unavailable, the decline is less pronounced for MOVIES/TV compared to BEAUTY. This finding suggests that the LLM (PaLM 2-M) possesses a more extensive knowledge base in MOVIES/TV domain, enabling it to infer information about the recommended movies or TV shows even in the absence of descriptions.
One-shot learning. We compare one-shot learning against our zero-shot results (Table 3). Surprisingly, the one-shot results are significantly worse, yielding only slight improvements over random performance. We believe the inclusion of a single example considerably increases the input text length. This could potentially hinder the model’s ability to disentangle information from the provided example and the actual inputs (Mi,Hu), resulting in poorer performance.

heading: 4.3 Fine-tuning with Reasoning Data
text: We utilize Flan-T5 [4] models as they are readily available to conduct fine-tuning experiments. Although these models are all trained on a variety of data and tasks, it should be noted that PaLM-2-M reports significantly higher quality than Flan-T5 on common benchmarks, including the Massive Multitask Language Understand-
ing (MMLU) [16]. Unless specified, we employ the Flan-T5 XL model (3B parameters) to output reasoning followed by a final rating prediction. We fine-tune the models for 100,000 steps with a batch size of 64, a dropout rate of 0.25, and a learning rate of 1e-4. We report the evaluation metrics on the test set at the end of training. In Table 4 we compare two fine-tuned XL models: one that outputs reasoning and rating, and another that outputs rating only. For our first experiment, we use only 1 reasoning sample for each user-item pair, i.e. m = 1 for Eq. (3), and refrain from further filtering to maintain consistent training sample numbers for a fair comparison. We collect model probabilities by extracting and normalizing the logits corresponding to the 5 rating class tokens, allowing us to compute ROC-AUC metrics. The results from both domains con-
sistently indicate that fine-tuning models to engage in reasoning prior to making predictions leads to improved performance across all metrics.
In addition to comparing the XL models with reasoning outputs against those without, we fine-tune various Flan-T5 model sizes: Small (80M params), Base (250M), Large (780M), and XL (3B). All models are trained with one reasoning sample per example without any filtering. Table 4 illustrates a clear trend indicating that larger models perform better on the rating task. This observation aligns with the general understanding that larger models typically possess greater knowledge capacity, leading to enhanced performance on downstream tasks after fine-tuning. Moreover, larger models also tend to have better reasoning quality, which we discuss further in Section 5.2.
We also include results on the Flan-T5 XL model without any fine-tuning for reference, showing that finetuning is absolutely necessary to improve results for this model family. For this non fine-tuned model, we tried both with and without an additional reasoning output. When asked to output reasoning, the model was unable to follow instructions and did not output a final rating in almost all cases. Therefore, we only report no finetuning results without reasoning. Additionally, although we see drastic improvements for Flan-T5, our best FlanT5 result still underperforms our best PaLM 2-M result without fine-tuning. This difference is likely attributable to the enhanced capabilities of PaLM 2-M relative to Flan-T5.
Training with multiple reasoning paths. In this experiment, we increase the number of reasoning samples m to 8 for each user-item pair, as defined in Eq. (3), providing diverse reasoning paths for each user-item relation. We also apply different filtering methods based on comparing the zero-shot LLM rating predictions r̂mu,i to the ground truth ratings ru,i. Table 5 presents the results of training with multiple reasoning paths and with different filtering methods:
• None: No filter is applied. The model is trained with all reasoning samples ŝmu,i per user-item pair. • 5-class: Reasoning samples are removed where r̂mu,i 6= ru,i in terms of the 5-class rating. • Binary: Reasoning samples are removed where the binary conversion of (r̂mu,i > 3) 6= (ru,i > 3). • 1-off: Reasoning samples are removed where the absolute difference |r̂mu,i − ru,i| > 1.
In the BEAUTY domain, fine-tuning with 8 reasoning paths without any filtering slightly outperforms finetuning with only 1 reasoning path. Surprisingly, applying filtering methods significantly diminishes performance on the rating task. We hypothesize that filtering may remove a substantial portion of training samples, leading to poorer performance. This is particularly evident when the "5-class" filter, the most stringent filter, is applied. Conversely, in the MOVIES/TV domain,
the best results are achieved with the "1-off" filtering method. We attribute this to the LLM’s strong pretrained knowledge in the MOVIES/TV domain, allowing it to tolerate the removal of examples where the reasoning does not align with the ground truth rating. In contrast, the BEAUTY domain may require more examples of user history and user-item relations for effective learning. Removing examples, even those with misaligned reasoning, may inadvertently reduce domain information, resulting in diminished performance. Further analysis of these effects will be conducted in future work.

heading: 5 Reasoning Evaluation

heading: 5.1 Human Judgment Alignment Analysis
text: As proposed in Sec. 3.2, we design a human judgement alignment study to evaluate the effectiveness of RecSAVER . We presented a total of 100 samples to human raters, with 50 examples from the BEAUTY domain and 50 examples from the MOVIES/TV domain. Each rating category was evenly represented. Each sample was annotated by 3 different annotators, resulting in a total of 300 annotated data points. Table 6 presents the inter-annotator agreement of weighted Cohen κ [6] and the average Pearson correlation (Avg. ρ) among the human annotated scores. The achieved statistical significance of the average correlation between 3 human annotators across all 3 measurements, as indicated by the p-values, signifies the level of consensus among annotators.
We evaluated four commonly used natural language generation (NLG) metrics: BLEU [23], ROUGE-1 [20], METEOR [3], and BERTScore [33]. BLEU and ROUGE-1 measure syntactic similarity by computing the exact n-gram overlap between the generated output and the reference texts. On the other hand, METEOR and BERTScore consider semantic similarity, providing a more comprehensive evaluation by incorporating contextual information. Table 7 reveals a consistently positive correlation between coherence and all NLG metrics, suggesting that our proposed evaluation method using LLM-generated references aligns well with coherence. However, insightfulness exhibits no correlation with BLEU and a low correlation with ROUGE-1 F1, while demonstrating a slightly positive correlation with METEOR and BERTScore. Unlike “coherence” and ”faithfulness”, “insighfulness” is an exploratory metric aimed at understanding how LLM surprise human raters. It is anticipated that syntactic metrics such as BLEU and ROUGE-1 F1 may not correlate strongly with insightfulness. For instance, a response could be considered insightful even if it lacks significant n-gram overlap with the references. Although the semantic metrics METEOR and BERTScore show better correlation, they still do not align as closely with insightfulness as they do with coherence.
Effectiveness of Self-verification. To validate the effectiveness of the self-verification step in Rec-SAVER, we compared the metrics computed with and without the reference self-verification step. Table 9 shows that metrics computed from self-verified references show a stronger correlation with the coherence score compared to those without self-verification, indicating that selfverification contributes to increasing the credibility of LLM-generated references. Combining all results of this section, we observe a strong alignment between our proposed Rec-SAVER reasoning evaluation method and
human judgments regarding quality.

heading: 5.2 Analysis of Reasoning Quality
text: Having established Rec-SAVER as a method for generating references and evaluating reasoning, we can leverage it to further analyze model reasoning outputs. We display example reasoning outputs in Table 10. Our first investigation focuses on the question: “Are correct rating predictions generally associated with higher-quality reasons?” Table 11 demonstrates that reasoning metrics indeed improve for examples with correct predictions, both for zero-shot and fine-tuned models. Next, we analyze the reasoning quality across different methods discussed throughout the paper. Focusing on zero-shot models, we observe that in the MOVIES/TV domain, the NLG metrics decrease when we remove input information (Table 3). However, for BEAUTY, when we remove some input information like user reviews, some of the NLG metrics increase. This suggests that the LLM may encounter challenges synthesizing all of the information in BEAUTY but possesses a better overall knowledge of MOVIES/TV, allowing it to generate better reasons when provided with more information available.
Table 4 illustrates that beyond the rating prediction metrics, the NLG metrics also improve as we increase the fine-tuned model size. This suggests that in addition to producing better rating predictions, the models also generate better reasoning responses. In the MOVIES/TV domain, the “1-off” filtering method appears to yield the best rating metrics (Table 5), although a few other methods are comparable. However, when considering the NLG metrics, we observe more data showing that the “1-off” filtering method has an advantage over the other methods.
The comparison between the fine-tuned models and the zero-shot model for MOVIES/TV reveals that the zero-shot model outperforms the fine-tuned model in both rating and NLG metrics. This outcome suggests that the PaLM 2-M zero-shot model likely possesses superior pre-trained knowledge in this domain, which cannot be fully distilled into the Flan-T5 fine-tuned model. For example, the reasoning text used in the finetuning training data may include mentions of certain actors or directors in movies. However, this data might not cover a wide enough range of examples to provide the model a comprehensive understanding of the entire domain. Consequently, the final fine-tuned model may still exhibit information gaps in this domain compared
to the zero-shot model.

heading: 6 Related Work
text: LLM for RecSys. Recent advancements in the application of LLMs to RecSys have yielded diverse approaches. Typically, these approaches follow pretraining, fine-tuning, and prompting paradigms. In the context of recommendation tasks, fine-tuning LLMs is essential to acquire domain-specific knowledge. This fine-tuning process involves training the pre-trained model using task-specific recommendation datasets containing user-item interaction behaviors such as purchase, ratings, or click, and additional contextual information about users and items (e.g., social relations or item descriptions) [10, 7, 21]. Beyond the pre-training and finetuning paradigm, prompting has emerged as a recent paradigm to tailor LLMs for specific downstream tasks, employing task-specific prompts [27, 9, 12], along with prompting techniques like in-context learning [11] and chain-of-thought [30]. More recently, there has been exploration into instruction tuning, a hybrid approach combining the pre-training and fine-tuning paradigm with prompting. This involves fine-tuning LLMs across multiple recommendation tasks using instruction-based prompts, enhancing the zero-shot performance of LLMs on previously unseen RecSys tasks [13, 32, 18].
LLM reasoning. Recent studies have suggested that the ability to reason may emerge in language models at a certain scale [5, 29]. These models, when provided with a few examples of “chain of thought”, which represent intermediate natural language reasoning steps, demonstrate the capability to generate explicit rationales before producing final answers [30]. Advances in this
direction include zero-shot CoT [19], where the model is prompted with the phrase “Let’s think step by step” to elicit reasoning without the inclusion of few-shot demonstrations. Various strategies have been proposed to enhance language model performance by prompting reasoning, such as multi-step reasoning [8, 34], treeof-thoughts [31], iterative CoT prompting [26] and selfconsistency [28]. Despite the impressive performance of LLMs on various reasoning tasks, the clarify of whether their predictions are based on true reasoning remains a challenge. This ambiguity arises because most existing evaluations focus on accuracy in end tasks rather than directly assessing the quality of the reasoning. Recent efforts have introduced metrics such as ROSCOE [15] and dataset such as PrOntoQA [24] for a more formal analysis of reasoning in LLMs. However, the application of these metrics and benchmarks to a broader range of tasks is still an area of limited depth.

heading: 7 Conclusion and Discussion
text: We explore reasoning in the context of personalized recommender systems, showing that adding reasoning steps can improve LLM task performance. It is important to have rich user context and explicit feedback in order for the LLMs to reason adequately. Having good pretrained domain knowledge is also useful. RecSAVER, our proposed method for analyzing reasoning quality, aligns well with human judgment on the coherence of reasoning outputs and can be used to further evaluate model quality beyond numerical task results.
Limitations. In this work we started with rating predictions in the Amazon review dataset for two categories, BEAUTY and MOVIES/TV. However, the extent of recommender systems is vast. It is unclear to what extent our methods generalize more broadly to other categories such as music, video games, website articles, etc. Furthermore, more work is needed to explore these methods on other tasks, including candidate retrieval or ranking.
Now that we see evidence that reasoning is useful in RecSys, more work should be done to understand the extent and mechanisms behind this. Does the LLM actually reason in a manner that helps make a final decision similar to human thought? Or is there some other underlying procedure that yields these results, such as more overall computation or better attention? Future work
looking at different prompting strategies and reasoning plans could help uncover more details in this area.
Ethical Considerations. In this study, biases may exist for reasoning results for different users, including users that speak different languages or users with different genders. The dataset we use focuses on users that speak English. Also, users from different genders may interact differently with certain domains or with products in those domains, leading to skewed distributions in the data. Broader experiments are needed to understand these potential biases further in the context of reasoning for recommender systems.

heading: 8 Acknowledgements
text: We would like to thank Jianmo Ni, Nikhil Mehta, Ramkumar Rajendran, and Lakshmi Chakrapani for their helpful discussions and support.

heading: A Reasoning Generation Prompt

heading: B Additional Experimental Results
text: The weighted Cohen κ is calculated as followed:
κ = 1− ∑n i=1 ∑n j=1 wijxij∑n
i=1 ∑n j=1 wijmij ,
where n = 5 is the number of rating scale, and w, x and m are elements in the weight, observed, and expected matrices. Here, we use a quadratic weight where wij = (i−j)2 (k−1)2 to amplify the difference between scores.
Table 13 provides a comparison of human-rated reasoning outputs between incorrect and correct predictions, as depicted in Figure 5. The table shows the mean
scores for coherence, faithfulness, and insightfulness for reasoning outputs associated with incorrect and correct predictions. We observe that reasoning outputs corresponding to correct predictions receive higher scores across all three dimensions, indicating a higher reasoning quality when the prediction is correct.

heading: Future Work
text: Does the LLM actually reason in a manner that helps make a final decision similar to human thought? Or is there some other underlying procedure that yields these results, such as more overall computation or better attention? Future work
looking at different prompting strategies and reasoning plans could help uncover more details in this area. Ethical Considerations. In this study, biases may exist for reasoning results for different users, including users that speak different languages or users with different genders. The dataset we use focuses on users that speak English. Also, users from different genders may interact differently with certain domains or with products in those domains, leading to skewed distributions in the data. Broader experiments are needed to understand these potential biases further in the context of reasoning for recommender systems