Tree of Preferences for Diversified Recommendation
Authors: Hanyang Yuan
Ning Tang
Tongya Zheng
Jiarong Xu
Xintong Hu
Renhong Huang
Shunyu Liu
Jiacong Hu
Jiawei Chen
Mingli Song
Venue: NeurIPS 2025 poster ()
Keywords: Large language model
recommendation diversity
Primary area: deep_learning

Abstract: Diversified recommendation has attracted increasing attention from both researchers and practitioners, which can effectively address the homogeneity of recommended items.
Existing approaches predominantly aim to infer the diversity of user preferences from observed user feedback.
Nonetheless, due to inherent data biases, the observed data may not fully reflect user interests, where underexplored preferences can be overwhelmed or remain unmanifested. Failing to capture these preferences can lead to suboptimal diversity in recommendations. To fill this gap,  this work aims to study diversified recommendation from a data-bias perspective.
Inspired by the outstanding performance of large language models (LLMs) in zero-shot inference leveraging world knowledge, we propose a novel approach that utilizes LLMs' expertise to uncover underexplored user preferences from observed behavior, ultimately providing diverse and relevant recommendations.
To achieve this, we first introduce Tree of Preferences (ToP), an innovative structure constructed to model user preferences from coarse to fine. ToP enables LLMs to systematically reason over the user's rationale behind their behavior, thereby uncovering their underexplored preferences.
To guide diversified recommendations using uncovered preferences, we adopt a data-centric approach, identifying candidate items that match user preferences and generating synthetic interactions that reflect underexplored preferences.  These interactions are integrated to train a general recommender for diversification.
Moreover, we scale up overall efficiency by dynamically selecting influential users during optimization.
Extensive evaluations of both diversity and relevance show that our approach outperforms existing methods in most cases and achieves near-optimal performance in others, with reasonable inference latency.

Tree of Preferences for Diversified Recommendation
Hanyang Yuan It. .ing Tang", Tongya Zheng?* " Xu²* Xintong Hu' Renhong Huang', Shunyu Liu', Jiacong Hu Hu', Jiawei Chen', Mingli Song'
'Zhejiang University, FFudan University
Hangzhou City University, "Nanyang Technological University (yuanhanyang, 320010383, renh2) juju. edu. en {jiaconghu, sleepyhunt, brooksong Hzzaanngg@euuuu edu cn ningtang240m. fudan edu. en, jiarongxu@fudan edu en doujiang. 2ergggg__eeeee11.. com, com, shunyu. liu@ntu. edu sg
Abstract
Diversified recommendation has attracted increasing attention from both re- searchers and practitioners, which can effectively address the homogeneity recommended items. Existing approaches predominantly aim to infer the diversity user preferences from observed user feedback. Nonetheless, due inherent data biases, the observed data may not fully reflect user interests, where underexplored preferences can be overwhelmed or remain unmanifested. Failing to capture these preferences can lead suboptimal diversity in recommendations, To fill this gap, this work aims to study diversified recommendation from a data-bias perspective. Inspired by the outstanding performance of large language models (LLMs) in zero-shot inference leveraging world knowledge, we propose a novel approach that utilizes LLMs' expertise uncover underexplored user preferences from observed behavior, ultimately providing diverse and relevant recommendations, To achieve this, we first introduce Tree Preferences (ToP), an innovative structure constructed to model user preferences from coarse to fine. ToP enables LLMs to systematically reason over the user's rationale behind their behavior, thereby uncov- ering their underexplored preferences. To guide diversified recommendations using uncovered preferences, we adopt a data-centric approach, identifying candidate items that match user preferences and generating synthetic interactions that reflect underexplored preferences. These interactions are integrated to train a general recommender for diversification. Moreover, scale up overall efficiency by dy- namically selecting influential users during optimization. Extensive evaluations both diversity and relevance show that approach outperforms existing methods in most cases and achieves ncar-optimal performance in others, with reasonable inference latency.
Introduction
Recommender systems have gained significant value in recent years, powering diverse applications such social media 21 e-commerce [3 4 and video streaming platforms (5) To improve recommendation performance, numerous models have emerged, which are essentially built upon interactions between users and items 7-10, such shares, likes in social media, ratings in
'Equal contribution.
"Corresponding authors.
State Key Laboratory of Blockchain and Security, Zhejiang University.
'Zhejiang Provincial Engineering Research Center for Real-Time SmartTech in Urban Security Governance, School of Computer and Computing Science, Hangzhou City University.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

e-commerce, or viewing time in video streaming 14). While these systems strive to optimize alignment with user behavior data, it is increasingly recognized that user feedback, typically based on observed data, carries inherent biases, which can have various impacts on recommendation systems Among these, recommendation diversity is an aspect crucial to user satisfaction.
7 15
Recommendation diversity represents the dissimilarity of items recommended to an individual user [16]. Numerous studies have shown that higher diversity can provide users with freshness, meet their diverse interests, and lead to higher engagement I [7 Recent works have explored various methods to enhance recommendation diversity. Some focus on the reranking phase, leveraging user- bser-behavior models [17) or graph clustering [T] to capture user interests. Others focus on the matching stage, including multi-vector representations (7), separate matching of categories and items [9]. or box embedding [6]. Another set of approaches comes from data perspective, such as node-copying for diverse sample graphs [8]. knowledge distillation from diversified teachers (T8), or rebalanced neighbor sampling (16). Although these methods make considerable progress, they rely solely on existing observed user-item interactions, without supplementary knowledge. However, due to inherent biases in the data itself (e.g., exposure bias, selection bias (10). it may not be sufficient to comprehensively infer user preferences. Consequently, to improve recommendation diversity, these methods can carry a higher risk of irrelevant suggestions, leading to a degradation in recommendation relevance, which contradicts the fundamental goal of accuracy for recommender systems.
Ideally, recommender systems are expected to capture comprehensive aspects of user interests based on observed user behavior, thereby providing a diversified set of recommendations. Nevertheless, due to biases, user behavior may not accurately reflect user interests. Here, we consider two key scenarios. First, due to limited system recommendations, items matching user's certain interests may not have been interacted with, causing lack of feedback, known as exposure bias. For example, a user with an interest in travel may like tourism posts on social media, but they may also be interested in photography tips that haven't It appeared in their feed yet. Second, due to individual differences, users adopt different interaction strategies, leading to selection bias [TO]. For instance, some users may primarily rate the shoes they purchase while leaving everyday items unrated, which does not imply lack of interest in those items. As such, data bias makes difficult for conventional methods to fully capture users' underexplored preferences without external knowledge, which may be overshadowed by dominant preferences in the observed data or may not yet appear, leading to narrow suggestions
that cater only dominant interests and placing users in "rabbit hole" [17 19]. To mitigate this issue, we aim to investigate the task of diversified recommendation.
In this paper, we explore the feasibility of enhancing recommendation diversity while minimizing relevance loss by leveraging the domain knowledge provided by large language models (LLMs). As LLMs demonstrate unprecedented capability in zero-shot inference using world knowledge, our key insight is to leverage their expertise to analyze the rationale behind user historical interactions and uncover underexplored preferences, thus addressing the negative impact diversity from data bias. For instance, a user who frequently browses travel and transportation posts on social media can be profiled travel enthusiast by the LLM, which can further infer potential interests in local cuisines or photography. In doing so, we strive to unbiasedly recover user preferences, thereby facilitating diverse yet relevant enhancements. Nevertheless, achieving this goal entails several challenges.
The first challenge is how to effectively leverage LLMs to capture a user's underexplored preferences from biased observations. Existing works using LLMs for diversified recommendations often fall short in systematic, fine-grained analysis of user preferences (20 Some approaches directly match user history with items (20) or infer preferences at the category level (21). We argue that coarse-gras -aasse-raiied matching is unsuitable for exploring latent preferences in diversity enhancement, as broad preferences may introduce noise, thereby diminishing relevance. To solve this, we design Tree of Preferences (ToP), which models user preferences from coarse to fine, to help the LLM better analyze the rationale behind user historical behaviors and improve the inference of latent interests.
The second challenge is how to leverage the uncovered preferences to guide diverse and relevant recommendations for users. A straightforward solution is to leverage LLMs to generate items in the embedding space [22), such as a two-step grounding paradigm [21] 23 but it suffers from suboptimal inference latency. Another is to have LLM rank items within the candidate set, either the full item set or a subset narrowed by external aids 220 24), but its performance depends heavily on the size or quality of the candidates. To address this challenge, we adopt a data-centric approach, where candidate items matching latent user preferences are identified via the LLM. We then generate
2

---

synthetic interactions that best reflect user underexplored interests and integrate them into general recommender for training. Moreover, we speed up the efficiency by dynamically selecting influential users during the optimization process. Our contributions are as follows:
We propose ToP-Rec, novel approach that explores diversified recommendation from data-bias perspective, aiming to enhance diversity while maintaining relevance with expertise from LLMs.
We design Tree of Preferences to model fine-graing grained user interests, serving as vehicle for LLMs to uncover underexplored preferences from observed behaviors. Synthetic interactions are generated to supplement existing data, training general recommender for diversified suggestions.
Extensive experiments three real-world datasets show that ToP-Rec Rec achieves advantages in both diversity and relevance in most cases, with a dominant trade-off and efficient inference latency compared to baselines.
2
Preliminary
The recommendation diversity referred to in this paper measures the dissimilarity of items rec- ommended to an individual user [16]. A closely related but orthogonal concept is the novelty of recommendations (25), also referred to as serendipity, popularity bias, or even diversity in some works
15 26 28 For consistency, refer to this concept as novelty in this paper. Novelty measures the proportion of long-tail or unpopular items among the recommendations for different users [15 25 Given the fundamental difference between diversity and novelty, this paper focuses enhancing recommendation diversity, excluding novelty from its scope.
Diversity-relevance trade-off in recommendation. The trade-off between recommendation di- versity and relevance has been extensively studied prior work (I] 6 16). In essence, this phenomenon arises from the fact that introducing dissimilarity may lead to additional noisy recom- mendations that are irrelevant to the user. Nevertheless, we emphasize that dissimilar items are not necessarily irrelevant; they may harbor implicit correlations by sharing common aspects aligned with user potential preferences. For example, list comprising sneakers, sports socks, a smartwatch, and yoga mat is generally more relevant to fitness enthusiasts than list comprising office supplies, home decor, and kitchenware. Therefore, latent user preferences can captured, enhancing diversity while preserving relevance becomes feasible. In this work, we investigate the balance between diversity and relevance recommender systems with the aid of LLMs. With this in mind, we next present formal definition of the target problem.
Problem formulation. Let 24 denote the set of users and I denote the set of items. R C is the interaction set between users and items. In this paper, we assume each user u E U is associated with attribute set A., and each item i I is associated with attribute set A:, where attributes are described by natural languages. In this context, recommender system aims to maximize the diversity among the items suggested to each user while promoting their relevance. Formally, define this as the diversity-relevance aware recommendation problem.
Problem (Diversity-relevance aware recommendation). Given a user set U with attributes (Aulu W), an item set I with altributes {Aili E Τ), and an interaction set R. C U x II the diversity- relevance aware recommendation aims to learn scoring function f x I R with respect to the following two objectives:
Objective 1: relevance. The recommender aims to maximize the relevance of the recommended items for each user u, i.e., maxx Rel(u, X)), where X I is the set of recommended items for user typically comprising items with the highest scores assigned by the recommender. Rel(u, X) is relevance metric for item set X and user u, such as recall.
Objective 2: diversity. The recommender aims to enhance the diversity of the recommended items for each user u, ie., maxxçr Div(X),, where Div(X) a diversity metric for item set X, such as category entropy.
3

---

Nickname
Visual
Photography Tips
Diversity
Implicit preference
Profile
Tree
Arts
Relevance
Age: 24 Gender. female
Transport
Airborne
Gen.
Fix: posks inclimbad, unchortcal My hungers.
Inter.
SF
Daily Life
likes
Root
Cuisine
Item Selection
Fisunmer escape
Urban
Item matching
suanrry
Kail Travel the Charming Coontal
Observed preference
Leisure
Traveling
Nature
Cand. Items
Obs.
Inter,
Figure 1: Illustration of our approach: Tree of Preferences for diversified Recommendation (ToP- Rec). Given "Alice" with her attributes and interacted items, ToP-Rec infers her rationale along the constructed ToP by prompting LLMs to comprehensively uncover her preferences until reaching the leaf nodes. Items aligned with her preferences are then matched, and synthesized user-item interactions concerning diversity and relevance are generated and integrated with the observed interactions. The combined data enables the recommender to offer diversified suggestions.
3 Methodology
In this section, we detail the design of ToP-Rec. We begin with overview, then introduce the two steps of ToP-Rec, where user's latent preferences are first unveiled ($..2) to guide the generation of synthetic interactions
3.3)
We further discuss a strategy for scaling up our approach in 3.44
3.1 Overall framework
Given users' textual attributes and their interaction history. ToP-Rec aims to generate diverse and relevant recommendation lists for them. We provide illustration in Figure [I]
(1) ToP-Rec first uncovers latent user preferences by constructing ToP (black arrows) and inferring latent preferences through systematic reasoning on ToP (dashed lines);
(2) ToP-Rec generates synthetic interactions that recover the preference distribution by identifying items aligned with inferred preferences and selecting those that enhance diversity and relevance.
(3) Synthetic interactions integrated with the original, and general recommender is trained the combined data. To scale up. ToP-Rec dynamically selects users for synthetic interaction generation.
3.2 Uncovering underexplored preferences
Despite LLMs' domain knowledge, uncovering user's unexplored preferences from biased historical interactions remains challenging. Existing works use LLMs to analyze user preferences based on history, yielding promising results [20] 21 However, these approaches lack systematic analysis of fine-grained user interests, which are essential for providing diverse recommendations while main-
taining relevance. Inspired by ToT 229 33). we propose establishing hierarchical user preference structure to enable the LLM to reason the rationale behind user historical behaviors from coarse to fine, ultimately uncovering user's unexplored preferences.
Constructing Tree Preferences over items. First, we discuss how to construct a hierarchy tree representing user preferences. Essentially, this process involves progressively dividing the space of user preferences over items. Figure [ shows an example partition social media context, where user interest in travel posts is initially separated from leisure, and subsequent partitions lead to fine-grained preferences like nature or urban tours. Such hierarchical structure serves as a vehicle, embodying the LLM's systematic reasoning of the user's rationale behind their behavior.

---

Starting from the root node, we instruct the LLM to partition the preference space progressively until a fine-grees gree-gra division is achieved. To ensure sufficient knowledge of item distribution, we use k-means-based method to sample smaller, text-rich subset of items S C I for the LLM.. Specifically, we first encode each item's text-based attributes using pretrained language model (e.g., BERT (3T).. To avoid excessive semantic redundancy, we then perform k-means clustering on the item embeddings and randomly sample items from each cluster. The sampled items are aggregated as the item subset S. Then, the construction of ToP can be formally represented as
T(V,E) LLM
where V denotes the node set of hierarchical preferences, T denotes the tree of preferences, and C V x V denotes the edge set. Each node a E V represents a kind of user preference generated by LLM, denoted by token sequence (Tokeno, Token J. Each edge e E & represents a finer preference division from its parent node. Promptrop means the instructions for constructing ToP (see Appendix A.2p. In practice, if the item set has predefined categorization, it will also be provided as a reference the prompt of ToP.
Note that the tree of preferences is constructed as a global, one-time structure shared across all users. This design stems from the insight that while individual preferences diverge, all users navigate the same underlying preference space shaped by item attributes.
Capturing user latent preferences via rationale reasoning. Next, we explain the systematic reasoning behind the user's 's behavior. Given their historical interactions, the L.L.M performs a top- down exploration of the hierarchical preferences, identifying the coarse-to-fine path that best matches the user's behavior (see dashed lines in Figure This enables a systematic analysis of user rationale and fine-grained preferences. The LLM also reevaluates the exploration with respect to the user's rationale, checking for unobserved preferences not reflected in the interaction history.
Based on user interactions, breadth-firs, search algorithm is used. Starting from the root node, the LLM selects preferences s' that best explain the user behavior at each level, storing the corresponding nodes. As moves to the next L- 1-level, only the stored nodes' children are activated. continuing until the leaf node. The LLM then summarizes the user's behavior rationale and revisits the exploration to check for unobserved preferences, which are added if found, following the same root-to-leaf path. Finally, the leaf nodes of all selected paths are returned. The process can be summarized as:
U.,
p, LLM(PromptpR (A., Ru)), fo,, } C VLeat
(2)
where (UI,.... 2n) denote the selected leaf nodes representing user complete preferences, Ry C R. denotes user u's interactions, and PrompteR is the instruction for preference reasoning.
3.3 Generating synthetic interactions
As we uncover undercxplored preferences, the next challenge lies in generating unbiased interactions based on them. We adopt a data-centric approach, leveraging data augmentation to mitigate potential biases in the observed feedback while ensuring efficient inference latency. First, candidate items that align with user preferences are identified. Then, items that cover underexplored preferences while considering relevance are selected and incorporated into existing user interactions. The synthetic unbiased interactions are ultimately used to train diversified recommender.
Matching candidate items with Tree of Preferences. To find candidate items matching the preferences of any user, we aim to assign items from the entire set to the corresponding leaf nodes in ToP (cf. Eq. 2 To reduce repeated computation, we pre-assign each item to its best matching leaf node by providing the item 's textual attributes A; to the LLM, which identifies the suitable leaf node in ToP. Finding candidate items that match specifie preferences can then be easily achieved. The pre-assigmment of each item is represented as follows:
i = (PM(PropttM((A)) v. E VLeaf:
where vi denotes the assigned leaf node of item via LLM and Promptım denote the prompt for item matching, Note that this process can be completed once ToP is constructed (see $ $3.2). and we present here for clarity. To improve the load imbalance of leaves, we also incorporate refinement mechanisms after assignment. Please refer to Appendix A.2 for detailed description.
5

---

Data generation for debiasing user interactions. Next, we select items that cover underexplored preferences, which are either overlooked or not yet manifested the user's behavior. To do this, we calculate each item's contribution to diversity by measuring its impact on debiasing. Intuitively, selecting an item that reflects a latent preference with low (no) occurrence in the user's history has higher impact. Thus, the diversity score of an item i is defined as Sdiv (u, i) o 1/freq, where freq; represents the frequency of the preference associated with item i in u's history. We also measure the relevance of each item by calculating its semantic alignment with the user: Sral (u, i)
(Enc(An); Enc(A:)), where Enc(-) denotes a pretrained language model (e.g., BERT (3]) and (: denotes cosine similarity. Finally, the overall score is computed as:
suu i) = (1 - >) (rel(u, i) + À Sdiv (u, i),
where À is a hyperparameter used to balance relevance and diversity. For a given user, we calculate the scores of all candidate items and select those above a predefined threshold. To reduce bias and better reflect user interests, the selected items are added to the user's s interaction history, obtaining synthetic interactions R2 R$ R where R$ represents the selected items. Finally, the synthetic interactions are used for training a general recommender, leading to diverse yet relevant performance.
3.4 Cost-efficient interaction generation
Limited by token throughput and LLM latency, generating interactions for every user can be costly. To address this, we propose cost-efficient strategy that identifies influential users for interaction generation, balancing improvements with costs. Given the complexity of data and recommender designs, infeasible to design static heuristic to distinguish user importance, so we dynamically quantify each user's influence based on the recommender's feedback during training. To be specific, the recommender is initially trained on the original interactions. As user influence varies during training, we backtrack parameter updates and compute user influence at fixed intervals. The most influential users are selected to generate synthetic interactions, which are integrated into the training set. This continues until the model reaches peak performance. To quantify user influence, design a custom criterion based on gradient alignment, measuring each user's ' contribution by the alignment between their local gradient and the model's parameter trajectory.
Assume the recommender optimized using Bayesian Personalized Ranking (BPR) los (32], defined as: L = - -'nu (a..)ER. ((n In (Uui (uu)). where ý denotes similarity scores. The local loss incurred by user u is: l(u; 0) - E(ujyER, In (Imi - Muj). Based on this, we define a user's influence in the gradient descent process over i steps.
Definition (k-stepp influence). Given the local gradient Vl(u; 9) of user u, and the gradient descent trajectory of model parameters (8"....0) backward from step t, the k-step infuuecc of user is defined as Inf,
11-1).
With numerous users high-dimensional gradients, the computational cost of user inffuence increases. We apply gradient dimension reduction (33 and group users to compute the group influence. More theoretical and empirical analyses are provided in Appendix A.2 and A.4
Discussion with existing work. (I) Conventional diversified recommendations adopt various solutions to capture user preferences, such as uncertain masking (6). contrastive context learning (18), and user-category matching (9), which rely solely on observed data. However, due to inherent data bias, they may fail to fully capture preferences. In contrast, our approach moves beyond the scope of observed data, leveraging world knowledge from LLMs to reason about user rationale, offering greater potential to enhance diversity. (2) LLMM-asee diversified recommenders propose reranking solutions (20). or use LLM fine-tuning 22 to capture user preferences for item genres (2I). However, these approaches focus affect accuracy. In contrast, our ToP models user preferences in coarse-to-fine manner, facilitating nuanced reasoning over user rationale for better diversity and relevance.
coarse category level, which can lead noisy recommendations and
4 Experiments
In this section, we evaluate the performance of ToP-Rec through extensive experiments. Due to space limitations, please refer to Appendix A.3 and A.4 for more experimental settings and results.
"Note that our method can be applied to other types of losses, such as binary cross-entropy loss.

---

4.1 Experimental setup
Datasets. We use the Twitter 11 Weibo 12 and Amazon 14] datasets. Twitter and Weibo are social network datasets with user attributes (e.g., username, location, bio) and posts as items, including attributes like retweet counts and content. User feedback consists of likes and retweets. Amazon is an e-commerce dataset, where we combine seven categories as in [34]. For each dataset, we follow [16) to extract a subset of the original, then drop items with missing attributes and apply 10-core filters (5-core for Twitter), resulting in a dataset that retains relatively informative and active entities. For each user, split their interactions into train, validation, and test sets with ratio of 0.6:0.2.0.2. See Appendix [A.3 for details and statistics of datasets.
Following prior work (24), we also enhance the quality of user and item textual attributes in these datasets using L.LMs. Specifically, we leverage an LLM to generate user summaries based on their profile and the textual attributes of previously interacted items, as well as to generate item summaries from their textual attributes. By generating these summaries, we aim to reduce noise, redundancy, and inconsistencies in the original text, leading to improved summaries that retain key semantic information.
Evaluation metrics. To evaluate the relevance of recommendations, we follow [T6] and adopt the metric @ (R @ k), indicating the proportion of relevant items retrieved in the top-k recommendation list. To assess diversity, we use the Category-Entropy @ k (CE@k), which measures the distribution of different categories within the top-k list. We report k 50 and 100 in this work.
Baselines. We adopt nine baselines to compare with the proposed approach, categorized into three types: (1) Heuristic methods: Random, MMR [35), and DPP [36]; (2) Conventional diversity-
enhancing methods: Box/LCD-UC [6] and CDM [18]; (3) LLM-based diversified recommender: LLM4Rerank-AN L.MM4Reran-A/LLMM-Reerr -/L.L.. rations for all baselines are provided in Appendix A.3
[20] and LLMRec- 24]. Detailed descriptions and configu-
Implementation details. We implement LightGCN with 2 hidden layers and hidden size of 32, which is optimized using Adam optimizer with a learning rate of 5e-3. We also evaluate the performance of ToP-Rec sampling with a 1:50 ratio and use early stopping. For hyperparameters affecting diversity and relevance, search the number of selected leaves in [4, 7] (step size 1), number of augmentations per user in [3,99 (step size 2), and the item sampling weight À in [0.2, 0.8] (step size 0.2). We utilize Qw3B2.5-32B-- IIIstrcct [37] to complete tasks involving LLMs. To ensure fairesss, we employ the same LLM for our approach and all baselines involving LLMs. Experiments are repeated times to report the average performance with standard deviation.
other backbones (see Appendix A.4). We employ a random negative
All experiments are conducted on a machine of Ubuntu 20.04 system with AMD EPYC 7763 (756GB memory) and NVIDIA RTX3090 GPU (24GB memory), All models are implemented in PyTorch version 1 with CUDA version 11.8 and Python 3.. Our code is publicly available at wttpa//b...om//xx0///////oo/o Bttp://9666oo_RR____ee_ ReC_NIPS
4.2 Evaluation of performance
We first evaluate ToP-Rec's overall performance in terms of diversity and relevance. For faimess, we select balanced result for methods with adjustable hyperparameters and run other baselines with their original settings. For reranking methods like MMR and DPP, we use 10 times the top-k value the candidate list, and for LLM4Rerank-A and LLM4Rerank-AD. we use twice the top-k value due to instability. several insights: (1) ToP-Rec dominates in most cases, with only one suboptimal result, showing its ad advantage in both diversity and relevance. (2) LLM-based methods perform relatively well in
presents average recall and category-entropy comparisons, revealing
relevance, but improvement in diversity is limited, likely due to a lack of fine- ine-grrineed preference analysis, leading to redundant item selections. (3) LCD-UC and Box struggle with high relevance, as box embeddings increase similarity with irrelevant items. (4) Heuristies like MMR are hard to achieve a balance, excelling in one aspect while underperforming in another, as observed in [16].
7

---

Table 1: Comparison of performance on diversity (R@k) and relevance (CE@ k)) denotes the backbone model, and +/- indicates performance improvements or declines compared to the backbone. The optimal performance is in bold, and the second-best performance is underlined.
Twitter
Welbo
Amazon
R@:50
R@ 100
CE
CE
100
R@:0
R@0 100
CE@50
CE@ 100
RØ:50
R@ 100
CF
CF@ 100
LighiGCN"
0.0567
0.0830
1,3413
0.1052
0.1669
0.9905
1.0763
0.2105
0.5004
0.5609
Random
0,0494
0.0730
1.2954*
1.3475
0.0988
0.1577
0.9995
1.0801
0.2062
0.5184*
MMR
0.0540
0.0790
1.3078
1,3550
0.0990
0.1578
1.0011
1.1005
0.1371
0.2115
0.5141
0.5755
DPP
0,0467
0.0765
1.3048
1.3532
0.0963
0.1530
1.0362
1,1150
0.1283
0.2035
0,5181
0.5748
CDM
0.0562
0.0814
1
1,3461
0,1014
0.1620
1.0018*
1.0912*
0.1349
0.2103
0.5228
0.5816
Box
0.0527
0.0741
1.2844*
1.3407
0.0996
0.15877
1.0238*
1.1034*
0.1228
0.2019
0.5186
0.5844
LCD-UC
0.0517
0.0768
13154
1.3784
1038
0.1625
1.0211
1.0956
0,1295
0.2068
0.5202
0.5842
LLMRae-MMR
0,0558
0.0820
3.356
1.351
0,1041
0.1662
1.0246
L1182
0.1363
0.2113
0,5177
0.5836
LLM4Rc-A
0.0562
0.0827
1.2855
1.3424
0.1052
0.1656
0.9891
1.0745
0.1359
0,2049
0,5028*
0.5863
LLM4Re-AD
0.0560
0.0822
1,2864*
1,3466
0. 10404
0.16
1.0001
1.0823+
0.1332
0.2042
0,5131
0,5827
ToP
0,0586
0.0841 + 1,3775
1.382
0.1054*
0.1667
1.033**
1.1399
0.1380
0.2120
0.5298 + 0.5902
Relevance-diversity trade-off. To further demonstrate the robustness of our proposed method, we evaluate the relevance-diversity trade-off of ours and the baselines that support adjustments to balance this trade-off, including Random, MMR, DPP, CDM, LCD-UC, and LLM.ec-MMR. We tune the hyperparameters of each method to explore their trade-offs, selecting the best three trade-off points for each method and visualizing those for the Twitter dataset in Figure 2 The upper-right corner represents the ideal performance, with higher recommendation relevance and diversity. The result shows that our approach achieves the best trade-off compared with other baselines. In particular, while the diversity is enhanced with a larger margin, the relevance under our approach is consistently improved upon the backbone LightGCN (shown by a star mark). We also compare the trade-off's on Weibo and Amazon, which can be found in the Appendix A.4
1.41
LigheGCN
Random
MMR
Diversity (CE.)) (CE) (00)
DPP
1.39
CDM
LUD-UU
LIMRR-MMM
1.38
RoB
1.37
1.36
1.35
1.34
0.072
0.076
0.078
0.080 0.082
0.084
Relevance (R@100)
Figure 2: riveract trade-off compari- son. The upper-right represents the ideal.
Inference latency. To evaluate our method's ' efficiency, we compare inference time with other methods, measuring the average time to generate a recommendation list for a user. Figure Bla shows that our method achieves similar latency to traditional methods like CDM and LCD-UC, while outperforming the reranking method MMR. Notably, it also shows significant advantage over LLM- based method LLM4Rerank. In summary, our method can optimize diversity performance while maintaining efficient inference latency compared baselines.
4.3 Ablation study
We conduct ablation studies to evaluate the effectiveness of each component in our approach, including four variants: (1) w/o Div and (2) w/o Rel: We ignore the Item's contribution to diversity or relevance
in item selection (cf. Eq. Eq.44: (3) w/o US: We discard the influential user selection (cf. $ 3..4 and random select user for augmentation; (4) w/o PR: We avoid LLM to infer user preferences, instead randomly selecting leaf nodes (cf. (2) Figure 3bb) visualizes their changes in diversity and relevance, ordered by relevance in descending order. First, w/o Div and w/o Rel outperform Ours in relevance and diversity, respectively, but perform poorly in the other aspect due to considering only one factor. Second, w/o US improves both aspects but remains weaker than Ours, showing that augmentation on influential users boosts performance. Finally, w/o PR performs worst in relevance, indicating that ignoring user interests increases the risk of irrelevance.
8

---

LLM
0.085
1,38
0.085
LCDAUC
MMR
053.8x
1,38
0.084
0.084
1.37
0.084
1.4x
COM
1,36
0.083
0.083
Risttnn CE50300
-Rg0100
1.0k
Ous
1.58
CEs0100
00,082
0.082
CR50100
0.082
10
10a
10
10
10
wiJUS work
08
Timeteed
(a) Inference latency.
(b) Ablation study.
(c) Impact of gen. inter. (d) Impact of weight 2.
Figure 3: (a) Comparison of average time to generate recommendations; (b) Evaluation of each component in ToP-Rec; (c) and (d) Impact of generated interactions per user and selection weight A. We use dashed lines represent the performance of the backbone recommender.
4.4 Hyperparameter analysis
Next, evaluate the influence of important hyperparameters in approach, which impact the diversity-relevance performance. These mainly include: (1) the weight À for item selection. (2) the number of generated interactions per user. (3) the number of selected leaf nodes per We tune the weight À among (0.2,0 0.4, 0.6, J.8} and number of generated interactions among {3, 5, 7, 9). The results in Figure d) show that as the weight À for item selection increases, diversity rises while relevance declines. This is because a higher weight increases the focus on diversity, raising the possibility of irrelevance. Figure shows that the number of generated interactions increases, relevance decreases, while diversity initially rises and then stabilizes. This is due to the decline in item relevance, which reduces recall, while the contribution diversity also diminishes, stabilizing its growth. Due to page limitations, we analyze the impact of other hyperparameters in the appendix.
5 Literature Review
Diversified recommendation. Research on diversified recommendation has well-established history. Early works concentrate on re-ranking diversification 335 leverage greedy solutions to balance utility and diversity [35
36
42). These approaches often
39
40 employ Determinantal Point
Processes [43] to generate diverse recommendations by measuring set diversity (36 41 42) Recently, more complex diversified re-ranking methods have been proposed, such introducing behavior model to maximize knowledge diversity [17]. or employing graph clustering to capture user interests and sub-models assess diversity (T) Unlike these post-matching works, framework directly incorporates diversification the matching stage. Many recent works also focus on the matching stage, with some built upon Bayesian graph convolutional neural networks (8), multi-vector representations (7). two-stage category-item matching (9. or rebalanced sampling techniques (16). Compared to them, we propose a universal pipeline for with generic recommender backbones, instead of designing a specific model. Our work is most similar to 6 18] where they propose general framework for diversified recommendation via box embedding and uncertainty masking (6). or knowledge distillation learning from MMR [18]. A common limitation of traditional methods is their reliance on observed data, which hinders overcoming diversity decline due inherent data bias. In contrast, our approach moves beyond the scope of observed data, leveraging world knowledge from L.MSs to reason about the user rationale, offering greater potential to enhance diversity.
LLM-based recommendation. With the impressive capabilities demonstrated by LLMs 4M growing body of work explores their application in recommender systems [48) Early research primarily focuses on leveraging L.MSs to enhance the relevance of recommendations 24 54).
utilizing the reasoning abilities of I.LMs to analyze potential user interests and generate tailored recommendations. Recently, LLM-based recommendations have expanded beyond relevance, with a growing focus on other performance aspects [55], especially on recommendation diversity (20 To mitigate the homogeneity issue in LLM-based recommendations. the decoding strategy in 22 integrates a text-free assistant model to refine the token scores. However, its effectiveness depends on the assistant model's 's quality; if the model provides poor suggestions, may lead to irrelevant or low- quality recommendations. DLCRec (2I) introduces framework for diversity control in 1-M--ased recommendations by breaking down the recommendation task into three sub-tasks; L.M4RRerank [20) proposes a LLM-based reranking approach that leverages graph structure to represent accuracy, diversity, and fairness reranking, allowing for the refinement the final recommendations, Despite some improvement in diversity, these approaches lack fine-graines user preference analysis and item

---

space partitioning, leading to a higher risk of irrelevant recommendations and affecting accuracy. In contrast, Tree of Preferences hierarchically refines user preferences from coarse to fine, enabling the LLM to uncover underexplored user preferences through nuanced reasoning, thereby facilitating diverse and relevant recommendations.
Influential data selection. Existing research on influential data selection aims to estimate the influence of individual or sets of training records on model performance and select the most influential ones [33 58) In this paper, we primarily discuss two widely used approaches: influence function-
based methods [66] [966 and gradient descent tracing-based methods [33] 662 [64]. While influence functions provide insights into how perturbations to certain parts of the training data affect model behavior [65]. the computation of the Heverse-Hessian limits the effectiveness of the selection process [66]. Furthermore, existing work has pointed out that traditional influence functions may fail on certain types of data and model architectures. For instance, when applied to Graph Neural Networks (GNNs) (67 (70,, computing influence requires analyzing the mutual interactions between nodes 751 which adds significant computational overhead when using influence functions for user selection in many BoN-based recommendation models [76]. On the other hand, studies [62] utilize first-order approximation to estimate the influence of training sample on model performance under stochastic gradient descent. (33) extends this assumption the Adam optimizer. The methods in
63 (4) are closest to ours, calculating the alignment between the local gradient of pretrain samples and the downstream loss gradient. However, the downstream loss gradient may not align with the actual parameter update during fine-tuning. In contrast, we calculate the alignment between the local gradient of a sample and the actual parameter update trajectory, ensuring more accurate measure.
Broader impacts
Our approach improves recommendation diversity by uncovering underexplored user preferences through LLMs, addressing two important societal challenges in recommendation systems. First, reduces information redundancy by diversifying recommendations, helping users break free from repetitive suggestions and explore niche topics, which reduces the impact of algorithmic echo chambers. Second, by diversifying recommendations users are given more opportunities to encounter content they may not have come across but are likely to be interested in, thereby fostering the diversity of socictal culture. These improvements contribute more balanced, inclusive recommendation systems that prioritize user-driven discovery over algorithmic determinism.
7 Conclusion
In this paper, we explore diversified recommendation from data-bias perspective, identifying two key scenarios that may introduce bias affecting diversity. To address this, we propose ToP-Rec, which leverages external knowledge from LLMs to complement data bias. We construct Tree of Preferences to model user preferences from coarse to fine, helping the LLM analyze user behaviors and improve latent interest inference. To ensure efficient recommendations, candidate items matching latent preferences are identified via the LLM, and synthetic data is generated through a relevance- diversity-aware strategy for training. Additionally, we introduce a dynamic user selection mechanism to reduce costs by selecting influential users based on gradient feedback. We extensively evaluate the performance of Too--Rec on three real-world datasets, comparing to nine competitive baselines. The results demonstrate that ToP-Rec outperforms in most cases, achieving second-best performance in others, with the optimal trade-off between diversity and relevance and efficient inference latency.
Acknowledgments and Disclosure of Funding
This work is supported in part by Zhejiang Provincial Natural Science Foundation of China (Grant No. LMS2250200112), the Hangzhou Joint Fund of the Zhejiang Provincial Natural Science Founda- tion of China under Grant No.1HHZDD24F020001 Zhejiang Province High-Level Talents Special Support Program "Leading Talent of Technological Innovation of Ten-Thousands Talents Program" (20220 the Fundamental Research Funds for the Central Universities (2021FZZX001- 23), the advanced computing resources provided by the Supercomputing Center of Hangzhou City University, and CIPSC-SMP-Zhipl Large Model Cross-Disciplinary Fund.
10

---

References
[1] Yue Xu et al. "Multi-factor sequential re-ranking with perception aware diversification". In: Proceedings of the 29th ACM SIGKDD Conference knowledge discovery and data mining. 2023, pp. 5327-5337,
[2] Kartik Sharma et al. "A survey of graph neural networks for social recommender systems". In:
ACM Computing Surveys 56.10 (2024). pp. 1-34.
[3] Farah Tawfiq Abdul Hussien, Abdul Monem S Rahma, and Hala Bahjat Abdul Wahab. "Recom- mendation systems for e-commerce systems overview". In: Journal of Physics: Conference Series. Vol. 1897. 1. IOP Publishing. 2021, p. 012024.
[4] Zefeng Chen et al. "Data scareity in recommendation systems: A survey". In; ACM Transac- tions Recommender Systems 3.3 (2025). pp- 1-31.
[5] Qijiong Liu et al. "Multimodal pretraining, adaptation, and generation for recommendation: survey". In: Proceedings of the 30th ACM SIGKDD Conference Knowledge Discovery and Data Mining. 2024, pp. 6566-6576.
[6] Cheng Wu et al. "Enhancing Recommendation Accuracy and Diversity with Box Embedding: A Universal Framework". In: Proceedings of the ACM Web Conference 2024. 2024, pp. 3756- 3766.
[7] Shilong Bao et al. "The minority matters: A diversity-promoting promoting collaborative metric learning algorithm". In: Advances Neural Information Processing Systems 35 (2022), pp. 2451-24112 12244. [8] Jianing Sun et al. "A framework for recommending accurate and diverse items using bayesian graph convolutional neural networks". In: Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining. 2020, pp. 2030-2039.
[9] Tao Zhang et al. "On practical diversified recommendation with controllable category diversity framework". In: Companion Proceedings of the ACM Web Conference 2024. 2024, pp. 255- 263.
[10] Jiawei Chen et al. "Bias and debias in recommender system: A survey and future directions". In: ACM Transactions on Information Systems 41.3 (2023), pp. 1-39.
[11] Shangbin Feng et al. "Twibot-222 Towards graph- based twitter bot detection". In: Advances in Neural Information Processing Systems 35 (2022), pp. 35254-35269.
[12] Jing Zhang et al. "Social influence locality for modeling retweeting behaviors." " In: IJCAL.
Vol. 13. Citeseer. 2013, pp. 2761-2767.
[13] Chongming Gao al. "KuaiRec: A Fully-Observed Dataset and Insights for Evaluating Recommender Systems". In: Proceedings of the 31st ACM International Conference on Information & Knowledge Management. CIKM '22. Atlanta, GA, USA, 2022, pp. 540- 550. DOI: 10 1145/ 351 1801100.... 3557220
URL: https //ooi org/ 10. 1145 /445/351188
3557220
[14] Jianmo Ni, Jiacheng Li, and Julian McAuley. "Justifying recommendations using distantly- labeled reviews and fine-grained aspects". In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), 2019, pp. 188-197.
[15] Ziwei Zhu et al. "Popularity-opponte
bias in collaborative filtering". In: Proceedings of the 14th ACM international conference on web search and data mining. 2021, pp. 85-93.
[16] Yu Zheng et al. "DGCN: Diversified recommendation with graph convolutional networks". In:
Proceedings of the web conference 2021. 2021, pp. 401-412.
[17] Erica Coppolillo, Giuseppe Manco, and Aristides Gionis. "Relevance meets diversity: user- centric framework for knowledge exploration through recommendations" In: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2024, pp. 490-501.
[18] Fan Li et al. "Contextual Distillation Model for Diversified Recommendation" In: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2024, pp. 5307-5316.
[19] Muhammad Haroon et al. "Youtube, the great radicalizer? auditing and mitigating ideological biases in youtube recommendations" In: arXiv preprint arXiv:2203. 10666 (2022).
[20] Jingtong Gao et al. "LLM4Rerank: -ased Auto-Reranking Framework for Recommen- dations". " In: Proceedings of the ACM on Web Conference 2025. 2025, pp. 228-239.
11

---

[21] Jiaju Chen et al. "DLCRec: A Novel Approach for Managing Diversity in LLM-Based Recom- mender Systems". In: Proceedings of the Eighteenth ACM International Conference on Web Search and Data Mining. 2025, pp. 857-865.
[22] Keqin Bao et al. "Decoding matters: Addressing amplification bias and homogeneity issue for Ilm-bases recommendation". In: arXiv preprint arXiv:2406. 14900 (2024).
[23] Kegin Bao et al. "A bi-step grounding paradigm for large language models in recommendation systems". In: ACM Transactions on Recommender Systems 3.4 (2025), pp. 1-27.
[24] Wei Wei et al. "Limrec: Large language models with graph augmentation for recommendation".
In: Proceedings of the 17th ACM International Conference Web Search and Data Mining.
2024, pp. 806-815.
[25] Minghao Zhao et al. "Investigating eccuracy-novelyy performance for graph-based collaborative filtering". In: Proceedings of the 45th international ACM SIGIR conference on research and development in information reirieval. 2022, pp. 50-59.
[26] Li Chen et al. "How serendipity improves user satisfaction with recommendations? a large- scale user evaluation". In: The world wide web conference. 2019, pp. 240-250.
[27] Tao Zhou et al. "Solving the apparent diversity-acuuracy dilemma of recommender systems". In: Proceedings of the National Academy of Sciences 107.10 (2010), pp. 4511-4515.
[28] Chongming Gao al. "SPRec: Self-Play Debias LLM-based Recommendation", In: Pro-
ceedings of the ACM on Web Conference 2025. 2025, pp. 5075-5084.
[29] Shunyu Yao et al. "Tree of thoughts: Deliberate problem solving with large language models".
In: Advances in neural information processing systems 36 (2023), pp. 11809-11822.
[30] Jieyi Long. "Large language model guided tree-of-thought". In: arXiv preprint
arXiv:2305. 08291 (2023).
[31] Jacob Devlin al. "Bert: Pre-training of deep bidirectional transformers for language un- derstanding". In: Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume (long and short papers). 2019, pp. 4171-4186.
[32] Steffen Rendle et al. "BPR: Bayesian personalized ranking from implicit feedback". In: arXiv (2012).
preprint 1205. 02...
[33] Mengzhou Xia et al. "Less: Selecting influential data for targeted instruction tuning". In: arXiv preprint 2:Xx:22402204333 (2024).
[34] Jiacheng Li et al. "Text is all you need: Learning language representations for sequential recommendation" In: Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2023, pp. 1258-1267.
[35] Jaime Carbonell and Jade Goldstein. "The use of MMR, diversity-based reranking for reorder- ing documents and producing summaries". In: Proceedings of the 2 annual international ACM SIGIR conference on Research and development in information retrieval. 1998, pp. 335- 336.
[36] Laming Chen, Guoxin Zhang, and Eric Zhou, "Fast greedy map inference for determinantal point process to improve recommendation diversity". In: Advances in Neural Information Processing Systems 31 (2018).
[37] Qwen Team. Qwen2.5: A Party of Foundation Models. Sept. 2024. URL: httth::t///:/eeets cithub.io/bloo http///llggg/....eeeeeee qwen2.5/
[38] Lijing Qin and Xiaoyan Zhu. "Promoting Diversity in Recommendation by Entropy Regular- izer." In: JJCAI. Vol. 13. Citeseer. 2013, pp. 2698-2704
[39] Cai-Nicolas Ziegler et al. "Improving recommendation lists through topic diversification". In: Proceedings of the 14th international conference on World Wide Web. 2005. pp. 22-32.
[40] Azin Ashkan et al. "Optimal Greedy Diversity for Recommendation." In: IJCAI. Vol. 15. 2015,
pp. 1742-1748.
[41] Lu Gan et al. "Enhancing recommendation diversity using determinantal point processes on knowledge graphs". In: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2020. pp. 2001-2004.
[42] Romain Warlop, Jérémie Mary, and Mike Gartrell. "Tensorized determinantal point processes for recommendation" In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining. 2019, pp. 1605-1615.
12

---

[43] Odile Macchi. "The coincidence approach to stochastic point processes". In: Advances in Applied Probability 7.1 (1975), pp. 83-122.
[44] Yu Wang et al. "Adaptive Location Hierarchy Learning for Long-Tailed Mobility Prediction". In: arXiv preprint arXiv:2505. 19965 (2025).
[45] Wenkai Fang al. "SeRL: Self-Play Reinforcement Learning for Large Language Models
with Limited Data". In: arXiv preprint arXiv:2505.20344 (2025).
[46] Shunyu Liu et al. "A survey of direct preference optimization". In: arXiv preprint
arXiv:2503. 112003117000 170/ (2025).
[47] Tianli Zhang al. "Language Models-enhanced Semantic Topology Representation Learning For Temporal Knowledge Graph Extrapolation". In: Proceedings of the 33rd ACM International Conference on Information and Knowledge Management. CIKM '24. New York. NY, USA:
Association for Computing Machinery, 2024, pp. 3227-3236. ISBN: 9798400704369, DOI: 10. 11.11/63627663.. 3679602 URL: battps://dop org/ 10. 1145/36276777 3679602
[48]
Zihuai Zhao al. "Recommender systems in the era of large language models (Ilms)". In:
IEEE Transactions on Knowledge and Data Engineering (2024).
[49] Qiyao Peng et al. "A Survey on L...-powere. Agents for Recommender Systems". In: arXiv preprint arXiv:2502. 10050 (2025).
[50]
Likang Wu et al. "A survey on large language models for recommendation" In: World Wide Web 27.5 (2024). p. 60.
[51] Junjie Zhang al. "Agentef: Collaborative learning with autonomous language agents for recommender systems". In: Proceedings of the ACM Web Conference 2024. 2024, pp. 3679- 3689.
[52] Zewen Long et al. "GOT4Rec: Graph of Thoughts for Sequential Recommendation". In: arXiv preprint arXiv:241 14922 (2024).
[53] Xinfeng Wang et al. "Rdrec: Rationale distillation for llm-based recommendation" In: arXiv preprint arXiv:2405.1055. (2024).
[54] Jun Hu et al. "Enhancing sequential recommendation via Ilm-based semantic embedding learning". In: Companion Proceedings of the ACM Web Conference 2024. 2024, pp. 103-111.
[55] Chumeng Jiang et al. "Beyond Utility: Evaluating LLM as Recommender". In: Proceedings of the ACM on Web Conference 2025. 2025, pp. 3850-3862,
[56] Yuzheng Hu et al. "Most influential subset selection: Challenges, promises, and beyond". In: Advances in Neural Information Processing Systems 37 (2024), pp. 119778-119810.
[57] Jiarong Xu et al. "Better with less: A data-active perspective pre-training graph neural networks". In: Advances in neural information processing systems 36 (2023), pp. 56946 56978.
[58]
Yuxuan Cao et al. "When pre-train graph neural networks? from data generation perspective!" In: Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2023, pp. 142-153.
[59] Zichun Yu al. "Data-Efficien Pretraining with Group-Level Data Influence Modeling". In:
arXiv preprint arXiv:2502.14700 (2025).
[60] Pang Wei Koh and Percy Liang. "Understanding black-box predictions via influence functions".
In: International conference on machine learning. PMLR. 2017, pp. 1885-1894.
[61] Zichun Yu, Spandan Das, and Chenyan Xiong. "Mates: Model-aware data selection for efficient pretraining with data influence models". In: Advances in Neural Information Processing Systems 37 (2024), pp. 108735-108759.
[62] Garima Pruthi et al. "Estimating training data influence by tracing gradient descent". In: Advances in Neural Information Processing Systems 33 (2020), pp. 19930.19930.
[63] Xiaochuang Han al. "Understanding in-context learning via supportive pretraining data". In: (2023).
arXiv preprint 233..
[64] Xiaochuang Han and Yulia Tsvetkov. "Orca: Interpreting prompted language models via locating supporting data evidence in the ocean of pretraining data". In: arXiv preprint arXiv:2205.1266 (2022).
[65] Juhan Bae et al. "If influence functions are the answer, then what is the question?" In: Advances in Neural Information Processing Systems 35 (2022), pp. 17953-17967.
[66] Roger Grosse et al. "Studying large language model generalization with influence functions". In: arXiv preprint arXiv:2308.03299 (2023).
13

---

[67] Yuwen Wang et al. "Unveiling global interactive patterns across graphs: Towards interpretable graph neural networks". In: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2024, pp. 3277-3288.
[68] Jie Yang et al. "From GNNs to Trees: Multi- Granular Interpretability for Graph Neural Networks". In: The Thirteenth International Conference on Learning Representations. 2025.
[69] Yu Wang et al. "Spatiotemporal-Augmie Augmented Graph Neural Networks for Human Mobility Simulation". In: IEEE Transactions on Knowledge and Data Engineering 36.11 (2024),
pp. 7074-7086.
[70] Yu Wang et al. "Spatiotemporal gated traffic trajectory simulation with semantic-aware graph learning". In: Information Fusion 108 (2024), p. 102404.
[71] Jiancan Wu et al. "Gif: A general graph unlearning strategy via influence function". In:
Proceedings of the ACM Web Conference 2023. 2023, pp. 651-661.
[72] Hanyang Yuan et al. "Can Graph Neural Networks Expose Training Data Properties? An Efficient Risk Assessment Approach". In: arXiv preprint arXiv:241 1.03663 (2024).
[73] Kun Wu et al. "Certified edge unlearning for graph neural networks". In: Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2023, pp. 2606- 2617.
[74]
Jiarong Xu al. "Blindfolded attackers still threatening: Strict black-box adversarial attacks on graphs". In: Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 36. 4. 2022, pp. 4307.
[75] Jiarong Xu et al. "Unsupervised adversarially robust representation learning on graphs". In: Proceedings of the AAAI conference on artificial intelligence. Vol. 36. 4. 2022, pp. 4290-4298.
[76] Shiwen Wu et al. "Graph neural networks in recommender systems: survey". In: ACM Computing Surveys 55.5 (2022), pp. 1-37.
[77] Guillaume Garrigos and Robert M Gower. "Handbook of convergence theorems for (stochastic) gradient methods". In: arXiv preprint arXiv:2301. 12335 (2023).
[78] Xiang Wang et al. "Neural graph collaborative filtering". In: Proceedings of the 42nd interna- tional ACM SIGIR conference Research and development in Information Retrieval. 2019,
pp. 165-174. [79] Abhimanyu Dubey et al. "The Ilama 3 herd of models". In: arXiv e-prints (2024), arXiv-2407.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We clearly outline the contributions and scope of the paper in both the abstract and the introduction sections.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The limitations are comprehensively discussed Appendix A.5
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
15

---

We conduct theoretical analysis and provide the assumptions and proof in Appendix A..
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The experimental details for reproducing the main claims and conclusions are provided in $4.1 and Appendix A.3
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
16

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The data is collected from open-source sites, with references provided in the main text, and the code is available via link.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
//nips.ccpublle wwwppppgpuuiiieeeee hetpi///guidessoeeeuuubiiii
(CodeSubiisss wwtaaabmmsssooni ssiiiiolcccy for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The experimental settings and details described in $4.1 and Appendix A.3
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: The standard deviations of different methods are reported in Appendix A.4
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for
example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
17

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma enror bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The complete computational resources used in our experiments are reported in
41
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics atttps:/neerrpss httpp//wwwpss.pppcceeeeeeeeeeeee http://p.....ppoooeccccccccccaea
Answer: [Yes]
Justification: The research presented in this paper adheres fully to the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Broader impacts of this work are thoroughly discussed in
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
18

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
This work does not involve such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
The data is available under a public access license, and the code and models utilized in this paper comply with the legal usage of sources.
Justification:
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
19

---

If assets are released, the license, copyright information, and terms of use in the thcode. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
batasets.papersstt
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is available online, the authors encouraged to reach out the asset's s creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer:
[Yes]
Justification: The implementation code for this work is provided through link along with introductory steps.
Guidelines:
The answer NA means that the paper does not release assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
This work does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
This work does not involve research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
20

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [Yes]
Justification: We use an LLM to reason user preferences based on our constructed tree of preferences, with detailed description of LLM usage provided to facilitate the reproduction our method.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non non-standard components.
Please refer to our LLM policy for what should or should be described.
21

---

A Appendix
A.1 Notations
Table 2: Description of major notations, ordered by appearance.
Notation
Description
U,I.R
user set, item set, interaction set user, item, attribute set user or item scoring function
u, u, A
X, Div()
set of recommended items, relevance metrie, diversity metric set of sampled items for constructing ToP input prompt for LLM, output of LLM tree, node set of T, edge set of T node of T, edge of T user u's interactions cosine similarity, pretrained language encoder
S
Prompt, LLM(-)
T.V.E
Ru
(, Enc(-)
freq;
frequency of the preference associated with item i user u's history item selection score, hyperparameter synthetic interactions, generated interactions from selected items loss of recommender, parameter of recommender user local loss, gradient influence of user
s(u, i), A
R', R
2
(( Inf,
A.2 Supplementary the method
Illustration of prompts. We provide an illustration of the prompts used to complete the essential processes of ToP-Rec, as summarized in Figure
Details of ToP refinement. To find candidate items matching the preferences of any user, we preassign every item to a leaf node in ToP (cf. E3)()). However, this assignment may lead to slight load imbalance, with varying numbers of items assigned different leaf nodes. Empirically, we observe that the imbalance can carry over into the recommended items. Therefore, after assigning all items, we introduce refinement mechanism to improve load balance, involving two operations.
(1) Merge. Underloaded sibling leaves are merged to create load-balanced leaf. This is achieved by integrating the assigned items and instructing the LLM summarize their corresponding preferences, thus forming merged leaf.
(2) Split. Leaves with excessive load split into several boad-blannced leaves. Given excessively loaded leaf, the LLM is prompted to further refine the corresponding preference into several finer- grained ones, reassigning the items accordingly.
In practice, underloaded and excessively loaded leaves are identified using two predefined lower and upper thresholds, enabling self-stabilizatio through refinement. To prevent infinite refinement, we set the maximum number operations as 10.
Theoretical analysis of user selection. To quantify user influence based on gradient alignment, we assess the alignment between each user's local gradient and the model's 's parameter trajectory during gradient descent (see Definition 1). The intuition is that the stronger the alignment between user's ' optimization direction and that of the recommender model, the greater their contribution to the model's 's improvement. Here, we present the theoretical analysis supporting our approach, starting with theorem that establishes the upper bound of the convergence step in gradient descent (GD).
Theorem A.1. Let L(() be convex loss function with leurnable parumeter 0, and let 0* E arg ming L()) denote the optimal parameter. Define the initial error eq == L(00) - L(("", and let v V((') denote the gradient at optimization step t. Assume L is B-Lipschitz smooth. Then the number
22

---

Example prompts used for ToP construction, preference reasoning, and item matching.
Notation: PrompttaP.
Usage: ToP construction.
Content: You are an expert system designed to classify and organize user preferences based on the following information of item examples. Your task is to generate multi-level tree structure for user preferences, called Tree of Preferences (ToP), progressively refining them from broad to specific preferences. Each node represents type of user preference, and each edge signifies finer preference division from its parent node.
Constraints:
Each node should be divided into 3-5 finer preferences (branches), except for leaf nodes.
Use diverse inferenceedividiig criteria each level.
Nodes must represent clear, actionable preferences.
Items samples: {sampled seppledditess____ ttt___ittm____nrmiee information]. Return ToP in the following format: "ToP_forman ).
Notation: PromptpR-
Usage: Preference reasoning.
Content: You are expert system designed to capture user latent preferences through rationale reasoning. You will be provided with the user's profile, observed interactions with items, and a multi-level tree structure representing different user preferences, organized from broad to specific preferences (Tree of Preferences, ToP). Using the profile and interactions, your task is to identify the user's latent preferences by exploring the ToP top-down, following the coarse-to-fine paths that best match the user's behavior and reasoning the rationale be- hind their actions, while identifying any unobserved preferences not reflected in the history.
Constraints:
Perform breadth-firs search. At each level, select the preferences that best match the user behavior, storing the corresponding nodes.
For subsequent levels, activate only the child nodes of the stored nodes at the previous level, continuing the selection until reaching final level.
Reevaluate the exploration to identify any unobserved preferences, adding them if found, following the root-to-leaf path.
Control the total number of selected paths [number_of_patteee The final output should be the leaf nodes of all selected paths.
User profile: [user_ profilee........o
User historical interactions: {user interactions).
ToP: ).
Return selected leaf nodes in the following format: (leaf_nodes formttt))) along with a concise explanation of each selection in the format of creason_format,
Notation: PromptIM.
Usage: Item matching.
Content: You are an expert system designed to match items with user preferences. You will be provided with items and their specifie information, along with multi-level Tree of Preferences (ToP) representing user preferences, structured from broad to specific divisions, where each node represents preference type and each edge further refines the preferences. Your task is to find the most relevant preference for each item in ToP. moving through the levels of ToP based the item's information, and return the final leaf node.
Constraints:
At each level. select only the most appropriate node from the available options.
For the next level, select only from the child nodes of the node selected in the previous level.
ToP: [ToP_ )
Items information: fitems information). Return the selected leaf node in the following format: (leaf _ode__noee____ format]
Figure 4: Illustration of the prompts used in this work.
23

---

Table Dataset statisties.
Dataset
Interactions
Users
Items
% Density
Twitter
40,223
2,118
7,199
0.2639%
Weibo
661,783
14,663
5,711
0.7903%
Amazon
134,857
3,598
8,747
0.4285%
of steps T required to achieve L(OT) L()") S satisfies:
23(en 5)
T minogigs [V
where B = -
Proof. Since L is convex function with 3-Lipschitz smoothness, obtain the following inequality: L(0') < (()) (L( 0) + / 11' 0112. Suppose the learning rate is a, substitute the gradient descent update Gt+1 O² C(("):
a²9 |V2(0)
C(O++1) S (())) a. V2 + 2
Choosing the optimal learning rate (maximizing the descent under smoothness), we can
obtain:
(((++1) < C(0') 2 |
(7)
The decrease in the objective function per iteration has the lower bound:
L(0') - 2(0 2
|V2(0')1
(8)
23
C| and from 77 we know that given
Since each epoch reduces the error by at least
2
the convergence rate of GD method is O(!), thus the minimum number of epochs T satisfies
T' <
2,(eg-cl
where = (
mings [
Theorem demonstrates the inverse relation between the convergence step and the gradient norm, which inspires our definition the user influence. We use the inner product summation to measure the user influence, which essentially prioritizes projection onto the global update trajectory. Users with high influence contribute more significantly to increasing the gradient norm's magnitude, thereby enhancing the training efficiency and convergence performance according to Theorem 1.
whose gradients exhibit persistent positive
A.3 More experiment settings
Dataset details.
The statisties of the datasets used in this work summarized in Table
Twitter (T]:
This dataset is originally collected from Twitter for bot detection. We remove bot users, considering human tweet user as user and tweet as item. We consider user behaviors such as likes and retweets as interactions. User attributes include name, description, location, and more, and item attributes include content, retweet count,
This dataset is collected from Weibo, one of China's largest social media platforms. We treat microblog posts as items and reposts as interactions. User attributes include name, gender, number of followers, etc. Item attributes include content and repost count.
Weibo 12):
Amazon 14] Amazon is an e-commerce dataset. Following (34), seven categories are combined: Automotive, Cell Phones and Accessories, Clothing, Shoes and Jewelry, Electronics,, Grocery and Gourmet Food, Home and Kitchen, and Movies and TV.. Attributes include item category, brand, features, description, user reviews, and others.

---

Baseline details. To ensure fairesss, we use the same recommender backbone for ToP-Rec and the baselines. For reranking methods like MMR and DPP, we found that using 10 times the top-k value as the candidate list can bring the performance close to its peak. For I.M.4erann--A and L.MM4Rerank-AD, we use twice the top-k value due to their instability. We categorize the nine comparative methods used in this work into three types.
Heuristic methods:
Random: Randomly generate a fixed number of user-item interactions and add them to the original history interaction set, then train the recommender.
MMR 35
heuristic algorithm optimizing the trade-off between relevance and diversity, selecting items that are pertinent to the query and minimally redundant.
DPP 36 DPP is a parametric model for selecting a diverse subset from a larger pool of items. (36 accelerates DPP by proposing an algorithm that significantly speeds up the greedy MAP inference.
Conventional diversity-enhancing methods:
Box 6 A framework that enhances diversity and accuracy using box embedding to create hypereubes for users and items, with a similarity scoring function measure their relationship.
LCDDUC [6]: To better balance accuracy and diversity, LCD-UC adds an L-Step. C-Step, and D-Step design, along with an uncertainty masking mechanism, on top of box embedding.
CDM [8] A recommendation framework leveraging contrastive context encoder with attention mechanisms, distilling knowledge from MMR-based teacher output, and combining scores for diverse results.
LLM-based diversified recommenders:
LLM4Rerank-A [00 A reranking framework addressing accuracy, diversity, and fairness by abstracting requirements into interconnected nodes and dynamically adjusting aspect priorities through a Shain-of-Thoughe process. For I.LM4Rerank-A, we set the L.LM's focus solely on improving accuracy during reranking.
LLM44erank-AD [20]: Similar to LLM4Rerank-A, the difference lies in setting the LLM's focus improving both accuracy and diversity during reranking.
LLMRec-MMR [24]: LLMRec uses LLMs to enhance recommendation relevance through three graph augmentation strategies; reinforeing user-item interactions, improving item attributes, and refining user profiling. MMR is then applied to balance diversity by reranking the list.
A.4 More experiment results
Empirical details of ToP. During the construction of ToP. prompt the LLM generate multi-level user preference tree (see 3.22 and Appendix A.2p. By instructing the LLM that each node represents a preference type and each edge denotes finer- grained division, hierarchical nodes are ultimately generated. We provide additional empirical details of the tree of preferences below. As an illustration, key architectural specifications of ToP constructed on Twitter across runs include 133.6 leaf nodes, 248.4 4 total nodes, an average depth of 6.2, 53.3 average items per leaf, and an average branching factor of 2.16; to better showcase the structure of ToP, we then present partial visualization of an actual ToP instance employed in the Twitter dataset, as shown in Figure
finer-rriied
Empirical analysis of user selection. The effectiveness of influential user selection in ToP-Rec mainly influenced by three factors. First, the number of user groups determines group size, affecting user selection granularity. Sccond, the dimension of gradient reduction impacts the amount of retained information, and the step length & affects the influence evaluation period. We analyze their impact on overall performance.
Figure 6 a) and (b) illustrate the changes in diversity and relevance on Twitter with different group numbers (maintaining similar total number of augmented users) and different reduced dimensions. Increasing group size or dimensionality improves recall and category entropy, but with smaller marginal returns. This suggests that a small number groups or dimensionality provides sufficient gains while keeping computational costs lower. Figure visualizes the impact of step k showing that smaller values (within 5) maintain low costs while yielding better performance. Larger step sizes degrade performance, likely due to the extended training history losing dynamic user influence.
25

---

Root
Political Systems & Governance
Cultural Arts & Entertainmeat
Governmental Actions Decisions
Movies Television
Streaming Series
Policy Decisions
Healthcare Policy
Theatrical Releases
Tax & Environment
Music
Pubile Statements
Political Figures
Prime Ministers and Presidents
Cabinet
Members
Political
Gatherings
Figure 5: A partial visualization of the constructed ToP.
1.41
0.087
1.40
0.090
1.38
-0.086
0.086
00085
Cogtton
1.38
+0.085
R500001
CEA0 00
1.38
0.087
R:8000
CERR100
1.36
0.084
Re010n
1.36
1.36
0.085
0.084
0.083
0.083
0.083
1.34
1.33
0.082
1.33
10
20
30
40
256
512
1034
2043
0.080
(a) user groups
(b) Reduced dimension.
(c) Step k.
Figure Impact of (a) number of user groups, (b) reduced dimension, and (c) step k the performance of ToP-Rec. Dashed lines represent the performance of the backbone recommender.
Performance evaluation on more recommender backbones. We further evaluate the performance of ToP-Rec on two widely used recommenders, namely MF [32] and NGCF [78). We the hidden size to 32 and utilize the Adam optimizer with a learning rate of 5e-3, keeping other settings the same as LightGCN. We compare ToP-Ree with representative baselines, all reporting relatively balanced results for diversity and relevance. Tables 4 and 5 show the performance comparison on Twitter, using MF and NGCF as backbones, respectively. denotes the backbone model, and +/- indicates performance changes. The optimal performance is in bold, and the suboptimal is underlined. The
26

---

Table 4: Performance comparison using MF as recommender backbone.
R@50
R@ 100
CE@50
CE@ 100
MF"
0.0329
0.0501
1.2249
1.3189
MMR
0,0319
0.0485
1.2596*
1.3533
LCD-UC
0.0327
0.0503*
1.2558*
1,3504
LLM4Re-AD
0,0323
0.0498
1.2620
1,3444
ToP-Rec
0.0345*
0.0510
1.2980
1.3813
Table 5: Performance comparison using NGCF as recommender backbone.
R@50
R@ 100
CE@50
CE@ 100
NGCF
0.0468
0.0748
1.2478
1.2977
MMR
0.0435
0.0708
1.2800
1.326I
LCD-UC
0.0466
0.0733
1.2897*
1.3365*
LLM4Re-AD
0.0467
0.0752
1,2722+
1.3189*
ToP-Rec
0.0473
0.0763
1.3630
1.4028
results show that ToP-Rec achieves optimal performance in both cases, further demonstrating its generalizability.
Standard deviations. shown in Table 1.
Table 6 presents the standard deviations for the recall and category entropy
Relvestce trade-off. We provide trade-off analysis on the Weibo and Amazon datasets, as shown in Figure 7 Our approach consistently achieves the best trade-off, demonstrating its outstanding performance.
Hyperparameter analysis. We first examine the impact of the number of selected leaf nodes per user ToP-Rec's performance. Varying the number of selected leaves among {4, 5, 6, 7), the results on the Twitter dataset in Figure (j)) show that as leaf nodes increase, diversity improves, while relevance rises initially and then slightly declines. This is likely due more leaf nodes providing ToP-Rec with greater capacity to explore unobserved preferences. However, as the number of leaf nodes increases further, the risk of introducing noise also rises. Nevertheless, due our systematic preference reasoning design, ToP-Rec performance remains robust, showing substantial improvement.
Then, we evaluate the impact of the structure of ToP by adjusting prompts to modify requirements for tree depth and branching number. Figure b) and (c) report ToP-Rec's performance across different runs, with variations in tree depth and average branching factor. We find that shallow structures constrain preference resolution, while excessive depth also degrades performance, potentially at- tributed to semantic fragmentation. Following similar logic, the average branching factor exhibits a comparable trend: relatively fewer or more branches both lead suboptimal performance.
Stability analysis of ToP-Rec under key variations. We further investigate the stability of ToP- Rec across variations the applied LLMs, prompt strategies, and initial sampled item counts. Specifically, integrate ToP-Rec with two more LLMs of distinct parameter scales and architectural designs, including Qwen2.5-7BB - (37) and Llama3-78 7OB [79]. Stability evaluated along three key dimensions: the number of leaf nodes in the constructed ToP, the depth of ToP, and the final recommendation performance. Table presents the results on the Twitter dataset. Our findings demonstrate that ToP-Rec maintains robust stability across different LLMs.
Next, we evaluate the impact of distinct prompt strategies on ToP's structural stability and the final recommendation performance. We design two variants: a simplified prompt (with specific task- related instructions removed) and an enhanced prompt (incorporating more detailed task guidance). Corresponding results presented in Table 8 reveal that the enhanced prompt leads to modest increase
27

---

Table Standard deviation.
Twitter
Weibo
Ammzon
R@50
10100
CE@50
CE@ 100
R@50
R 100
CE#50
CE@100
R@50
R@100
CE@50
CE@ 100
LightGCN"
0.0088
0,0007
0.0030
0.0038
0.0002
0.0006
0,0036
0.00266
0.0007
0.0006
0.0071
0.0057
Random
0.0044
0.0024
0.0.11
0.145
0.00003
0,0008
0.0039
0.0065
0.0015
0.0024
000000
60f2r
MMR
0.0004
0.0004
0.0027
0.0021
0.0002
0,0002
0,0084
0.0076
0.0014
0.0014
0.0030
0.0059
0.0017
0.0073
0.0007
CDM
0.0012 0.0008
0.0046
0.0051
0,0002
0.0003
0.0057
0,0055
0.0021
0.0019 0.0161
0.0162
Box
0.0018 0,0028
0.0058
0.0044
0.0028
0.0016
0,0078
0.0121
0.0023
0.0020 0.0072
0.0089
LCD.UC
0,0016 0.0032
0.0093
0.0085
0.0016
0.0024
0,0081
0.0039
0.0021
0.0018
0.0082
0.0078
LIMRec-MMR
0.0011
0.0012
0.0044
0.0057
0.0006
0.0008
0.0062
0.0050
0.0017
0.0019
0.0101
0.0092
LLM4Ro-A
0.0008
0.0010
0.0098
0.0083
0.0003
0.0008
0.0079
0.0013
0.0015
0.0089
0.0091
LLMARe-AD
0.0010
0.0012
0.0161
0.0147
0.0004
0.0005
0.0096
0.0065
0.0025
0.0028
0.0156
0.0114
ToP Rec
0.0012
0.0013
0.0127
0.0141
0,0006
0.0008
0.0066
0.0059
0.0010
0.0006
0.0142
0.0136
LightOEN MMR
Divasity (CE:8100)
1.13
L.12
Rendvnn
XD4R
CDM
Diroriity (CE-@100)) 100)
0.594
DP7 CDM
TEID-1X:
LLMKee-MMM Tet-Rec
1.11
-
1.10
0.576
1.09
1.08
1.07
01550
01175
(.1525
0.667
0,2010 0,3025 0:2040 2.55 2,2000 0.2100 0.2115
Relevance (R@1100)
Relevance (R@ (00)
(a) Weibo.
(b) Amazon.
Figure 7: Diversity- trade-off comparison. The upper-right represents the ideal.
in the number of leaf nodes within the ToP structure. Notably, regardless of the prompt variant employed, ToP-Rec consistently outperforms all baselines, demonstrating the framework's robustness to variations in prompt design.
Finally, assess the impact of varying initial sampled item counts the stability of ToP-Rec. Consistent with prior analyses, results are presented in Table ToP-Rec maintains robust stability even with reduced initial sampling ratio, exhibiting significant performance degradation and continuing to demonstrate superiority over baselines.
Comparison with generative LLM-based recommendation. We also provide a comparison with the state-of-the-art LLM-based generative diversified recommender, DLCRec (21]. Since DLCRec generates recommendations by predicting item names rather than IDs, it struggles with scenarios like Twitter or Weibo, where items are posts with long text and lack explicit names. Therefore, we conducted evaluations on Amazon, where items are products with brief names. We followed DLCRec's paradigm for LLM fine-tuning and generated 10 items per user during inference, calculating recall 10 and entegory-entropy @ The results are shown in Table IO indicating a significant improvement for approach. We empirically found that DLCRec carries high risk of generating homogeneous items, which can be attributed to its limited ability to fully capture users' latent preferences, leading poorer performance. Furthermore, due to the instability of LLM generation, DLCRec performs poorly under our original settings, where metrics are evaluated at Top 50 and 100.
A.5 Limitation and future work
While ToP-Rec shows promise in diversifying recommendations, two limitations need attention. First, the quality of user and item textual attributes affects its effectiveness. cases of minimal or low-quality text, such platforms with primarily video image content, the current design may perform less effectively. Second, ToP-Rec does not focus on enhancing recommendation novelty. This limitation arises from prioritizing diversity in preference-aligned item matching without
28

---

1.40
1.38
C1/0)100
CE 100
-0.084
Rg8100
C32R.00
0.084
1.36
1.36-
0.083
1.35
1.34
1.34
0.083
0.082
0.083
2,01
selected leaves.
(b) Depth of ToP.
(c) Branching factor of ToP.
Figure 8: Hyperparameter analysis of (a) number of selected leaves, (b) depth of ToP, and (c) branch-
ing factor of ToP. We use dashed lines represent the performance of the backbone recommender.
Table 7: Stability analysis across different LLMs.
leaf nodes
Depth
R@50
100
CE@50
CE@ 100
Llama3-70B
137.4
6.8
0.0584
0.0840
1.3367
1.3940
Qwen2.5-7B 2enn2.5-7B-nsstruct
138.2
7.0
0.0579
0.0836
1.3291
1.3857
Qwen2.5-322 3-BBBBBB3BBB.. Instruct (Ours)
133.6
6.2
0.0586
0.0841
1.3275
1.3852
explicitly quantifying each item's potential surprise users. Future work will focus integrating novelty-aware objectives, further enhancing the user experience.
29

---

Table Stability analysis across different prompt strategies.
# leaf nodes
Depth
R@50
R@ 100
CE@50
CE@ 100
Simple prompt Complex prompt
129.4
6.6
0.0583
0.0853
1.3329
1.3821
147.8
6.0
0.0573
0.0846
1,3208
1.3853
Ours
133.6
6.2
0.0586
0.0841
1.3275
1.3852
Table 9: Stability analysis across different initial sampling ratios.
nodes
Depth
R@50
R@ 100
CE@50
CF@
sampling
137.4
0.0575
0.0840
1.3284
1.3845
6% sampling
126.8
0.0583
0.0840
1.3273
1.3830
9% sampling (Ours)
133.6
6.2
0.0586
0.0841
1.3275
1,3852
Table 10: Performance comparison with DLCRec.
R 10
CE@ 10
DLCRec
0.0021
0.4714
ToP-Rec
0.0119
0.9639
30


Reviews:
[{"review_id": "TjH3XKjYFB", "reviewer": "Reviewer_K1gm", "summary": "This paper introduces ToP-Rec, a novel LLM-based framework designed to enhance recommendation diversity while preserving relevance. The authors designed a Tree of Preferences, a hierarchical preference structure constructed by prompting LLMs to reason about user interactions and uncover underexplored preferences from observed behaviors. By leveraging ToP, the method identifies latent user preferences and generates synthetic interactions that augment existing data, modeling user preferences from coarse to fine. Extensive evaluations of both diversity and relevance demonstrated that the proposed approach outperforms existing methods in most cases with efficient inference latency.", "questions": "Aside from the questions in Weaknesses, I have the following questions: \n- How does ToP-Rec perform with smaller or less capable LLMs? Qwen2.5-32B is powerful but costly. Can your method maintain reasonable performance with smaller models? How robust is ToP-Rec to LLM inaccuracies?\n- Minor question out of curiosity: If a user shows interest in multiple, contradictory topics (e.g., budget travel and luxury goods), how does ToP-Rec resolve this during item selection?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The problem of balancing diversity and relevance in recommendations is clearly stated and well-motivated. The challenges and the literature are fully discussed.\n- The design of the Tree of Preferences is novel and effectively combines LLM reasoning with preference modeling.\n- The methodology section is well-explained and solidly grounded. \n- The strong performance on the three datasets in comparison with both heuristic methods and LLM-based diversified recommenders validates the effectiveness of the proposed method.\n- The paper is well-written. It is a pleasure to read the paper.\n\n\nWeaknesses:\n- For the design of the relevance score, this work uses the semantic similarity of the user and item. How to ensure the quality of user and item textual attributes is a concern.\n- The paper does not analyze failure cases, e.g., when diversity gains hurt relevance. Examples of incorrect LLM inferences or misaligned recommendations, if they exist, would help calibrate expectations. It would also be beneficial to show actual ToP trees and user-item matches to improve interpretability.\n- The choice of the evaluation metrics could be further justified. Is category-entropy alone expressive enough to showcase the diversity of the recommendation results?", "was_revised": true, "final_justification": "Given the strengths and the addressed concerns, I will keep my score to indicate my recommendation of the paper.", "rebuttal": ">W1: Quality of user and item textual attributes.\n\nThank you for your feedback. Following prior work [1], we enhance the quality of user and item textual data with the aid of LLMs. Specifically, we leverage an LLM to generate user summaries based on their profile and the textual attributes of previously interacted items, as well as to generate item summaries from their textual attributes. By utilizing the LLM, we aim to reduce noise, redundancy, and inconsistencies in the original text, leading to improved summaries that retain key semantic information. We will update the paper to include these refinement details.\n\n\n>W2: Analysis of failure cases and demonstration of ToP and user-item matches.\n\nThank you for this insightful suggestion. To address cases where diversity gains negatively impact relevance, we conducted a post-hoc failure analysis on Twitter. Our investigation reveals that in 2.1% of cases (45/2,118 users), diversity improvements incurred relevance costs, quantified as an average recall decrease of 17.2%, accompanied by entropy increases of 14.2%. This trade-off is primarily driven by two factors: First, we find that more than half of these cases arise from the user's sparse interaction histories or the presence of uninformative interacted items, which leads to preference misinference. Second, for users with broad but shallow interaction histories, the risks of exploration boundaries may cause diversification to outweigh relevance, leading to a mismatch with their actual preferences.\n\nDue to text-only constraints, we provide a partial illustration of the actual ToP used in the Twitter dataset. We will include visualizations of the ToP structures and visual user-item matches in the manuscript to improve interpretability.\n\n```\nROOT\n├── Political Systems & Governance\n│   ├── Governmental Actions and Decisions \n│   │   ├── Policy Decisions\n│   │   │   ├── Healthcare Policy\n│   │   │   │   ├── ......\n│   │   │   ├── Tax & Environment\n│   │   │    ......\n│   │   ├── Public Statements\n│   │       ├── ......\n│   ├── Political Figures\n│   │   ├── Prime Ministers and Presidents\n│   │   │   ├── .....\n│   │   ├── Cabinet Members\n│   │   ......\n│   ├── Political Events\n│   │   ├── Parties and Gatherings\n......\n├── Cultural Arts & Entertainment\n    ├── Movies & Television\n    │   ├── Streaming Series\n    │   ├── Theatrical Releases\n    ...\n    ├── Music \n    ....\n```\n\n>W3: Evaluation metrics of diversity.\n\nThank you for raising this insightful question. As a widely used metric [2,3,4], category entropy captures the distributional spread of item categories within a recommendation list, which aligns directly with the goal of diversity in our work. \n\nTo provide a more comprehensive evaluation, we also implemented two additional diversity metrics: the Gini Index (GI) [5] and Intra-List Distance (ILD) [6]. The following table presents the corresponding results on Twitter, comparing Top-Rec with comparable baselines and the backbone recommender. The results demonstrate that our method consistently outperforms others, highlighting its effectiveness.\n|   | Gini@50 $\\downarrow$ | Gini@100 $\\downarrow$ | ILD@50 | ILD@100 |\n|----|--------|-----------|-------|-------|\n|LightGCN$^*$  |0.6763|0.6580|1.1756|1.1959|\n|LCD-UC|0.6723|0.6517|1.1849|1.2081|\n|LLM4Re-A|0.6733|0.6521|1.1775|1.1967|\n|Ours|**0.6678**|**0.6450**|**1.1954**|**1.2112**|\n\n>Q1: Performance with smaller LLMs and robustness with LLM inaccuracies.\n\nWe evaluate our method with three smaller LLMs, including Qwen-2.5-7B-Intstruct, Qwen-3-8B, and Llama-3-8B-Instruct. The following table reports their performance on Twitter, compared with two comparative baselines and the backbone recommender. The results show that even with small LLMs, ToP-Rec can still outperform baselines in most cases, further demonstrating its robustness.\n\n|   | R@50  | R@100 | CE@50 | CE@100 |\n|----|--------|-----------|-------|-------|\n|LightGCN$^*$  |0.0567|0.0830|1.2841|1.3413|\n|LCD-UC|0.0517|0.0768|1.3154|1.3784|\n|LLM4Re-A|0.0562|0.0827|1.2855|1.3424|\n|Qwen3-8B|0.0578|0.0837|1.3227|1.3792|\n|Llama3-8B-Instruct|0.0573|**0.0842**|1.3089|1.3696|\n|Qwen2.5-7B-Intstruct|$\\underline{0.0579}$|0.0836|**1.3291**|**1.3857**|\n|Qwen2.5-32B-Instruct (Ours)|**0.0586**|$\\underline{0.0841}$|$\\underline{1.3275}$|$\\underline{1.3852}$|\n\nThe robustness of ToP-Rec can be attributed to the design of the tree of preference, which acts as a vehicle to help the LLM systematically reason through the user's rationale and uncover their preferences, thereby mitigating the potential impact of hallucinations or inaccuracies.\n\n>Q2: Resolution for contradictory preferences.\n\nThank you for this enlightening question. While a user's preferences may seem contradictory at first, we believe these apparent contradictions can often be rationally explained, provided the user’s choices are driven by reason rather than randomness. For example, a user interested in both budget travel and luxury goods might value practicality and exclusivity in different contexts.\n\nTo explore this further, we analyzed how ToP-Rec handles such contradictions in practice. Our findings show that ToP-Rec is generally capable of rationalizing these preferences. For instance, we observed that some users may purchase budget-friendly daily items while also buying higher-end goods. ToP-Rec interprets this as a preference for affordable consumables and premium, long-term items. During item selection, ToP-Rec considers the context of different items,  seeks to provide recommendations that align with their preferences.\n\n\n[1] LLMRec: Large Language Models with Graph Augmentation for Recommendation\n\n[2] Dual-Process Graph Neural Network for Diversified Recommendation\n\n[3] Controlling Diversity at Inference: Guiding Diffusion Recommender Models with Targeted Category Preferences\n\n[4] DGCN: Diversified Recommendation with Graph Convolutional Networks\n\n[5] Post Processing Recommender Systems for Diversity\n\n[6] Avoiding monotony: improving the diversity of recommendation lists"}, {"review_id": "bZUAsiyeXn", "reviewer": "Reviewer_F6jv", "summary": "This paper introduces ToP-Rec, a novel framework designed to improve diversity in recommender systems by addressing inherent data biases. The core contribution is the \"Tree of Preferences\" (ToP), a hierarchical item structure generated by a Large Language Model (LLM) to represent its world knowledge. The LLM uses this tree to reason about a user's latent interests and generate synthetic interaction data, specifically targeting underexplored preference areas. This augmented data is then used to train a conventional, lightweight recommender model. This data-centric approach separates the expensive LLM reasoning from the efficient online serving phase, with an additional strategy to select influential users for augmentation to manage computational costs.", "questions": "1. The method's validity depends on the stability of the generated Tree of Preferences. How sensitive is the ToP structure to variations in the prompt, the choice of LLM, or the initial item sampling? A quantitative analysis of the tree's structural variance across different runs is needed to establish the method's reliability.\n2. Could you provide a full breakdown of the offline computational costs (e.g., in GPU-hours) and compare the proposed influential user selection method against simpler, standard active learning heuristics (e.g., uncertainty sampling)? This is needed to justify the method's complexity and assess its true cost-efficiency.", "limitations": "yes in appendix.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\nThis paper presents a highly original and significant approach to diversified recommendation by reframing it as a data bias problem solvable with LLM-generated data. The primary strength is the \"Tree of Preferences\" (ToP) methodology, an innovative application of tree-based reasoning that allows an LLM to infer a user's unobserved interests in a structured, coarse-to-fine manner. This moves beyond simply augmenting existing data and into the more ambitious realm of correcting systemic bias by filling knowledge gaps. The system design is also elegant and practical, using the powerful but costly LLM offline to generate debiased training data, which then informs a fast, scalable conventional model for real-time inference.  \n\nWeaknesses:\nThe work exhibits some areas for potential improvement, including the current lack of a comprehensive robustness analysis for the ToP generation across different LLM models or prompt variations, which could further solidify the reliability of its foundational artifact. Additionally, while inference latency is well-addressed, the paper could provide more detailed quantification of the substantial offline computational costs, and further elaborate on the justification for its complex gradient-based influential user selection method, perhaps with more direct comparisons to simpler active learning heuristics. Lastly, the choice of a simple linear combination objective function for synthetic item selection, analogous to MMR, might benefit from additional exploration or justification regarding its optimality in this crucial data generation step.", "was_revised": false, "final_justification": "", "rebuttal": ">  W1 & Q1: Analysis for the ToP generation across different LLM models, prompt variations and initial item sampling.\n\nWe appreciate your insightful comment. To improve the stability of the ToP, we employ several designs, including using a k-means sampling strategy during the ToP construction to select a subset of items that is sufficiently representative of user preferences, thereby reducing the impact of randomness. Additionally, we incorporate a refinement mechanism that enhances the balance of the ToP through merge and split operations (detailed in Appendix A.2).\n\nFollowing your suggestion, we further evaluate the stability of the ToP with different LLMs, different prompt strategies, and varying numbers of initial sampled items. First, we integrate ToP-Rec with LLMs with different parameter sizes and architectures, including Qwen2.5-7B-Instruct and Llama3-70B. We report the stability of the ToP in terms of the number of leaf nodes, depth, and final performance. The following table presents the evaluation on Twitter, in comparison with the backbone recommender and other comparative baselines. The results show that ToP is relatively stable across different LLMs, with the final performance consistently outperforming the baselines.\n\n|   | # leaf nodes| Depth  | R@50  | R@100 | CE@50 | CE@100 |\n|----|--------|--------|-----------|-------|-------|-------|\n|LightGCN$^*$  |-|-|0.0567|0.0830|1.2841|1.3413|\n|LCD-UC|-|-|0.0517|0.0768|1.3154|1.3784|\n|LLM4Re-A|-|-|0.0562|0.0827|1.2855|1.3424|\n|Llama3-70B|137.4|6.8|0.0584|0.0840|**1.3367**|**1.3940**|\n|Qwen2.5-7B-Instruct |138.2|7.0|0.0579|0.0836|1.3291|1.3857|\n|Qwen2.5-32B-Instruct(Ours)|133.6|6.2|**0.0586**|**0.0841**|1.3275|1.3852|\n\nNext, we apply different prompt strategies, including a simpler version where we remove some specific task instructions, and a more complex version where we provide more detailed task instructions. Similarly, we report the evaluation in the following table. The results show that the more complex prompt leads to a slight increase in the number of leaf nodes in the ToP, while all versions of ToP-Rec outperform the baselines.\n\n|   | # leaf nodes| Depth  | R@50  | R@100 | CE@50 | CE@100 |\n|----|--------|--------|-----------|-------|-------|-------|\n|LightGCN$^*$  |-|-|0.0567|0.0830|1.2841|1.3413|\n|LCD-UC|-|-|0.0517|0.0768|1.3154|1.3784|\n|LLM4Re-A|-|-|0.0562|0.0827|1.2855|1.3424|\n|Simple prompt|129.4|6.6|0.0583|**0.0853**|**1.3329**|1.3821|\n|Complex prompt|147.8|6.0|0.0573|0.0846|1.3208|**1.3853**|\n|Ours|133.6|6.2|**0.0586**|0.0841|1.3275|1.3852|\n\nFinally, we examine the impact of varying numbers of initial sampled items. Similar to the previous analysis, we report the evaluation in the table below. We observe that ToP remains relatively stable, even with a lower sampling ratio.\n\n|    |# leaf nodes| Depth  | R@50  | R@100 | CE@50 | CE@100 |\n|----|--------|-----------|-------|-------|-------|-------|\n|LightGCN$^*$  |-|-|0.0567|0.0830|1.2841|1.3413|\n|LCD-UC|-|-|0.0517|0.0768|1.3154|1.3784|\n|LLM4Re-A|-|-|0.0562|0.0827|1.2855|1.3424|\n|3% sampling|137.4|6.6|0.0575|0.0840|**1.3284**|1.3845|\n|6% sampling|126.8|6.8|0.0583|0.0840|1.3273|1.3830|\n|9% sampling(Ours)|133.6|6.2|**0.0586**|**0.0841**|1.3275|**1.3852**|\n\n>  W2 & Q2: Detailed quantification of offline computational costs and comparison to simpler active learning.\n\nThank you for your valuable comment. We’ve provided a detailed breakdown of the main steps in training Top-Rec that contribute to the overall computational costs. For tasks performed on our local device, we provide the computational cost in GPU hours based on a single NVIDIA RTX 3090. For the steps executed via the LLM API, we compute the equivalent computational cost expressed in GPU hours on a single NVIDIA A100. The computational cost analysis for Top-Rec on the Twitter dataset is summarized in the table below.\n\n|  Task | GPU hour  | GPU type |\n|----|--------|-----------|\n|Constructing ToP  | 1.14 | A100 |\n|Matching items with ToP| 0.67 | A100 |\n|Training recommender backbone| 0.16| RTX3090 |\n|Selecting influential users| 0.01| RTX3090 |\n|Capturing user preferences| 1.64| A100 |\n\nAs suggested, we also compare our selection of influential users with standard active learning by adopting the widely used uncertainty sampling [1]. Specifically, user uncertainty is calculated based on their average item scores, and users with the highest uncertainty are selected, with all other procedures remaining unchanged. The results presented below demonstrate that our method of selecting influential users leads to better overall performance, while maintaining comparable time costs, facilitated by PyTorch’s automatic vectorized computation.\n\n|   | R@50  | R@100 | CE@50 | CE@100 | User selection time (seconds)|\n|----|--------|-----------|-------|-------|-------|\n|Uncertainty|0.0569|0.0832|1.3219|1.3779|6.7|\n|Ours|**0.0586**|**0.0841**|**1.3275**|**1.3852**|37.6|\n\n> W3: Additional exploration of synthetic item selection.\n\nThank you for this insightful comment. To further evaluate the effectiveness of our design in synthetic data generation, we conduct a comparison with two more complex mechanisms. First, we adopt the item scoring strategy from [2], which utilizes the Clayton copula function to account for both diversity and relevance. Second, following [3], we utilize an LLM to select items that a user might be interested in from a further filtered item candidate pool. We keep all other settings consistent with the original ToP-Rec and report the performance comparison on Twitter below. We observe that our original design maintains strong performance even when compared to these more complex methods. The main reason is that user preferences can be effectively inferred through ToP, which enhances the quality of the candidate item set and ensures reliable performance in our synthetic data generation.\n\n|   | R@50  | R@100 | CE@50 | CE@100 |\n|----|--------|-----------|-------|-------|\n|Clayton copula|0.0582|0.0838|1.2996|1.3685|\n|LLM selection |0.0576|0.0833|1.3237|1.3802|\n|Ours|**0.0586**|**0.0841**|**1.3275**|**1.3852**|\n\n\n[1] A Survey on Active Learning: State-of-the-Art, Practical Challenges and Research Directions\n\n[2] Relevance meets Diversity: A User-Centric Framework for Knowledge Exploration through Recommendations\n\n[3] LLMRec: Large Language Models with Graph Augmentation for Recommendation"}, {"review_id": "nBVufRDy6F", "reviewer": "Reviewer_5Xsf", "summary": "The authors present a method (Top-Rec) for using LLMs to promote diversity while maintaining relevancy in recommendations. Users and items are assumed to have textual descriptions available. An LLM is asked to constructed a detailed, multi-layer tree of preferences (ToP) organizing a representative sampled set of items. Given the ToP and a list of items a user likes, the LLM is then asked to generate a list of leaf nodes from the ToP representing the user interests, where implicit, latent preference leaf nodes are extracted as well as the more obvious ones. Synthetic data is then generated from items which fall under the implicit leaf nodes and that synthetic data is added to the real data for the sake of training a recommendation system. For the sake of of efficiency, particularly influential users are identified using gradient alignment methods and the generation of synthetic data focuses on those influential users. Highly positive results are obtained on Twitter, Weibo and Amazon datasets in terms of both relevance and diversity of recommendations, with Top-Rec beating all competitors in 11 out of 12 comparisons. Ablation studies are presented to confirm the significance of both diversity and relevance criteria as well as the utility of user influence measures.", "questions": "My questions have mostly already been outlined in the strengths weaknesses section, but to reiterate:\n\n1) What can you tell me to assure me that there is no test set contamination which would help Qwen \"cheat\" in subtle statistical ways when helping to generate synthetic data for Amazon, Weibo, etc?\n\n2) Is the tree of preferences a single, global tree or a per-user tree? Can you edit the paper to make this clearer?\n\n3) Does the method require significant user-profile text or can you substitute text from items the user likes?\n\n4) Can you tell me more about how prompt-length limits interact with constraints on how many sample items can be supplied to the LLM and how long the item descriptions can be? Can you assure me that this won't limit the utility of the method in other contexts?", "limitations": "Adequately addressed.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "The work is original to the best of my knowledge.\n\nThe experimental results are quite impressive. If indeed the experimental results are indicative of the improvements TopRec could generate in real-life settings, the work should have substantial significance. \n\nOne concern I have is the possibility of some sort of test set contamination. Qwen2.5-32B-Instruct was created after all 3 datasets were created. Is it possible that some of the crawled text that Qwen absorbed somehow lets it cheat in creating the tree of preferences, assigning the items to leaves, etc? My understanding is that test set leakage can arise in subtle and surprising ways with LLMs.\n\nI also have a clarity concern which may be my fault, but maybe not. I'm slightly embarrassed to ask this, but if I have to ask this, I think it's fair to wonder whether the paper could be clearer on this point,  Is the tree of preferences created one-time, globally, across all users, or is there a per-user tree of preference? Maybe this should be clear to me, but I went over the paper several times and still wasn't totally sure what the answer is here. Again, maybe the fault is mine for not reading carefully enough, but I've won top-reviewer NeurIPs awards multiple times, so if it wasn't clear to me, then I think the paper could probably be clearer on this point.\n\nI am curious about the typical limits of prompt length for LLMs and whether that does or could pose constraints or difficulties for this approach when supplying the LLM with a list of items. Clearly, the results are positive, so this constraint wasn't a big deal here, but are we sure that the sampling will be able to overcome the prompt length constraints in the future? I'd be interested in some simple stats like e.g. Qwen can take prompts of max length X characters and if an item description has an average length of Y characters then that means we can put Z items in a prompt and this should usually be enough because <insert reasons here>. \n\nI am also curious about the cosine similarity method on line 210 and its reliance on A_u. More generally, I am curious about the importance of the assumption of the existence of user profile text A_u. Might it not be better (and more widely applicable) instead of using text from A_u to use a summary of text from items the user likes, i.e. from their real interaction history? I want to make sure I understand A_u.  A_u is (for instance) the profile text from Figure 1 (Bio: peaks unclimbed, etc) , correct? Does your method need such a profile? I could imagine many applications where no such profile (or a profile with minimal, useless text) exists, but we still have the reposts and likes (Summer escape), etc? Is there any reason you can't use a sample of the reporsts and likes if the user profile is not available? \n\nSome suggested rewrites and typos;\n\nLine 32 refers to the interaction data on line 30 as 'implicit feedback', but ratings are explicit, not implicit\n\nLine 50 a diversified recommendation -> a diversified set of recommendations.\n\nLine 8 appendix Frist -> First\n\nAppendix, page 2, figure 4: Notation, not Notaion\n\n*** Update after rebuttal ****\n\nIn light of the clarifications supplied by the authors as well as the uniformly positive ratings from other reviewers (which is rare to see), I have chosen to increase my score to a '5'.", "was_revised": true, "final_justification": "The authors resolved my confusion regarding (1) whether or not the tree is a single tree, (2) prompt-length constraints and (3) whether the method relies on substantial user profile text.  \n\nI have been reviewing for NeurIPs for many years, and in my experience, it is rare to see all reviewers rate a paper as 4 or above. In light of this uniformity of agreement, I decided to raise my score to a 5.", "rebuttal": ">Q1: Test set contamination which would help Qwen \"cheat\".\n\nThank you for raising this valuable concern. Research on test set contamination in LLM-based recommender systems is indeed challenging. We address this by systematically analyzing risks at each stage of Top-Rec.\n\nFirst, during the construction of the tree of preference and assignment of items to leaves, Top-Rec exclusively utilizes item-side textual information. This process is orthogonal to user-item interaction data in the test set. We believe test set contamination is highly unlikely at this stage.\n\nSecond, in the synthetic data generation phase, the LLM indirectly participates in user preference inference. While LLMs possess the capability to memorize the pretraining texts, this memorization ability is unlikely to extend to user-item relations, as such interactions are typically not stored as plain text. Consequently, the feasibility of the LLM cheating in preference inference by memorizing specific interactions remains low.\n\nTo quantitatively evaluate this risk, we adopted the LLM pretraining data detection method [1], which identifies whether arbitrary data points belong to the LLM’s training data. Specifically, to investigate potential leakage of test set user-item interaction, we constructed two text corpora (10,000 samples each) describing user-item relationships (e.g., \"User A with profile B liked Tweet C with content D\"). One corpus contained ground-truth relationships from the test set, while the other contained randomly paired false relationships serving as a control group. We applied a state-of-the-art detector MIN-K% [1] to detect the likelihood of these relationships belonging to Qwen2.5-32b-Instruct’s training data. Results from MIN-K% showed that the proportion of test set relationships identified as **non-training data (99.7%)**  was nearly identical to that of randomly generated false relationships (99.5%), indicating minimal test set leakage risk. We further conducted significance testing on the perplexity distributions of both text groups, which reflect the model’s predictive confidence.  We applied two statistical tests: the t-test and the Kolmogorov–Smirnov  test. Both yielded **p-values above 0.9**, showing no significant statistical difference between the distributions of the perplexity scores. This confirms that Qwen’s perplexity for true and false interactions is statistically indistinguishable, providing additional evidence against test set contamination.\n\n>Q2: Is the tree of preferences a single, global tree or a per-user tree?\n\nWe sincerely apologize for any confusion caused. We would like to clarify that the tree of preference is constructed as a **global, one-time structure shared across all users**. This design is based on our understanding that, while individual preferences may vary, all users ultimately navigate the same underlying preference space defined by item characteristics. We have revised the paper to clarify this point.\n\n\n>Q3: Does the method require significant user-profile text or can you substitute text from items the user likes?\n\nThank you for raising this important point. \n\nFirstly, the user profile refers to basic registration information, as shown in Figure 1 (e.g., age, bio). Similar to prior work [2,3], our method incorporates such a profile, but does not rely on a large amount of user-profile text. The profile is not the only source of user information. As you rightly pointed out, we also utilize the text attributes from items the user has interacted with, which provide valuable insights into their preferences. These interacted items serve as another crucial source of user information. Ultimately, we follow [2] to provide both the text attributes from user profile and the items they interacted with to the LLM. By focusing on the user's interests and filtering out noise, we obtain an enhanced user summary from the LLM as the final textual feature.\n\nWhen the user profile is unavailable, Top-Rec can rely on the text attributes from the interacted items as a substitute to generate the user's textual feature via LLM. To evaluate performance in such scenarios, we masked the original user profile and used only summaries from the interacted items as users' text features, then evaluated Top-Rec on this dataset. The table below presents the results on Twitter, which show that our method still outperforms the comparative baselines.\n|   | R@50  | R@100 | CE@50 | CE@100 |\n|----|--------|-----------|-------|-------|\n|LightGCN$^*$  |0.0567|0.0830|1.2841|1.3413|\n|LCD-UC|0.0517|0.0768|1.3154|1.3784|\n|LLM4Re-A|0.0562|0.0827|1.2855|1.3424|\n|w/o profile|$\\underline{0.0584}$|$\\underline{0.0839}$|$\\underline{1.3258}$|**1.3895**|\n|Ours|**0.0586**|**0.0841**|**1.3275**|$\\underline{1.3852}$|\n\n>Q4: How prompt-length limits interact with constraints on how many sample items can be supplied to the LLM and how long the item descriptions can be? And the utility of the method in other contexts.\n\nThank you for raising this important question. We acknowledge that such constraints warrant consideration, but they do not significantly impact the construction of the tree of preference. This is because ToP relies not on the absolute size of the entire item set, but on the semantic breadth of a sampled subset. Crucially, our approach focuses on identifying a representative subset that captures the underlying preference space of the full item collection.\n\nTo achieve this, we designed a k-means clustering-based sampling method (details in Appendix A.2). By clustering items and sampling proportionally from each cluster, we obtain a compact yet representative subset for LLM processing. As an empirical example, Qwen2.5 supports up to 131,072 tokens of context, and in the Twitter dataset, each item averages 27.6 tokens. This results in a maximum theoretical capacity of approximately 4749.0 items per prompt. In practice, our sampled subsets consist of around 650 items (13.7% of the capacity), demonstrating substantial operational headroom.\n\nWe also investigate the impact of the number of sampled items on Top-Rec's performance. As illustrated in the table below, we find that Top-Rec performs well even with a lower sampling ratio.\n\n| Sampling ratio  | R@50  | R@100 | CE@50 | CE@100 |\n|----|--------|-----------|-------|-------|\n|3%|0.0575|0.0840|**1.3284**|1.3845|\n|6%|0.0583|0.0840|1.3273|1.3830|\n|9%|**0.0586**|**0.0841**|1.3275|**1.3852**|\n\nIn other contexts, we expect the utility of ToP to remain robust against prompt length constraints. Our sampling method intrinsically mitigates such limitations, and the trend toward longer LLM contexts provides additional flexibility. In extreme cases, potential solutions include partitioned ToP construction based on auxiliary metadata (e.g., item categories).\n\n> Suggested rewrites and typos.\n\nWe greatly appreciate your valuable feedback. As you suggested, we have carefully reviewed the paper, addressed the imprecise writing, and corrected the typos. Additionally, we have enhanced the paper to ensure a clearer and more rigorous presentation.\n\n\n[1] Detecting Pretraining Data from Large Language Models\n\n[2] LLMRec: Large Language Models with Graph Augmentation for Recommendation\n\n[3] LLM4Rerank: LLM-based Auto-Reranking Framework for Recommendations"}, {"review_id": "n5qPf5Sw93", "reviewer": "Reviewer_2PtT", "summary": "This paper proposes a method for improving diversified recommendation by addressing potential data biases, such as exposure and selection bias, which may obscure users' underexplored preferences in observed interactions. The authors introduce the Tree of Preference, a hierarchical structure designed to guide LLMs in reasoning about user behavior from coarse to fine granularity. Based on the inferred preferences, the approach generates synthetic interactions to supplement training data and enhance diversity in recommendations. A dynamic user selection mechanism is also employed to improve training efficiency. Experimental results show that the method achieves competitive performance in terms of both diversity and relevance, with inference latency remaining within a reasonable range.", "questions": "See limitations.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths：\n1. The authors propose a tree-based structure to guide LLMs in reasoning about users’ latent preferences, which introduces a degree of novelty to the modeling approach.\n2. The figures and tables are clearly presented and easy to interpret, which aids understanding of the methodology and results.\n\nWeaknesses：\n1. While the paper acknowledges the presence of exposure and selection bias in user interaction data and aims to mitigate these biases through supplementary knowledge, it still relies entirely on offline datasets for training and evaluation. As a result, the test and validation sets may suffer from the same biases, making it difficult to assess whether the model truly captures users’ latent interests.\n2. The proposed method is highly heuristic in nature and heavily relies on a pretrained LLM without any task-specific fine-tuning, which may limit its adaptability and effectiveness.\n3. The approach depends critically on the constructed preference tree, yet the paper does not provide sufficient detail on how the tree is built, such as the number of levels, how nodes are generated, how many leaf nodes exist, or how many items each leaf node corresponds to. This lack of clarity significantly hinders reproducibility.\n4. The hyperparameter analysis does not include any discussion related to the construction of the preference tree, making it difficult to understand how different tree structures may affect the model’s performance. \n5. The proposed method assumes the availability of rich user profiles or user-related textual information. However, in many practical recommendation scenarios, such data may be limited or unavailable due to privacy constraints, which could restrict the applicability of the method.", "was_revised": true, "final_justification": "Based on the quality of the manuscript and the rebuttal, most of my concerns have been addressed. Authors should combine with rebuttal contents to enhance the manuscript. I recommend for 4: Borderline accept.", "rebuttal": "> W1: Test and validation sets may suffer from the same biases, making it difficult to assess whether the model truly captures users’ latent interests.\n\nWe sincerely appreciate your insightful comment. We acknowledge that conventional offline datasets may carry biases in test and validation sets. However, online validation typically requires randomized interventions (e.g., A/B testing), which present practical challenges such as high costs and reproducibility issues.\n\nTo rigorously evaluate ToP-Rec under biased test assumptions, we follow established work in debiasing evaluation [1], which aims to conduct unbiased evaluation when offline datasets exhibit selection or exposure bias.  We focus on relevance evaluation under these biased assumptions, using two strategies: Inverse Propensity Scoring Recall ($ R_{IPS} $) [2] and ATOP [3]. $ R_{IPS} $ weights observations using propensity scores, providing an unbiased recall estimator, while ATOP uses item set ranking, also proven to be unbiased for relevance. We report ToP-Rec's performance,  compared with both the backbone recommender and comparable baselines. The results show that ToP-Rec retains significant advantages under the assumption that test and validation sets may contain inherent biases.\n\n|   | $\\text{R}_\\text{IPS}$@50  | $\\text{R}_\\text{IPS}$@100 | ATOP |\n|----|--------|-----------|-------|\n|LightGCN$^*$  |0.0138|0.0188|0.4936|\n|LCD-UC|0.0122|0.0178|0.4915|\n|LLM4Re-A |0.0134|0.0181|0.4905|\n|Ours|**0.0161**|**0.0221**|**0.5008**|\n\nSince the calculation of category entropy depends solely on the distribution of categories in an item list and is not influenced by unobserved user-item interactions, our diversity evaluation remains unaffected by bias.\n\n> W2: The proposed method is highly heuristic in nature and heavily relies on a pretrained LLM without any task-specific fine-tuning, which may limit its adaptability and effectiveness.\n\nThank you for raising this important point, which inspired us to clarify Top-Rec's design.\n\nFirstly, we would like to emphasize that the key contribution of Top-Rec lies in its general framework rather than the incidental use of LLMs. The tree of preference serves as a structured vehicle for uncovering underexplored user preferences. Our experiments demonstrate that Top-Rec is compatible with a variety of LLMs, including smaller-scale models, which maintain robust performance (Please refer to our response to Q1 from Reviewer K1gm and W1 from Reviewer F6jv). This highlights the framework’s advantages.\n\nSecondly, we utilize pretrained LLMs because Top-Rec's fundamental objective is to harness their rich world knowledge for latent preference discovery, not as specialized solvers for complex tasks (e.g., mathematical reasoning or code generation). Since pretrained LLMs inherently satisfy this requirement, we find them well-suited to our design. Empirical evidence consistently confirms Top-Rec's effectiveness.\n\nFollowing your insightful suggestion, we further evaluated Top-Rec with fine-tuning. Specifically, we fine-tuned the LLM for user preference inference, which is a key step in the Top-Rec process. Using Twitter training data, we constructed a fine-tuning dataset where the LLM is instructed to infer preferences from users' historical interactions and the constructed Tree-of-Preference. We applied LoRA to fine-tune the LLM used in the paper and compared its performance with the original Top-Rec. The results showed that the fine-tuned version did not lead to significant improvements, confirming that pretrained LLMs are sufficiently effective while offering greater flexibility for diverse scenarios.\n\n|   | R@50  | R@100 | CE@50 | CE@100 |\n|----|--------|-----------|-------|-------|\n|Fine-tuned|0.0583|0.0829|1.3252|1.3841|\n|Ours|**0.0586**|**0.0841**|**1.3275**|**1.3852**|\n\n\n> W3: The paper does not provide sufficient detail on how the tree is built. \n\nWe sincerely appreciate your valuable comment. Regarding how nodes are generated, during the construction of the ToP, we prompt the LLM to generate a multi-level tree structure for user preferences. By instructing the LLM that each node represents a type of user preference and each edge represents a finer preference division, the hierarchical nodes are ultimately generated by the LLM. Please refer to our example prompt in Appendix A.2 and the actual ToP presented in our response to Reviewer K1gm, W2.\n\nBelow, we provide key architectural specifications of the tree of preference constructed on Twitter across different runs, including the number of leaf nodes, total nodes, depth, average items assigned per leaf, and average branching factor. To enhance reproducibility, we will revise the paper to include detailed quantitative information about the tree structure, along with a visualization of the ToP.\n\n| # leaf nodes  | # nodes | Depth  | # items per leaf  | Branch factor |\n|----|--------|-----------|-------|-------|\n|133.6|248.4|6.2|53.3|2.16|\n\n> W4: Hyperparameter analysis does not include discussions related to the construction of the preference tree.\n\nThank you for reminding us about this issue. We conduct evaluations to assess how the construction of the tree of preference influences performance. \n\nFirst, we examine how the number of sampled items provided to the LLM for ToP construction affects performance. As shown in the following table, we observe that Top-Rec achieves strong performance with a relatively small sampling ratio, which can be attributed to the fact that our k-means-based item sampling strategy helps maintain the representativeness of the samples.\n\n\n| Sampling ratio  | R@50  | R@100 | CE@50 | CE@100 |\n|----|--------|-----------|-------|-------|\n|3%|0.0575|0.0840|**1.3284**|1.3845|\n|6%|0.0583|0.0840|1.3273|1.3830|\n|9%|**0.0586**|**0.0841**|1.3275|**1.3852**|\n\nSecond, we evaluate the impact of the ToP structure by adjusting the prompt to modify the requirements for tree depth and branching number. In the two tables below, we report Top-Rec's performance on Twitter with varying tree depth and average branching factor across different runs. We find that shallow structures limit preference resolution, while excessive depth also restricts performance, potentially due to semantic fragmentation. Following a similar logic, the average branching factor exhibits a comparable trend, where relatively fewer or more branches lead to suboptimal performance.\nWe apologize for not clearly articulating this connection in the paper and will include a dedicated discussion of these findings in the revised version.\n\n| Depth  | R@50  | R@100 | CE@50 | CE@100 |\n|----|--------|-----------|-------|-------|\n|5.4|0.0565|0.0832|**1.3324**|**1.3886**|\n|6.2|**0.0586**|**0.0841**|1.3275|1.3852|\n|7.2|0.0572|0.0833|1.3295|1.3849|\n\n\n| Branch factor  | R@50  | R@100 | CE@50 | CE@100 |\n|----|--------|-----------|-------|-------|\n|2.01 |0.0571|0.0837|1.3269|**1.3893**|\n|2.16|**0.0586**|**0.0841**|1.3275|1.3852|\n|2.36|0.0584|0.0839|**1.3338**|1.3857|\n\n\n> W5: Availability of rich user profiles.\n\nThank you for your thoughtful comment. Top-Rec aligns with existing LLM-based recommendation methods [4,5] in terms of its reliance on basic user profile information, without requiring more detailed user data. This basic information is typically easy to obtain and leverage in industrial recommendation scenarios [6,7,8].\n\nMore importantly, following the approach in [4], we enhance user data by incorporating item information from the user’s historical interactions. These items serve as an additional, rich source of user information. We collect the textual features of items the user interacted with and provide them alongside the basic user profile information to the LLM for summarization. By focusing on user interests and filtering out noise, we obtain enhanced user textual data.\n\nTo evaluate the applicability of our method in scenarios where user profiles are missing, we conducted additional experiments. Specifically, we masked the user profiles and used only the LLM-generated summarization from the historically interacted items as user data. The table below presents the results on Twitter, showing that our method still outperforms strong baselines, demonstrating its robustness.\n\n|   | R@50  | R@100 | CE@50 | CE@100 |\n|----|--------|-----------|-------|-------|\n|LightGCN$^*$  |0.0567|0.0830|1.2841|1.3413|\n|LCD-UC|0.0517|0.0768|1.3154|1.3784|\n|LLM4Re-A|0.0562|0.0827|1.2855|1.3424|\n|w/o profile|$\\underline{0.0584}$|$\\underline{0.0839}$|$\\underline{1.3258}$|**1.3895**|\n|Ours|**0.0586**|**0.0841**|**1.3275**|$\\underline{1.3852}$|\n\n\n\n[1] Bias and Debias in Recommender System: A Survey and Future Directions\n\n[2] Unbiased offline recommender evaluation for missing-not-at-random implicit feedback\n\n[3] Training and Testing of Recommender Systems on Data Missing Not at Random\n\n[4] LLMRec: Large Language Models with Graph Augmentation for Recommendation\n\n[5] LLM4Rerank: LLM-based Auto-Reranking Framework for Recommendations\n\n[6] Large Language Model as Universal Retriever in Industrial-Scale Recommender System\n\n[7] Towards Open-World Recommendation with Knowledge Augmentation from Large Language Models\n\n[8] On Practical Diversified Recommendation with Controllable Category Diversity Framework"}]

Author rebuttal:

Meta-review: This paper introduces a novel LLM-based framework that addresses data bias in recommendation systems by constructing hierarchical "Tree of Preferences" structures to uncover users' underexplored interests and generate synthetic training data for improved diversity. The approach demonstrates strong empirical results, outperforming existing methods in 11 out of 12 comparisons across multiple datasets, while maintaining a practical separation between expensive LLM reasoning and efficient online serving. The work is technically sound with comprehensive evaluations and represents an original contribution to the intersection of LLMs and recommendation systems. While reviewers raised concerns about potential test set contamination, evaluation bias in offline datasets, and scalability constraints related to prompt length limitations, the authors provided satisfactory rebuttals that addressed most technical issues. Given the unanimous positive ratings from experienced reviewers, strong experimental validation, and the novelty of using LLMs to tackle recommendation diversity challenges, the paper is recommend for acceptance to NeurIPS. The authors should incorporate the suggested clarifications about tree construction methodology and include more detailed analysis of stability across different LLM configurations in the final version.
Decision: Accept (poster)