Density-based User Representation using Gaussian Process Regression for Multi-interest Personalized Retrieval

Authors: haolun  wu
ofer  meshi
masrour  zoghi
fernando  diaz
craig  boutilier
maryam  karimzadehgan
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-14 23:31:06

Abstract: Accurate modeling of the diverse and dynamic interests of users remains a significant challenge in the design of personalized recommender systems. Existing user modeling methods, like single-point and multi-point representations, have limitations w.r.t.\ accuracy, diversity, and adaptability. To overcome these deficiencies, we introduce density-based user representations (DURs), a novel method that leverages Gaussian process regression (GPR) for effective multi-interest recommendation and retrieval. Our approach, GPR4DUR, exploits DURs to capture user interest variability without manual tuning, incorporates uncertainty-awareness, and scales well to large numbers of users. Experiments using real-world offline datasets confirm the adaptability and efficiency of GPR4DUR, while online experiments with simulated users demonstrate its ability to address the exploration-exploitation trade-off by effectively utilizing model uncertainty.

Main paper: 
 introduction: 
 With the proliferation of online platforms, users have ready access to content, products and services drawn from a vast corpus of candidates. Personalized recommender systems (RSs) play a vital role in reducing information overload and helping users navigate this space. It is widely recognized that users rarely have a single intent or interest when interacting with an RS [1, 2, 3] . To enhance personalization, recent work focuses on discovering a user's multiple interests and recommending items that attempt to span their interests [3, 4, 5] . However, this is challenging for two reasons. First, user interests are diverse and dynamic: diversity makes it hard to detect all interests, while their dynamic nature renders determining which user interest is active at any given time quite difficult. Second, it is hard to retrieve items related to niche interests due to the popularity bias [6] .User representation is a fundamental design choice in any RS. The most widely used strategy for user modeling is the single-point user representation (SUR), which uses a single point in an item embedding space to represent the user. The user's affinity for an item is obtained using some distance measure (e.g., inner product, cosine similarity) with the point representing the item. However, SUR can limit the accuracy and diversity of item retrieval [7] ; hence, most RSs generally use high-dimensional embedding vectors (with high computation cost).To address the limitations of SUR, MaxMF [1] adopts a multi-point user representation (MUR), where each user is represented using K points in the embedding space, each reflecting a different "interest". MaxMF uses a constant, uniform K across all users (e.g., K = 4), which is somewhat ad hoc and restrictive. Subsequent research uses other heuristics [3, 8, 9, 4, 10, 11] or clustering algorithms [5, 2] to determine the number of interests per user. However, these all require the manual choice of K or a specific clustering threshold, limiting the adaptability of MUR methods, since interests generally have high variability across users. Moreover, uncertainty regarding a user's interests is not well-modeled by these methods, diminishing their ability to perform effective online exploration.Figure 1 : The t-SNE visualization of the prediction score between a picked user to all items in the MovieLens dataset. The score is computed as the inner-product between the user embedding and item embedding. The triangles (▲) indicate the latest 20 items interacted by the user. We use Matrix Factorization (MF) to obtain embeddings in this toy example. As depicted, only density-based method (bottom row) can well capture user interests with uncertainty.To address limitations of SUR and MUR pointbased representations, we propose a user representation that emphasizes (i) adaptability, adapting to different interest patterns; (ii) uncertaintyawareness, modeling uncertainty in assessing user interests; and (iii) efficiency, avoiding highdimensional embeddings. Specifically, we use a density-based representation, where the user's preferences are encoded using a function over the item embedding space. Under this representation, the relevance score for user-item pairs should be higher in regions of embedding space where a user demonstrated more interest in the corresponding items, and lower in regions where users have shown limited interest. We propose the density-based user representation (DUR), a novel user modeling method which exploits Gaussian process regression (GPR) [12, 13] , a well-studied Bayesian approach to non-parametric regression using Gaussian processes (GPs) to extrapolate from training to test data. GPR has been applied across a wide range of domains, though it has been under-explored in user modeling. Given a sequence of user interactions, GPR predicts the level of a user's interest in any item using its posterior estimates. This allows us to maintain a unique personalized GP regressor for each user, effectively capturing their evolving preferences and assessing uncertainty. Top-N item retrieval is performed using bandit algorithms, such as UCB [14] or Thompson sampling [15] , based on GPR posterior estimates.To illustrate, consider Fig. 1 , which shows the prediction score (i.e., the inner-product between user and item embeddings) between a user and all movies from the MovieLens 1M dataset [16] (reduced to 2D for visualization purposes). We examine 20 movies from the recent history of a particular user, shown as triangles (▲). We see that these movies lie in several different regions of the embedding space. However, when we fit either SUR or MUR (K = 4) models, they fail to capture the user's multiple interests and instead assign high scores only to movies from a single region (Fig. 1 , top row). By contrast, we see in Fig. 1 (bottom left) that GPR fits the data well, assigning high values to all regions associated with the user's recent watches (interests). Fig. 1 (bottom right) shows that our approach can also capture uncertainty in our estimates of a user's interests, assigning high uncertainty to regions in embedding space with fewer samples.Our approach has various desirable properties. First, it adapts to different interest patterns, since the number of interests for any given user is not set manually, but determined by GPR, benefiting from the non-parametric nature of GPs. Second, the Bayesian nature of GPs measures uncertainty with the posterior standard deviation of each item. This supports the incorporation of bandit algorithms in the recommendation and training loop to address the exploration-exploitation trade-off in online settings. Finally, our method can effectively retrieve items spanning multiple user interests, including "niche" interests, while using a lower-dimensional embedding relative to SUR and MUR.To summarize, our work makes the following contributions:• We develop GPR4DUR, a density-based user representation method, for personalized multiinterest retrieval. It is the first use of GPR for user modeling in this setting.• We propose new evaluation protocols and metrics for multi-interest retrieval that measure the extent to which a model captures a user's multiple interests.• We conduct comprehensive experiments on real-world offline datasets showing the adaptability and efficiency of GPR4DUR. Online experiments with simulated users show the value of GPR4DUR's uncertainty representation in balancing exploration and exploitation. 
related work: 
 Learning high-quality user representations is central to good RS performance. The single-point user representation (SUR) is the dominant approach, where a user is captured by a single point in some embedding space [17, 18] , for example, as employed by classical [19, 20, 21] and neural [22] collaborative filtering methods. While effective and widely used, SUR cannot reliably capture a user's multiple interests. To address this limitation, the multi-point user representation (MUR) [1] has been proposed, where a user is represented by multiple points in embedding space, each corresponding to a different "primary" interest. Selecting a suitable number of points K is critical in MUR. Existing algorithms largely use heuristic methods, e.g., choosing a global constant K for all users [1, 8, 3, 9, 4, 10, 11] . Other methods personalize K by setting it to the logarithm of the number of items with which a user has interacted [5] . More recently, Ward clustering of a user's past items has been proposed, with a user's K determined by the number of such clusters [2] . This too requires manual tuning of clustering thresholds.At inference time MUR is similar to SUR, computing the inner-product of the user embedding(s) and item embedding. Some methods compute K inner products, one per interest, and use the maximum as the recommendation (and the predicted score for that item) [1] . Others first retrieve the top-N items for each interest (N × K items), then recommend the top-N items globally [3, 2] . None of these methods capture model uncertainty w.r.t. a user's interests, hence they lack the ability to balance exploration and exploitation in online recommendation in a principled way [23] .Our density-based user representation, and our proposed GPR4DUR, differs from prior work w.r.t. both problem formulation and methodology. Most prior MUR methods focus on next-item prediction [3, 11] , implicitly assuming a single-stage RS, where the trained model is the main recommendation engine. However, many practical RSs consist of two stages: candidate selection (or retrieval) followed by ranking [24, 25] . This naturally raises the question: are the selected candidates diverse enough to cover a user's interests or intents? This is especially relevant when a user's dominant interest at the time of recommendation is difficult to discern with high probability; hence, it is important that the ranker have access to a diverse set of candidates that cover the user's range of potential currently active interests. In this paper, we focus on this retrieval task. As for methodology, almost all prior work uses SUR or MUR point-based representations, [1, 2, 3, 5] -these fail to satisfy all the desiderata outlined in Sec. 1. Instead, we propose DUR, a novel method satisfying these criteria, and, to the best of our knowledge, the first to adopt GPR for user modeling in multi-interest recommendation/retrieval. We frame our solution as a candidate generator to be used in the retrieval phase of an RS. Other candidate generators with different objectives can be used in parallel to ours.A related non-parametric recommendation approach is k-nearest-neighbors (kNN), where users with similar preferences are identified (e.g., in user embedding space), and their ratings generate recommendations (see, e.g., [26] ). Our approach differs by not using user embeddings, but fitting a GPR model directly to item embeddings (see Sec. 4.4), allowing for uncertainty in the user model. 
formulation and preliminaries: 
 In this section, we outline our notation and multi-interest retrieval problem formulation, and provide some background on GPR, which lies at the core of our DUR method.Notation. We consider a scenario where each item is associated with category information (e.g., genre for movies). Denote the set of all users, items, and categories by U, V, and C, respectively. For each u ∈ U, whose interaction history has length l u , we partition the sequence of items V u in u's history into two disjoint lists based on the interaction timestamp (which are monotonic increasing): (i) the history setV h u = [v u,1 , v u,2 , ..., v u,ℓ u] serves as the model input; and (ii) the holdout setV d u = [v u,ℓ u +1 , v u,ℓ u +2 , ..., v u,l u ]is used for evaluation. We define u's interests C(V u ) to be the set of categories associated with all items in u's history. Our notation is summarized in Appendix A.1. Problem Formulation. We formulate the multi-interest retrieval problem as follows: given V h u , we aim to retrieve the top N items R u (i.e., |R u | = N ) w.r.t. some matching metric connecting R u and V d u that measures personalized retrieval performance. We expect R u to contain relevant items, and given our focus on multi-interest retrieval, R u should ideally cover all categories in C(Vd u ).Our problem is related to sequential recommendation, where the input is a sequence of interacted items sorted by the timestamp per user, and the goal is to predict the next item with which the user will interact. However, here we focus on retrieving a set of items that cover a user's diverse interests. We defer the task of generating a precise recommendation list to a downstream ranking model.The core of our model is Gaussian process regression (GPR). A Gaussian Process (GP) is a non-parametric model for regression and classification. GPR models a distribution over functions, providing not only function value estimates for any input, but also uncertainty quantification via predictive variance, enhancing robust decision-making and optimization [27] .The key components of the GP are the mean function and the covariance (kernel) function, which capture the prior assumptions about the function's behavior and the relationships between input points, respectively [13] . Let X = {x 1 , . . . , x n } ∈ R n×d be a set of input points and y = {y 1 , . . . , y n } ∈ R n be the corresponding output values. A GP is defined as: f ∼ GP(µ, k), where µ(x) is the mean and k(x, x ′ ) is the covariance function. The joint distribution of the observation and the output at a new point is derived using a Gaussian distribution, as illustrated in Appendix A.1.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? Answer: [NA]• Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
methodology: 
 We outline density-based user representation using GPR and its application to multi-interest retrieval.We use GPR to construct a novel density-based user representation (DUR) for multi-interest modeling in RSs. Our key insight involves using GPR to learn a DUR, using a user's interaction history, that naturally embodies their diverse interest patterns. For any user u, letV u = [v u,1 , v u,2 , ..., v u,l u ] ∈ Rl u ×d be the embeddings of all items in their history. This is derived from their interaction list V u and an item embedding matrix V (we describe how to obtain V in Sec. 4.4). Leto u = [o u,v |v ∈ V u ] ∈ Rl u be the vector of u's observed interactions with items in V u . We employ a GP to model u's interests given the input points V u and corresponding observations o u , g u ∼ GP(µ u , k u ), where, g u , µ u and k u are u's personalized GP regressor, mean, and kernel function, respectively. The joint distribution of observation o u and the predicted observation of a novel item v * is:EQUATIONFor simplicity, we assume µ u = 0 and a commom kernel function and variance across all users. Assuming implicit feedback, we have o u = 1, i.e., u shows "interest" in all interacted items. Thus, the posterior (prediction) g u (v) for any v ∈ V is:EQUATIONwhere the GP mean ḡu and variance cov(g u ) are:EQUATIONEQUATIONAfter obtaining a DUR g u for u ∈ U using GPR, we generate the retrieval list using the posterior g u (v) over all unobserved items. The top N items with the highest values form our retrieval list. We consider two methods for selection: (i) Thompson sampling (TS), a probabilistic method that selects items based on posterior sampling [15] and (ii) Upper Confidence Bound (UCB), a deterministic method that selects items based on their estimated rewards and uncertainties [14, 28] . The details for the retrieval list generation is shown in Appendix A.2.The free parameters in our model are the kernel function k and the standard deviation σ in Eq. 1. We treat these as hyperparameters of GPR which are optimized using evaluation on a separate holdout set. Specifically, to generate R u for user u, we fit the GPR model not to the complete interaction history V u , but to the reduced history V h u , using the item embeddings and observed ratings. We assess retrieval performance using specific metrics (see Sec. 5.3) on both R u and the holdout set V d u . We tune GPR parameters using these criteria (we detail the adjustable parameters in Sec. 5.2).Following [2] , we assume that item embeddings are fixed and precomputed: this ensures rapid computation and real-time updates at serving time. For item embedding pre-training, we use extreme multi-class classification [3, 29, 30] : given a training sample (u i , v j ), we first compute the likelihood of u i interacting with v j , i.e., p(v j |u i ) = exp(u⊺ i v j )/ ∑ v ′ ∈V exp(u ⊺ i v ′ ), where u i and v j are embeddings of u i and v j . Our objective is to maximize the log-likelihood of a user interacting with their items. Moreover, we want to ensure that item embeddings align with their categories since categories explicitly indicate a user's interests. We capture item-category information by computing the likelihood that an item belongs to a category in a similar way:p(c k |v j ) = exp(v ⊺ j c k )/ ∑ c ′ ∈C exp(v ⊺ j c′ ). The overall objective for pre-training combines the two negative log-likelihoods using a scaling factor γ:EQUATIONwhere V u i is the set of items that u i interacted with and C v j is the set of categories that v j belongs to. The full item embedding matrix V is jointly learned and fixed after the pre-training phase. We do not use user and category embeddings after pre-training; other schemes, such as using item co-occurrence information to derive item embeddings without user or category embeddings, are possible. The full architecture for GPR4DUR is depicted in Figure 2 .The complexity of GPR is O((ℓ u )3 ), where ℓ u is the number of training samples, due to covariance matrix inversion (Eq. 3 and Eq. 4). This inversion is required only once and is manageable with a few thousand examples. If the history length is too large, we either select a representative subset of interactions or focus on the most recent interactions (we adopt this latter approach in our experiments). Inference costs for each test point are O(ℓ u ) for mean and O((ℓ u )2 ) for variance prediction, thus linear in the number of test points |V|. As shown in Appendix A.7, our method has reasonable computational costs, slightly higher than strong baselines but with better performance on retrieval and ranking, demonstrating its applicability in real-world scenarios. Our approach is best-suited Table 1 : Result comparison on the retrieval task. For the same metric on each dataset, the best is bold and the second best is underlined. We use four different symbols to indicate the different categories of methods detailed in Sec. 5.4. Cases where our model significantly outperforms the best baseline, with p ≤ 0.01 according to the paired t-test, are marked with *.Interest Coverage (IC@k) Interest Relevance (IR@k) Exp. Deviation (ED@k) Tail Exp. Improv. (TEI@k) The higher the better ⇑The higher the better ⇑ The lower the better ⇓ The higher the better ⇑ k=20 k=50 k=100 k=20 k=50 k=100 k=20 k=50 k=100 k=20 k=50 k=100Amazon to scenarios with millions or tens of millions of items, when retrieving hundreds for ranking, as is common in real-world RSs (e.g., see [31] ). Similarly, in music recommendation, it would be especially useful for defining user interests over artists (a few millions at most per geographical region/language) vs. individual songs. Previous work improves GPR efficiency using Toeplitz and Kronecker structures, achieving O(ℓ u ) training and O(1) prediction complexity [32] . Other sublinear approximations partition the item space to find candidates adaptively [33, 34] . While not our focus, it would be straightforward to apply these well-studied methods to our setting.♣Finally we note that we do not expect users' interest collections to be highly dynamic; hence, they will not require real-time updates. Our method is best-suited to use as a candidate generator, with the model updated periodically to create a pool of potentially interesting items for users. 
offline experiments: 
 We next evaluate our GPR4DUR on three real-world datasets, and compare our DUR technique with other state-of-the-art methods for multi-interest personalized retrieval.We use three widely studied datasets: Amazon [35] , MovieLens [16] , and Taobao [36] . Following previous work [37, 38] , we filter out items with fewer than 10 appearances and users with fewer than 25 interactions to ensure sufficient history for indicating multiple interests. Each interest corresponds to a category: for Amazon, we use the single principal category of each item; for MovieLens, we use 18 movie genres (each movie can belong to multiple genres); for Taobao, we cluster items into 20 categories using Ward clustering on the pretrained item embeddings as in [2] . The same categories are used during training and inference. Dataset statistics are shown in Table 6 , and an ablation analysis of various clustering algorithms and numbers of clusters is provided in Table 8 (See Appendix).Most prior work on recommendation and retrieval evaluates model performance on generalization to new items per user. Instead, similar to recent work [39, 3] , we assess model performance on the ability to generalize to new users. We split users into disjoint subsets: training users (U train ), validation users (U val ), and test users (U test ) in a ratio of 8:1:1. The last 20% of each user's interaction sequence is treated as a holdout set for evaluation, and the first 80% as a history set for fitting the GPR model. We cap history length to 60, 160, and 100 for the three datasets, aligning with their average user interactions. For item embedding pre-training, we train the recommendation backbones using the history set of all training users and tune parameters based on the performance on the holdout set for validation users (details in Appendix A.3). Training is capped at 100,000 iterations with early stopping if validation performance does not improve for 50 successive iterations. We tune GPR hyperparameters by fitting the GP regressor to the history set of all training and validation users, then using the holdout set to fine-tune parameters (kernel and standard deviation). Metrics are reported on the holdout set for all test users (see Appendix A.5 for hyperparameter sensitivity).Metrics used in prior work on sequential recommendation are not well-suited to assess performance in our multi-interest retrieval task, for several reasons: (i) Conventional metrics, such as precision and recall, often employed in multi-interest research, do not adequately quantify whether an item list reflects the full range of a user's (multiple) interests. A model may primarily recommend items from a narrow range of highly popular categories and still score high on these metrics while potentially overlooking niche interests. (ii) Metrics like precision and recall are overly stringent, only recognizing items in the recommendation list that appear in the holdout set. We argue that credit should also be given if a similar, though not identical, item is recommended (e.g., Iron Man 1 instead of Iron Man 2). This requires a soft version of these metrics. (iii) Multi-interest retrieval systems should expose users to niche content to address their diverse interests. However, conventional metrics may neglect the item perspective, potentially underserving users with specialized interests. Consequently, we propose the use of the following four metrics that encompass the aforementioned factors.Interest-wise Coverage (IC) This metric is similar to subtopic-recall [40] , which directly measures whether the model can comprehensively retrieve all user interests reflected in the holdout set. The higher the value of this metric the better:EQUATIONInterest-wise Relevance (IR) To further measure the relevance of retrieved items, we introduce a "soft" recall metric, calculating the maximum cosine similarity between items in the retrieval list and the holdout set within the same category. The motivation for IR is that the success of a retrieval or recommendation list often depends on how satisfying the most relevant item is:EQUATIONwhere S(v i , v j ) is the cosine similarity between item v i and v j . To obtain ground-truth similarities between items, and to mitigate the influence of the chosen pre-trained model, we pretrain the item embeddings using YoutubeDNN [41] with a higher dimension size (d = 256) to compute a uniform S i,j for any backbone. A higher value of this metric is better.Exposure Deviation (ED) In addition to measuring performance from the user side, we also measure from the item side to test whether exposure of different categories in the retrieval list is close to that in the holdout set. We treat each occurrence of an item category as one unit of exposure, and compute the normalized exposure vectors ϵ * u , ϵ 1∶k u ∈ R |C| for u's holdout set and retrieval list, respectively. Lower values of this metric are better.EQUATIONTail Exposure Improvement (TEI) With respect to category exposure, it is crucial to ensure that niche interests are not under-exposed. To evaluate this, we select a subset of the least popular categories and measure their exposure improvement in the retrieval list versus that in the holdout set. A higher value indicates better performance, and a positive value indicates improvement:EQUATIONHere, C tail refers to the set of niche categories (i.e., the last 50% long-tail categories), denoting those niche interests. 1 ϵ * u,c >0 indicates that we only compute the improvement for categories that appear in the user's holdout set, reflecting their true interests. Overall Performance Our overall performance comparison addresses two central research questions (RQs): whether our proposed method offers superior retrieval performance for users with multiple interests (RQ1); and whether it induces appropriate item-sided exposure w.r.t. both popular and niche interests (RQ2).Performance on the retrieval task. To assess RQ1, we evaluate the effectiveness of the retrieval phase. As shown in Table 1 , GPR4DUR outperforms almost all baselines across datasets w.r.t. Interest Coverage (IC@k) and Interest Relevance (IR@k), demonstrating high degrees of retrieval coverage and relevance. Specifically, on the Amazon dataset, GPR4DUR achieves the highest performance in 4 out of 6 interest metrics, in MovieLens it leads in 5 out of 6 metrics, and in Taobao, it excels across all 6 interest metrics. This shows that GPR4DUR covers a wide range of user interests while simultaneously maintaining high relevance. To assess the statistical significance of our model's performance compared to the best baseline, we performed a paired t-test, which evaluates whether the means of two paired samples differ significantly, ensuring that the observed improvements are not due to random variation. For RQ2, our objective is to ascertain whether item exposure is suitably balanced. We measure this using the Exposure Deviation (ED@k) and Tail Exposure Improvement (TEI@k) metrics. Lower values of ED@k suggest more satisfying category exposure, while higher values of TEI@k are indicative of enhanced exposure in the long tail item categories. GPR4DUR is highly effective on these metrics, consistently performing well across all datasets in most cases, and validating its ability to provide an optimal level of category exposure. Notably, GPR4DUR achieves the best results in 8 of 9 TEI@k metrics, indicating its superior ability to generate exposure to niche categories/interests. We point out, however, that all TEI@k values are negative, which suggests that none of the methods we tested improve exposure for niche categories relative to the exposure in the user holdout sets. This is explained by the well-known popularity bias inherent in most recommendation methods, and suggests that further work is required on diversification strategies to mitigate such effects.Performance on the ranking task. While our primary emphasis is on the retrieval phase, we also present model comparisons for the ranking task. As illustrated in Table 2 , GPR4DUR demonstrates competitive performance on the traditional relevance metrics relative to the baseline models. This further validates the strength of our proposed method.Performance across User Groups (RQ3) We conduct a fine-grained analysis of GPR4DUR performance on users grouped into quantiles, g1 (lowest) through g5 (highest), based on the number of interaction or number of interests; results are provided in Fig. 3 . Due to space constraints, we show results only for IR@20 and TEI@20 on MovieLens, and only plot the best SUR and MUR strategies; DUR denotes our GPR4DUR method. We see that GPR4DUR consistently outperforms the baselines w.r.t. relevance and exposure metrics. This improvement is more pronounced as the number of user interactions (resp., interests) increases. Interestingly, while no model enhances overall exposure of niche interests (see Table 1 ), GPR4DUR improves niche interest exposure considerably for users with large numbers of interactions (resp., interests); i.e., TEI@20 is positive for g4 and g5. These observations confirm the effectiveness of GPR4DUR in capturing a user's multiple interests-especially for those with non-trivial histories-and its potential to maintain fair exposure w.r.t. items.We underscored the importance of efficiency for a good user representation in Sec. 1. To shed light on this, we examine the IC@20 and IR@20 performance of various methods on Amazon. As shown in Fig. 4 , GPR4DUR is effective w.r.t. interest coverage even when operating with low-dimensional embeddings (i.e., d = 8 and d = 4). By contrast, the performance of the SUR and MUR methods degrades as the dimensionality decreases. GPR4DUR consistently outperforms other methods w.r.t. interest relevance across all dimensionalities examined. We note that lower dimensions facilitate higher interest relevance due to increased cosine similarity (Eq. 7), explaining the inverse correlation between IR@20 and dimension size in Figure 4 (right). 
online simulation: 
 To demonstrate the efficacy of GPR4DUR in capturing uncertainty in user interests and support exploration, we conduct an online simulation in a synthetic setting, using a specific model of stochastic user behavior to generate responses to recommendations.Data Preparation. We assume |C| = 10 interest clusters, each represented by a d-dim (d = 32) multivariate Gaussian. We randomly select a (user-specific) subset of these interests as the groundtruth interest set for each user. We set |U| = 1000 and |V| = 3000, with 300 items in each interest cluster. Each item belongs to a single cluster and its embedding is sampled from the corresponding interest distribution. Each user is modeled by a multi-modal Gaussian, a weighted sum of the corresponding ground-truth interest distributions. To simulate a sequence of item interactions V u (i.e., user history), we follow [46] , first running a Markov Chain using a predefined user interest transition matrix to obtain the user's interest interactions for S = 10 steps. We do not consider the cold-start problem in this experiment, so we simply recommend one item from each generated cluster to form the user history (i.e., |V u | = S). The observation o u over items in V u is set to 1 if the item belongs to a ground-truth cluster, and -1 otherwise.GPR Fit and Prediction. After obtaining item embeddings V, user history V u , and user observations o u , we use GPR4DUR to learn a DUR for each user, using the methods described in Sec. 4.User History and Observation Update. Using a predetermined user browsing model, clicked and skipped items are generated and appended to the user interaction history (and the corresponding user observation is likewise updated, 1 for clicked, -1 for skipped). This process continues until a maximum iteration of T = 10 is reached. In this online setting, we adopt the dependent click model (DCM) of user browsing behavior, widely used in web search and recommendation [47, 48] . In the DCM, users begin by inspecting the top-ranked item, progressing down the list, engaging with items of interest and deciding to continue or terminate after each viewed item. 3 . Specifically, we compute a "cumulative" version of interest coverage by reporting the interest coverage averaged across all users on all previously recommended items prior to the current iteration. Our goal is to test whether policies that use uncertainty models outperform those that do not, specifically, whether such policies can exploit the inherent uncertainty representation of user interests offered by GPR4DUR.We assume each policy recommends the top-10 items to each user at each iteration. Table 3 shows that methods using uncertainty (the bottom three rows) fairly reliably outperform those that do not (the top two rows, with Greedy being UCB with β = 0, where β is the scaling factor of the variance term in UCB). The observation confirms the benefit of using GPR4DUR to explicitly model uncertainty in a recommender's estimates of a user's interests and to use this to drive exploration. 
conclusion and discussion: 
 In this paper, we introduced a density-based user representation model, GPR4DUR, marking the first application of Gaussian process regression for user modeling in multi-interest retrieval. This innovative approach inherently captures dynamic user interests, provides uncertainty-awareness, and proves to be more dimension-efficient than traditional point-based methods. We also establish a new evaluation protocol, developing new metrics specifically tailored to multi-interest retrieval tasks, filling a gap in the current evaluation landscape. Offline experiments validate the adaptability and efficiency of GPR4DUR, and demonstrate significant benefits relative to existing state-of-the-art models. Online simulations further highlight GPR4DUR's ability to drive user interest exploration by recommendation algorithms that can effectively leverage model uncertainty. The future work and broader impacts of this study are discussed in Appendix A.9 and Appendix A.10, respectively. 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA] Justification: This study poses no such risks. Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort. 12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? Answer: [Yes] Justification: We properly cited all benchmarks used in this study. Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. 
 
 Appendix: A.1 Notations and PreliminariesTable 4 : Description of Notation.Notation DescriptionThe set of all users, items, and (item) categories. U, V, C Full embedding matrix of users, items, and categories.V uThe user u's interaction history.V h u , V d uThe history set and holdout set partitioned from V u . o uThe observed rating scores of u on items inV u . t u,vThe time step when u interacted with v. l uThe length of V u . ℓ uThe length of user history for model input.V u Item embeddings for items in V u . R uThe list of retrieved items to u.The set of categories of all items in the input sequence.Let X = {x 1 , . . . , x n } ∈ R n×d be a set of input points and y = {y 1 , . . . , y n } ∈ R n be the corresponding output values. A GP is defined as:EQUATIONwhere µ(x) is the mean function and k(x, x ′ ) is the covariance function (kernel). Given a new point x * , the joint distribution of the observed outputs and the output at the new point is given by:EQUATIONwhere µ(X) is the vector of mean values for the observed data points, K(X, X) is the covariance matrix for the observed data points, k(X, x * ) is the vector of covariances between the observed data points and the new input point, and σ 2 and σ 2 * are the noise variances. Without prior observations, µ is generally set as 0.The conditional distribution of f (x * ) given the observed data is:EQUATIONwith the predictive mean and covariance given by:EQUATIONEQUATION)Fig. 5 presents a visual illustration of GPR. The true underlying function is depicted in red, which is the function we aim to approximate through GPR. The observations, depicted as black crosses, represent known data points. As expected with GPR, where data points are observed, the uncertainty (represented by the shaded region) is minimal, signifying high confidence in predictions at those locations. On the other hand, in areas without observations, the uncertainty increases, reflecting less confidence in the model's predictions. The two dashed lines represent samples from the GP posterior. Around observed points, these sampled functions adhere closely to the actual data, representing the power and flexibility of GPR in modeling intricate patterns based on sparse data.After obtaining a DUR g u for u ∈ U using GPR, we generate the retrieval list using the posterior g u (v) over all unobserved items. The top-N items with the highest values are selected as our retrieval list. We consider two methods for estimating these values. The first is Thompson sampling (TS), a probabilistic method that selects items based on posterior sampling [15] . For each item v in a set, we generate a sample from the posterior g u (v), and rank the items using their sampled values. A key advantage of TS is its ability to balance the trade-off between exploration and exploitation, improving the diversity of the recommendation list. The sampling and selection process is:EQUATIONwhere s u,v is the value for item v from user u's sampled function, and R u is the final list of retrieved items for user u.The second method is Upper Confidence Bound (UCB), a deterministic method that selects items based on their estimated rewards and uncertainties [14, 28] . For each item v, we compute its upper confidence bound by adding the mean and a confidence interval derived from the variance of the posterior g u (v); items are ranked using these upper bounds. Unlike TS, UCB tends to aggressively promote items with a high degree of posterior uncertainty, giving a different flavor of diversity in the recommendation list. The selection process is:EQUATIONEQUATIONwhere b u,v is the upper confidence bound for v w.r.t. u's posterior, and β is a hyper-parameter that adjusts the exploration-exploitation trade-off. R u is the final retrieval list.Necessity of Pretraining Item Embeddings with Auxiliary Loss on Categories. Without augmenting the pre-training loss with categorical information as in Eq. 5, the resulting pre-trained embeddings do not respect category consistency. To illustrate this, we compare pre-trained embeddings with and without categorical information (on MovieLens) and show the results in Fig. 6 .We observe that, without categorical constraints, the learned embeddings do not align with the categories. There are still overlap between categories (genres) since each movie may belong to multiple genres. Hence, incorporating an auxiliary training loss based on item categories is essential for interest-coverage analysis based on the embeddings.Sensitivity to Different Pretraining Strategies. In this work, we use pretrained embeddings for our multi-interest user representation. To ensure a fair comparison, we use the same pretrained embeddings in all other models we compared. To test the impact of the embeddings on performance, we add experiments with other pretraining settings. Specifically, we obtain embeddings using matrix factorization (MF) and using YoutubeDNN with varying history lengths. We use the Amazon dataset as an example and measure performance at top-50. The observation is similar on the other two datasets. Other experiment settings are the same as presented in the paper. We use the learned Figure 6 : The t-SNE plot for pretrained item embeddings on the MovieLens dataset with and without using auxiliary loss on category information.Table 5 : Performance comparison of different pretraining settings on various metrics. ⇑ indicates the higher the better; ⇓ indicates the lower the better.Pretraining setting Retrieval Task Ranking Task IC@50 ⇑ IR@50 ⇑ ED@50 ⇓ TEI@50 ⇑ Recall@50 ⇑ nDCG@50 ⇑ embeddings with both ComiRec (one of the strongest baselines) and our GPR4DUR. The results, shown in Table 5 , demonstrate that embeddings from YoutubeDNN outperform those from MF and that longer history leads to better performance. These results highlight the effectiveness and robustness of our proposed method across different pretraining settings.The statistics of the three datasets, Amazon [35] , MovieLens [16] , and Taobao3 , is shown in Table 6 . We report the number of users, number of items, number of total interactions, and density of the three datasets, respectively.In short, we constantly use the radial basis function (RBF) kernel on different datasets due to its superior performance, where σ is unique for each dataset.The kernel function, k, is tuned from the cosine similarity kernel and the RBF kernel. For the RBF kernel, the standard deviation σ is tuned from {1e -2 , 1e -1 , 1, 10, 100}. For each dataset, we report one metric on the retrieval task and one metric on the ranking task, both evaluated at the top-50. The trend on other metrics is similar.As shown in Table 7 , we observe that the RBF kernel outperforms the cosine similarity kernel across various datasets and metrics because it better captures non-linear relationships and offers flexibility by mapping inputs into an infinite-dimensional space. Additionally, the σ parameter in the RBF kernel influences model performance-a larger σ yields smoother functions for better generalization over distances, while a smaller σ detects finer data variations. Dataset-specific optimal σ values correlate with data sparsity. MovieLens's high density demands a smaller σ to avoid overfitting due In this section, we provide the ablation analysis on using various clustering algorithms and different number of clusters on the Taobao dataset. Following paper [49] , we have selected four distinct clustering algorithms for our comparative analysis: (i) Ward [50] : This hierarchical clustering technique focuses on minimizing the variance within each cluster, effectively reducing the overall sum of squared distances across all clusters. (ii) K-Means [51] : An iterative clustering algorithm that aims to minimize the sum of squared distances from each data point to the centroid of its assigned cluster, thereby reducing in-cluster variance. (iii) Spectral Clustering [52] : This method performs clustering based on the eigenvectors of the normalized Laplacian, which is computed from the affinity matrix, facilitating the division based on the graph's inherent structure. (iv) BIRCH [53] : A hierarchical clustering approach that efficiently processes large datasets by incrementally constructing a Clustering Feature Tree, which groups data points based on their proximity and other features.A notable deviation from paper [49] is in our handling of categories (interests) for training and inference phases; we maintain consistency in categories across both phases, in contrast to their separate categorization. We further refine our analysis by tuning the number of clusters from 10, 20, and 50.Result and analysis. The results presented in the Table 8 indicate a nuanced understanding of the impact of clustering algorithms and the number of clusters on retrieval and ranking tasks in information retrieval systems. Across the board, the differences among the clustering algorithms-Ward, K-Means, Spectral, and BIRCH-are relatively small, suggesting that the choice of clustering algorithm might not be as critical as the configuration of the number of clusters within these algorithms for the tasks at hand.Notably, as the number of clusters increases from 10 to 20 and then to 50, there is a general trend of improvement in performance metrics. This improvement could be attributed to the algorithms' ability to capture more fine-grained user interests through a larger number of clusters, thereby enhancing both retrieval accuracy and ranking precision.However, the marginal differences observed between the configurations of 20 and 50 clusters across all metrics indicate diminishing returns on further increasing the number of clusters beyond 20. This observation suggests that while increasing the number of clusters to 20 contributes to significant improvements, escalating to 50 clusters does not yield proportionally higher gains. Consequently, this study opts for a configuration of 20 clusters as the optimal balance between performance improvement and computational efficiency. The relatively stable standard deviations across different configurations further support the robustness of our findings, emphasizing the consistency of the performance improvements achieved with an increased number of clusters up to a certain point.We show the efficiency comparison (with the performance comparison) across models in Table 9 .For a fair comparison, we run all experiments on a single NVIDIA A100 GPU with TensorFlow framework (version 1.12) without any further optimization on the computation. We choose the Taobao dataset and set the batch as 1.Result and analysis. In examining the efficiency of various recommendation models, notable differences in training and inference times highlight the diverse computational demands across these models. Models like YoutubeDNN and GRU4Rec present a more efficient profile, offering lower latency during both training and inference phases, while perform the worst on the retrieval and ranking tasks. The variability in model efficiency, as seen in the standard deviations of training and inference times, indicates a potential fluctuation in latency that could impact real-world deployment.Our proposed model, GPR4DUR, stands out with its superior performance on the retrieval task (e.g., measured by interest-wise relevance, IR@50) and the ranking task (e.g., measured by recall, Recall@50). This suggests that for applications valuing enhanced retrieval and ranking accuracy, the trade-off of increased latency is acceptable. Additionally, GPR4DUR exhibits a low standard deviation in its timing metrics. This consistency in processing times is advantageous for online deployment. It offers a predictable performance, which is crucial for developers when balancing between model efficiency and effectiveness.Conclusively, GPR4DUR, along with the other baseline models evaluated, meets the latency requirements for online application without the need for additional computational and serving optimizations. While GPR4DUR exhibits a higher time cost relative to some baselines, its performance improvement offers a compelling advantage.Table 10 : Result comparison between GPR4DUR and generative recommendation methods that also consider uncertainty. Here we use the Amazon dataset as an example to report the results and we confirm the observation is similar on the other two datasets. ⇑ indicates the higher the better; ⇓ indicates the lower the better.Retrieval Task Ranking Task IC@50 ⇑ IR@50 ⇑ ED@50 ⇓ TEI@50 ⇑ Recall@50 ⇑ nDCG@50 We note that some generative recommendation methods also consider user uncertainty, making them potential candidates for comparison with our method. These methods include VAECF [54] , which assumes a distribution over user/item representations, and more recently diffusion-based recommendation models, DiffuRec [55] and DREAM [56] . We do not compare them in the main paper due to (i) different design purpose: these methods are not designed for multi-interest retrieval, which is the focus of this work, and (ii) different model family: VAECF is a parametric model while our model is non-parametric, and diffusion-based methods require an entirely different training procedure and sampling strategies.To still satisfy this curiosity, we present a comparison between GPR4DUR and the generative recommendation methods on the Amazon dataset for both the retrieval task and ranking task. As shown in Table 10 , the results still confirm the effectiveness of our model.In Sec. 4.5, we discussed methods to reduce the time complexity of traditional Gaussian Process Regression (GPR). Despite these efforts, the training and inference times presented in Table 9 remain slightly higher than those of recent state-of-the-art methods. Further improving the efficiency of our approach would be an intriguing area for future research.Moreover, the integration of collaborative Gaussian processes offers a promising avenue. Currently, our model focuses primarily on personalization, using other users only for tuning GPR hyperparameters. We hypothesize that leveraging collaborative learning techniques, such as those described in [57] , could significantly enhance the performance and effectiveness of our method.The application of Gaussian Process Regression (GPR) for user modeling in recommendation systems has both positive and negative societal impacts. Below, we outline these impacts in detail.Positive Societal Impacts. Our approach, GPR4DUR, aims to significantly enhance the personalization and efficiency of recommender systems. By effectively capturing the diverse and dynamic interests of users, GPR4DUR can improve user satisfaction and engagement across various online platforms. This could lead to more tailored content delivery, helping users find relevant and interesting content more quickly and reducing information overload. Furthermore, the uncertainty-aware nature of our method allows for better exploration-exploitation balance, potentially uncovering niche interests and underrepresented content that users might find valuable. This can promote diversity and inclusivity in content consumption, providing a wider range of options to users.Negative Societal Impacts. Despite these positive aspects, there are potential negative societal impacts associated with the deployment of our method. One major concern is the potential for reinforcing existing biases in recommendation systems. If the training data reflects societal biases, GPR4DUR might inadvertently perpetuate these biases, leading to unfair treatment of certain user groups. Additionally, the improved personalization capabilities might increase the risk of creating filter bubbles, where users are only exposed to content that reinforces their existing preferences, limiting their exposure to diverse viewpoints.Another potential issue is privacy. As GPR4DUR leverages detailed user interaction histories to model preferences, there is a risk of sensitive information being inferred or misused. Ensuring robust data protection and adhering to privacy standards is crucial to mitigate this risk.Mitigation Strategies. To address these potential negative impacts, several mitigation strategies can be employed. Firstly, implementing fairness-aware algorithms that explicitly account for and mitigate biases during model training can help ensure equitable recommendations across different user groups. Secondly, incorporating mechanisms to introduce serendipity in recommendations can counteract filter bubble effects, exposing users to a broader range of content. Lastly, adhering to strict data privacy regulations and employing advanced anonymization techniques can safeguard user data and privacy.Overall, while GPR4DUR has the potential to significantly improve user modeling and personalization in recommender systems, careful consideration and implementation of mitigation strategies are essential to address its broader societal impacts.NeurIPS Paper Checklist Justification: The limitations of this study are discussed in Appendix A.9. Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.The implementation can be found at https://github.com/haolun-wu/GPR4DUR/.https://tianchi.aliyun.com/dataset/649?lang=en-us 


Decision: The paper proposes a method called GPR4DUR for multi-interest recommendation and retrieval by using Gaussian process regression and density-based user representation.

Reviewers mostly agree on the following contribution of the paper:
* using Gaussian Processes (GP) and density-based user representation to capture multi-interest of users and providing a measure of uncertainty on the recommendation is novel (all reviewers), 
* performance study is comprehensive (F9WH, Eeve, icLv)
* design new evaluation metrics tailored for multi-interest contexts (F9WH, Eeve)

The reviewers raised questions on scalability, clarity on adaptability and comparing with more relevant baselines etc.. Some of them are addressed during rebuttal. The authors are advised to incorporate these additional analysis and results in the final version.

Reviews: review: paper_summary: paper_summary: The authors propose GPR4DUR, which incorporates uncertainty-awareness and scales well to large numbers of users.  This method leverages Gaussian process regression (GPR) to effectively capture the diverse and dynamic interests of users without the need for manual tuning. Through experiments with real-world offline datasets and online simulations, the authors demonstrate that GPR4DUR can adapt to user interest variability and efficiently balance the exploration-exploitation trade-off by utilizing model uncertainty. The paper also presents new evaluation protocols and metrics for multi-interest retrieval, showing the adaptability and efficiency of GPR4DUR in capturing a user's multiple interests and recommending items that span those interests, including niche ones.
strength_weakness: strength_weakness: - As mentioned in section 4.5 of the document, GPR4DUR is more suited for the ranking stage of large-scale recommendation systems. However, the ranking stage often involves a variety of user features, such as user embeddings, to achieve more accurate ranking results. Yet, this paper only utilizes the user's historical interactions.

- The article still lacks an analysis of the relationship between multiple interests and uncertainty.
questions: questions: - Is there greater uncertainty associated with a user's niche interests? Or, how does the uncertainty differ between items of varying popularity and users with different interaction patterns?

- In the tuning of GPR parameters, would using different parameters for different users yield better results?
limitations: limitations: Yes
paper_summary: paper_summary: The research problem addressed in this study revolves around the challenge of effectively capturing user interests to enhance the quality of personalized recommendations. Traditional methods often struggle to model users with diverse interests accurately, leading to suboptimal recommendations.

The authors emphasize the limitations of existing user modeling techniques, such as collaborative filtering and matrix factorization, which may not adequately capture the multi-interest nature of users. To address this gap, the paper introduces the concept of Density-based User Representations (DURs) and proposes the GPR4DUR approach, which leverages Gaussian Process Regression to create more nuanced user representations based on the density of user interactions.
strength_weakness: strength_weakness: 1. A clearer articulation of the specific advancements or enhancements that Gaussian Process Regression for Density-based User Representation (GPR4DUR) provides over traditional methods—particularly in terms of adaptability.

2. Discussing the scalability of the GPR4DUR approach with larger datasets and its computational efficiency would help to assess the practical applicability of the technique.
questions: questions: Could the authors provide more details on the selection criteria for the real-world offline datasets used in the experiments? What specific characteristics were considered in choosing these datasets, and how do these characteristics make them particularly suitable for evaluating the effectiveness of the GPR4DUR approach?

In the online experiments involving simulated users, what specific metrics were employed to assess the effectiveness of the GPR4DUR recommendation policies? Can the authors detail the methodology used to simulate user interactions and describe how the performance of GPR4DUR was evaluated in this online setting?

Considering the emphasis on scalability, how does the computational complexity of GPR4DUR compare to traditional user modeling methods as the number of users and items increases? Have there been specific experiments conducted to analyze the scalability of GPR4DUR across different dataset sizes? What were the findings regarding the impact of increasing data volume on system performance?

How does GPR4DUR address and mitigate biases during the model training process to ensure equitable recommendations across diverse user groups? Can the authors elaborate on the specific mechanisms or algorithms implemented within GPR4DUR to promote fairness and reduce potential biases in the recommendation outcomes?
limitations: limitations: The authors have adequately addressed the limitations and identified no potential negative societal impacts of their work.
paper_summary: paper_summary: The paper introduces density-based user representations (DURs) using Gaussian process regression (GPR) to address limitations in existing user modeling methods for personalized recommender systems. The proposed GPR4DUR approach effectively captures user interest variability, incorporates uncertainty-awareness, and scales well, as demonstrated by experiments with realworld datasets and simulated users. This method balances exploration and exploitation, enhancing multi-interest recommendations without manual tuning.
strength_weakness: strength_weakness: 1 The difference between SUR and MUR is not clearly explained. Why MUR is better, is there a reasonable explanation or proof? The author should explain this in detail in the introduction section.

2 The adaptability and uncertainty that the author repeatedly emphasizes, what is the specific manifestation in the recommendation scenario. Whether their modeling of user representation is meaningful, they should provide specific definitions, clear assumptions and more specific explanations.

3 The core motivation of the paper is not clear. For example, in line 33, the author did not explain why manually selecting clusters will limit the adaptability of MUR. How to understand the variability of interests across users? Why this type of method cannot simulate uncertainty well?

4 The paper intro is vague and abstract, and it is hard for readers to establish a connection between the motivation and the proposed method.
questions: questions: 1 For some simple collaborative signal modeling, can your method also achieve good performance improvement on it?

2 Is there a more detailed definition or explanation of density-based? 

3 Figure 1 is not well explained. What do the x-axis and y-axis represent? How to understand the value range? Is normalization performed? The SUR and MUR graphs look the same. What do the color of the points mean? 

4 The explanation on line 35 is too abstract. How to understand since interests generally have high variability across users. Is there a strict mathematical definition to explain this problem?

5 Lines 40-42, this three words are too abstract. Is there a more detailed explanation or explanation in mathematical formulas? Why can Gaussian process solve these three points?

6 Line 36, why is uncertainty regarding a user’s interests not well-modeled by these methods, diminishing their ability to perform effective online exploration.
limitations: limitations: The core motivation is not clearly explained, and it is difficult for the author to understand how the proposed method solves the problems of existing methods. This paper is not recommended for acceptance.
paper_summary: paper_summary: This work presents a novel method for user representation in retrieval and recommendation systems. The authors propose a density-based approach that leverages the density of user interactions to enhance the representation of users. This method is aimed at improving the performance of retrieval and recommendation systems by better capturing the underlying user preferences and behaviors. The paper includes detailed experiments and evaluations demonstrating the effectiveness of the proposed approach compared to existing methods.
strength_weakness: strength_weakness: W1. The proposed method uses item categories as auxiliary information, while most of the baselines only consider user-item interaction information. This may lead to unfair comparisons. 

W2. Some important baselines are missing. Specifically, STOSA[1], DT4Rec[2], and BERD+[3] use Gaussian distribution to model the uncertainty of user preferences in sequential recommendation, which are highly relevant to this work.

W3. The Interest-wise Relevance (IR) metric might not be reliable, as it relies on the learned item embeddings which may be inaccurate.

W4. The manuscript contains some typos. For example, on line 124, 'a RS' should be 'an RS'. On line 155, there should be an 'is' before the covariance function.



References:

[1] 	Ziwei Fan, Zhiwei Liu, Yu Wang, Alice Wang, Zahra Nazari, Lei Zheng, Hao Peng, Philip S. Yu. Sequential Recommendation via Stochastic Self-Attention. WWW 2022: 2036-2047.

[2] Ziwei Fan, Zhiwei Liu, Shen Wang, Lei Zheng, Philip S. Yu. Modeling Sequences as Distributions with Uncertainty for Sequential Recommendation. CIKM 2021: 3019-3023.

[3] Yatong Sun, Xiaochun Yang, Zhu Sun, Bin Wang. BERD+: A Generic Sequential Recommendation Framework by Eliminating Unreliable Data with Item- and Attribute-level Signals. ACM Trans. Inf. Syst. 42(2): 41:1-41:33 (2024).
questions: questions: Please refer to the Weaknesses section.
limitations: limitations: The authors have acknowledged several limitations in their work, especially the potential challenges in scalability.

Comments: comment: Thank you for reading our rebuttal and for your thoughtful consideration. We are pleased that our responses addressed your concerns. We are happy to clarify any points in the remaining discussion window should you have additional questions.
Thank you for reading our rebuttal and for your thoughtful consideration. We are pleased that our responses addressed your concerns. We are happy to clarify any points in the remaining discussion window should you have additional questions.
Thanks for the authors' detailed responses. After reading their responses, I decided to raise my score since the authors have addressed some of my concerns.
Thanks for the authors' explanations. I have no further questions.