Unifying Generative and Dense Retrieval for Sequential Recommendation
Authors: Liu Yang
Fabian Paischer
Kaveh Hassani
Jiacheng Li
Shuai Shao
Zhang Gabriel Li
Yun He
Xue Feng
Nima Noorshams
Sem Park
Bo Long
Robert D Nowak
Xiaoli Gao
Hamid Eghbalzadeh
Venue: Accepted by TMLR ()

Abstract: Sequential dense retrieval models utilize advanced sequence learning techniques to compute item and user representations, which are then used to rank relevant items for a user through inner product computation between the user and all item representations. While effective, these approaches incur high memory and computational costs due to the need to store and compare a unique embedding for each item--leading to lower resource efficiency. In contrast, the recently proposed generative retrieval paradigm offers a promising alternative by directly predicting item indices using a generative model trained on semantic IDs that encapsulate items’ semantic information. Despite its potential for large-scale applications, a comprehensive comparison between generative retrieval and sequential dense retrieval under fair conditions is still lacking, leaving open questions regarding performance and resource efficiency trade-offs. To address this, we compare these two approaches under controlled conditions on academic benchmarks and observe performance gaps, with dense retrieval showing stronger ranking performance, while generative retrieval provides greater resource efficiency. Motivated by these observations, we propose LIGER (LeveragIng dense retrieval for GEnerative Retrieval), a hybrid model that combines the strengths of these two widely used approaches. LIGER integrates sequential dense retrieval into generative retrieval, mitigating performance differences between the two methods, and enhancing cold-start item recommendation in the evaluated datasets. This hybrid approach provides insight into the trade-offs between these approaches and demonstrates improvements in efficiency and effectiveness for recommendation systems in small-scale benchmarks.

Published in Transactions on Machine Learning Research (06/2025)
Unifying Generative and Dense Retrieval for Sequential Rec- ommendation
Liu Yang
University of Wisconsin-Madden
Al at Meta'
Fabian Paischer
ELLIS Unit, LIT AI Lab, Institute for Machine Learning, JKU Linz, Austria AI at Meta'
Kaveh Hassani, Jiacheng Li, Shuai Shao, Zhang Gabriel Li, Yun He, Xue Feng, Nima Noor- shams, Sem Park, Bo Long
AI at Meta
Robert D Nowak University of Madison
Xiaoli Gao AT at Meta
Hamid Eghbalzadeh AI at Meto'
heghbole@meta.coom
Reviewed on OpenReview: https: // openreview. net/ forum? id= jadnFIs
Abstract
Sequential dense retrieval models utilize advanced sequence learning techniques to compute item and user represontations, which are then usod rank relevant items for a user through inner product compulation between the user and all item representations. While effective, these approaches incur high memory and computational costs due to the need to store and compare unique embedding for each item leading to lower resource efficiency. In contrast, the recently proposed generative retrieval paradigm offers promising alternative by directly predicting item indices using a generative model trained on semantic IDs that encapsulate items' semantic information. Despite its potential for large-scale applications, a comprchensive comparison between generative retrieval and sequential dense retrieval under fair conditions is still lacking, leaving open questions regarding performance and resource efficiency tradc-offs. To address this, we compare these two approaches under controlled conditions academic benchmarks and observe performance gaps, with dense retrieval showing stronger ranking performance, while generative retrieval provides greater resource efficiency. Motivated by these observations, we propose LIGER (Leveraging dense retrieval for GEncrative Rotrieval), a hybrid model that combines the strengths of those two widely used approaches. LIGER integrates sequential dense retrieval into generative retrieval, mitigating performance differences between the methods, and enhancing cold-start item recommendation in the evaluated datascts. This hybrid approach provides insight into the trade-offs between these approaches and demonstrates improvements in efficiency and effectiveness for recommendation systems in small-scale benchmarks
Code available https://github. httpstttt.coo/ssccoooeesseaaaaae 21: Work done during Internship Mela., 1: Corresponding author.

---

Published in Transactions on Machine Learning Research (06/2025)
1 Introduction
Sequential rocommendation methods (Kang & McAuley 2018b Zhou et al. 2020). have predominantly relied on advanced sequential modeling techniques (Hochreiter Schmidhuber 1997 Vaswani et al. 2017] Radford et al.] 2019) to learn dense embeddings for each item and user. Using these embeddings, the most relevant items are retrieved through maximum inner product search. Despite their effectiveness, these approaches require computing inner products with every item in the datasct during retrioval stage, which can be computationally expensive as the number of items grows. Furthermore, each item must be represented by a unique embedding, which needs to be learned and stored, adding to the complexity.
In contrast, generative retrieval [Rajput et al. 2024) is an emerging approach, which deviates from the embeedding-centrie paradigm. Instead of pairwise comparisons between user and item embeddings, this approach utilizes a generative model to directly predict the item index. To better capture the sequential patterns within item interactions, items are indexed by "semantic IDs (SID)" (Lee et al. 2022a), which encapsulate their semantic characteristics. During the recommendation process, the model employs beam search decoding to predict the semantic ID of the next item based on the user's previous interactions. This method not only reduces the need for storing a number of embeddings that scales with the number of items, but also enhances the ability to capture deeper semantic relationships within the data. Additionally, adjusting the temperature during generation can naturally produce more diverse recommendations.
To distinguish the embodding- based retricval approach from generative retrieval, which also leverages se- quential information, we refer to it in this paper as (sequential) dense relrieval. The term dense relrieval is borrowed from the information retrieval domain (Tran & Yates 2022). , with "sequential" added to differ- entiate it from other dual-cncoder-basel architectures (Yi et al.
2019). The generative retrieval paradigm
is well-positioned for future scaling in industrial recommendation systems (Singh el. al.] 2023 Zheng et al., 2025), offering significant savings in storage and inference time. However, while prior works continue to 2023b), generative retrieval methods are
advance the dense retrieval paradigm (Hou et al. 2022c Li et al.
increasingly being integrated with pretrained models such as LLMs to improve item recommendation.
(Cao ct al., 2024b Paischer et al. 2024)
Despite these advancements, there is notable lack of direct comparisons under equivalent conditions, raising questions about which paradigm performs better given the same input information and how their performance balances against trade-offs in resource efficiency- such as computational and storage cost. Given the focus on academic benchmarks and limited resources, wc narrow our study to small-scale datasets commonly used in research (Rajput et al.j 20244; Cao el. al., 2024a] Liu et al., 2024b; Li et al.] 2023b] Hou el al. 2022c Zheng et al.] 2024).. Within this context, we aim to explore these questions by comparing two established reprosentative implementations of sequential generative and dense retricval models, ensuring consistency in input information and experimental setups.
As shown in Figure [ in our experiments, the sequential dense retrieval excels in ranking performance and cold-start item retrieval, albeit with a comparatively higher computational and storage footprint. In con- trast, generative retrieval demonstrates lower practical ad advantages terms of model design and scaling flexibility. These observations are not entirely sur- prising, as dense retrieval methods have been well-established and extensively developed over time, excelling at learning high-quality embeddings that are particularly effective for recommendation tasks. Mcanwhile, generative retrieval, as a relatively new paradigm, is still in its early stages of refinement. Despite its current limitations, generative retrieval holds significant potential (Singh et al. 2023 Zheng et al. 2025), and future improvements in architecture, training, or decoding strategies could further enhance its effectiveness.
particccllly on cold-start items- but may offer
At this stage, to harness the strengths of both paradigms, we propose a hybrid model, LIGER, that combines the resource efficiencies of generative retrieval with the robust embedding quality and ranking capabilities of dense retrieval. Our SID-based hybrid model applies dense retrieval to limited set of candidates generated by generative retrieval module, retaining the resource efficiency of generative retrieval while significantly improving performance, particularly for cold-start items. Specifically, our key contributions are follows:
We identify and analyze two primary observed limitations of the generative retrieval method on the small-scale academic benchmarks: (1) Generative retrieval exhibits performance difference
2

---

Published in Transactions on Machine Learning Research (06/2025)
Dense Retrieval: ((N)
Pricc
Inner Presl
Beanty
Sports
Toys
Steam
Cabegory
Brand
Sequential Model
Predicted
Item Repp
Itmm ID
Recall@101
In-set
Cold
In--et
Cold
In-set
Cold
In-set
Cold
Content
Generative Retrieval: (i()
Dense TIGER
0.0601
0.08K3
0.0828
0.0
0.0382
0.0553
0.0020
0.0575
0.0578
0.0852
0.0496
0.1898
0.201 1 0.0261
Information
Sequentiel
Model
Bearn Search
Perf Gap
0.0330
0.0171
0.0374
0.0113
Figure 1: Performance Comparison Between the Implemented Generative and Dense Retrieval Methods Across Dalasets. Dense retrieval computes the inner product between predicted item representations and the entire item set, scaling with ((N) and requiring storage for ((N) embeddings. In contrast, generative retrieval stores only Ol)) learnable embeddings and predicts the next item using beam search, scaling with O(LK), where K is the beam size and t is the number of Semantic IDs. Using identical item content information, hoth methods were trained on varions datasets, and their performance, measured by Recall@10, is reported in the table the right. While the implemented generative retrieval method reduces computational and storage costs, it shows lower performance compared to the implemented dense retrieval method the datasets we evaluated.
compared to dense retrieval, given the same item information imput, and (2) it tends to prioritize to items encountered during training, resulting in a lower probability of generating cold-start items due to under-representation.
We propose LIGER (Leveraging dense retrieval for GEnerative Retrieval), a novel method that synergistically combines the strengths of sequential dense and generative retrieval to significantly enhance the performance of generative retrieval. By integrating these methodologies, LIGER re- duces the observed performance differences between dense and generative retrieval while improving the generation of cold-start items on the dataset we explored.
The remainder of the paper is organized follows. In Section 2 we analyze the performance differences between the implemented generative and dense retrieval methods. Subsequently, we introduce the hybrid method LIGER in Section g The experimental results are presented in Section Duc to space constraints,
we move the related work to Appendix A
2 Analysis of Generative and Dense Retrieval Methods
In this section, we first introduce the generative retrieval (Rajput et al. 2024 and dense retrieval (Hou et al., 2022b) formula (sce Section 2.1 and 2.2 Then in Section 2.3. we examine the performance difference be- tween generative retrieval and sequential dense retrieval methods, and then discuss the challenges generative retrieval faces in handling item cold-start scenarios in Section 2.4
2.1 Generative Retrieval Review
typically follow two-stage training process. The first stage involves collecting textual descriptions for each item based on their attributes. These descriptions serve as inputs to a content model (c.g., a language encoder) that produces the text embeddings of the items, subsequently quantized by an RQ-VAF (Lee al. 2022a) to attribute a semantic ID for each item. Formally, for cach item i I, we collect its p key-value attribute pairs {(k), 01), (k2, v2)...., (kp, vp)} and format them into a textual description: T: = prompt(ki, ... kp. Up). The textual description T, is then passed to the content model, yielding the text representation e,axt for each item. We refer readers to Rajput et al. (2024); Lee et al. (2022a) for the training details of the RQ-VAE module. After the RQ-VAE is trained, we obtain the m-tuple semantic ID (s) ,"") for each item i. Notably, an m-tuple semantic ID, with a codebook size I, can theoretically represent 1T unique items.
Generative retrieval approaches such as TIGER (Rajput et al. 2024
In the sccond stage of training, the item text representation (o and the trained RQ-VAE model are discarded, retaining only the semantic IDs. For each interaction history indexed by item IDs (2,, in), the item IDs replaced with their corresponding semantic TDs:
3

---

Published in Transactions on Machine Learning Research (06/2025)
Dense Retrieval
Semanti: ID Generation
Next Item ID
SID=(5,
Item 233
Item Rep
Title, Brand, Category, Price
Text Rep
Bidirectional Transtormer Encoder
LIGER (Ours)
Item pos
2
Next Item 64 SiD=(5, 25, 55)
Item rep
Item 233 Text Rep
Item 515 Text Rep
Next Item 64 Text Rep
Transtormer
Item
ID_ 233
ID_515
Decoder
Bidirectional Transformer Encoder
Generutive Retrieval
Transformer
Next Item SID=(5, 25, 55)
Item pos
Decoder
Item rep
Item 233 Text Rep
Item 515 Text Rep
Bidirectional Transformer Encoder
SID pos
P_1
P_2
p_3
P_1
p_2
P_3
SID
1
23
5555
L
25
1788
SID
1.5
2,23
t_55
1_5
2_
78
Item 233
Item 515 SID=(5, 25, 78)
233
SID=(5, 23, 55)
SID=(5, 23, 55)
SID=(5, 25, 78)
Figure 2: Overview of Sequential Dense Retrievol, Generative Retrieval, and Our Hybrid Retrieval Method, LIGER. Dense Retrieval (upper left) uses an encoder model to map item and text representations dense embeddings, which are used to predict the next item in the sequence based similarity. Generative Retrieval (lower left) employs an encoder-decoder Transformer generate
next item's semantic ID from the given semantic trajectory. These semantic IDs are derived from item features such title, brand, price, and category (upper right). Our proposed Hybrid Retrieval, LIGER (lower right) combines both semantic ID input and item text representations, integrating dense and generative retrieval techniques. By taking item positions, text representations, and semantic IDs as input, and outputs both the predicted item embedding and the next item's representation.
sf.... )) (sb, si......... < .... (sh, = )) Given the semantic IDs of the last n items in- teracted with, the Transformer then optimized to autoregressively prodict the next semantic ID sequence ""
During inference, a set of candidate items is retrieved using beam search over the trained Transformer, selecting items based on their semantic IDs. A visual reprosentation of the gencrative retrieval method is provided Figure (Lower Left).
It is worth noting that although the item text representations are excluded from the second stage of training, they still contribute to the information utilized in LIGER since semantic IDs derived based on the se- mantic similarities of item text representations. To ensure a fair comparison, we incorporate this information into the development of a comparable dense retrieval method in the following section, thereby accounting for the impact of these embeddings on the overall performance.
2.2 Sequential Dense Retrieval in Transductive Setting
Sequential dense retricval mothods typically consist of two main components: (1) learning item represen- tations through sequence modeling, and (2) performing retrieval using dot-product search. To enhance the learning of item representations, several dense retrieval methods such as Hou al. (202bb) employ transductive setting, where item content information is integrated through text representation to enable transferable representation learning.
Building on these insights, we implement the dense retrieval method as follows: For each item we first obtain text representation oxext using the procodure described in the first stage of Section 2.1 Additionally, we retrieve the learnable item embedding e, Embd(i) from the embedding table Embd(.) and obtain the positional embedding pos from learnable positional embedding table. The input embedding for each item
4

---

Published in Transactions on Machine Learning Research (06/2025)
is then computed as:
E, e; + text + poe
and the sequence of imput embeddings (E Eg.... E is provided to the Transformer. Given these n. embeddings, the Transformer is trained to optimize the following objective:
exp (sim((((())) en+1 + #ff)//)
Ldense (9, (ei): )) = log Eiez exp (sim (E(O), e
where E(O) represents the output embedding of the Transformer with parameter + sim(..) denotes the cosine similarity metric, T is a temperature scaling factor, and I is the set of all items. Figure 2 (Upper Left) provides a detailed illustration of the dense retrieval model implementation.
2.3 The Observed Performance Difference
As detailed in Section 2.1 the generative retrieval model leverages both item text representations and sequential interaction information. In Section 2.2 we introduce sequential dense retrieval methods inspired by Hou et al. (2022c], designed to fully incorporate these sources of information in the training process.
To ensure fair comparison between the generative retrieval and dense retrieval methods, we maintain consistency in model architecture, data pre-processing, and information utilization. The specific details of our experiment setup are described in Section 4.1 and in Appendix B. Due to the unavailability of TIGER's codebase, we made significant. efforts to faithfully implement the TIGER method following the setup described in Rajput et al. (2024), conducting extensive hyperparameter tuning to optimize its performance, and configured the dense model's hyperparameters to align with the tuned TIGER settings. However these results, shown Figure [ (Right), indicate a performance difference between the generative and dense retrieval methods in the datasets we evaluated.
There notable differences between the two implemented methods: (a) Different Item Indexing and Number of Embeddings: As discussed in Section 2.1 representing N item requires the dense retrieval method to learn and store O(N) embeddings. In contrast, the semantic-ID-bassee generative retrieval method ideally only requires (tt) tokens, where tm se N, with m being the length of the semantic ID tuple. However, it remains unclear whether the learned semantic ID can sufficiently capture the item's 's semantic meaning and effectively replace the item ID; (b) Text Representation Input: Dense retrieval utilize the item's text representation additional input; and (c) Prediction Mechanism: Dense retrieval relies on maximum inner product search in the embedding space, whereas generalive retrieval predicts the next item through next-token prediction via beam search.
To analyze the effect of (a), we modify the dense retricval approach by replacing item IDs with semantic IDs while keeping all other components unchanged. This modified method is referred to as Dense (SID). Formally, for each item i with semantic ID ss),s. ......... s,"), we construct the input embedding for each semantic ID as:
ES- = = + text + c + c
(1)
where the e. is the learnable embedding for each semantic ID: e.j Emba(s!), and ej is the posi- tional embedding for each semantic ID. The final embedding for item is then represented as: E, = E.g, E.,. ]. During training, the cosine similarity between the predicted embedding and item's text representation compared and maximized.
To examine the effect of (b), we augment TIGER with the item's text representation, referred to as TIGER (T). Specifically, use the same input as in Equation I and, during training, apply the next-token prediction loss on the semantic ID tuple of the next item.
The results in Figure 3 show that incorporating semantic IDs as input in dense retrieval (Dense (SID)) results in similar performance to the sequential dense retrieval (Dense), despite the difference in item indexing. This suggests that semantic indexing effectively preserves the information contained item IDs, as does not lead to loss in retrieval performance. Moreover, supplementing TIGER with text representation as input (TIGER (T)) yields marginal improvements over TIGER alone; however, it still falls short of matching
5

---

Published in Transactions on Machine Learning Research (06/2025)
the performance of dense retrieval. This suggests that the performance gap primarily stems from the third factor: a discrepancy in the objective function, as the next-token prediction loss is less effective for retrieval than direct similarity-baser ranking.
set
0.20
TIGER
(SID)
Figure Comparison of Recall@10 on in-set and cold-stert dataset across various datasets. Dense retrieval methods (both ID- and SID-based) ex- hibit robust performance in both in-set and cold- start settings. While TIGER (T) shows marginal improvements over TIGER, it still lags behind the performance dense retrieval methods.
TIGER
Dense
0.10
0.00
Cold start
0.08
0.04
Beauty
Sports
Toys
steam
2.4
Challenges in Cold-Start Item Prediction with Generative Retrieval Models
Our investigation extends to the cold-start item generation problem, critical issue in the dynamic envi- ronment of real-world recommendation systems. As items are continuously introduced, they often lack sufficient user interactions, which impodes their prodictability until significant amount of interaction data is gathered.
For dense retrieval in the transductive setting (Hou et al., 2022c). the inclusion of item's text representations provide some prior information, thus partially retaining the ability to retrieve cold-start items, resulting in non-zero performance of cold-start item prediction shown Figure [ and Figure 3 In contrast, generative retrieval approaches such TIGER and TIGER (T) fail to retrieve unseen items, evidenced in the same figures. This highlights fundamental limitation of generative retrieval methods, where the decoding process itself may hinder effective cold-start item generation.
To investigate this further, we analyze the generation probabilities of cold-start items in the Amazon Beauty dataset (details in Section 4.1 using a trained generative retrieval model and sunmarized the results in Figure 4 (b and c). Specifically, we ask the model to generate K candidates using beam search, which ranks items by their generation probabilities. Among these candidates, we define the minimum generation probability as Рк- Separately, we calculate the generation probability of the ground-truth cold-start item, denoted p². In Figure (b), we present a histogram comparing p' for all cold-start items with their corresponding pK (K = 10). In Figure (c), vary K E: {10, 20, 40, 80} to analyze its impact. The results show that cold-start items consistently have lower generation probabilities than the beam scarch threshold (p* < PK), even when K = 80, effectively preventing them from being retrieved. Our findings indicate that the model's learned conditional probabilities tends to prioritize to items seen during training, leading to a significantly reduced capability to generate cold-start items.
It is worth noting that Rajput et al. 2024 propose an alternative solution mitigate the issue of cold-start item generation. Their approach involves setting a predefined threshold for cold-start. item within the retrieved candidate set of K items. This ensures that K-s of the retrieved candidates are cold-start items by excluding other candidates that have higher generation probabilities. However, this method relies on prior knowledge of the ratio between recommended cold-start and non-cold-start items, which may not always be available. Therefore, we argue that the challenges in cold-start item generation persist for generative retrieval models, indicating a need for more robust solutions that do not depend heavily on predefined parameters or assumptions.
3 Methodology
The notion that "there is no free lunch" (Wolpert & Macready 1997 holds true in the context of retrieval methods. As discussed in the previous section, a performance difference is observed between the generative retrieval and dense retrieval methods we implemented, with generative retrieval facing challenges in gener- ating cold-start items. The improved performance of dense retrieval, however, comes with higher storage, learning, and inference costs. These costs are manageable in our current setup, where the candidate pool is relatively small; however, they may become more significant at larger scales. On the other hand, genera-

---

Published in Transactions on Machine Learning Research (06/2025)
(b) Bistogram
(c) Pk Pp-
(a)
Ground Truth: Cold-start Item
40
Item 233
Item 515
TIGER
Item 64
Oount
30
Goneration
Fully Ranked Candidates
20
Probability:
P1 0.4
Pt 0.1
Px 0.01
Item 123
Item 55
Item 36
Item 64
10:1 10
16-09
1e-05
0.01
20
80
Item Generation Probebility
Top-K Generation
Figure 4: TIGER Fails to Generate Cold-Start Items. (a) The TIGER model generates ranked list of candidates, with pk denoting the generation probability of generating the K-th ranked item over all items. The ground-truth cold-start item has a generation probability of p" (b) A histogram compares pk (for K = 10) with p when the ground-Iruth item cold-start, highlighting the disparity between them. (c) The difference pair = PK pp" is plotted for K = 10, 20, 40, 80. A successful generalion of cold-start item occurs only when & 0, illustrating the model's limitations in handling cold-start items.
tive retrieval holds promise for more scalable and flexible retrieval particularly in open-ended or large-scale scenarios (Rajput et al. 2024), but its performance lags behind in the datascts we evaluated.
Theoretical trade-offs bet ween approaches are summarized in Table | where N represents the total number of itemss t denotes the total number of semantic DDs and K is the number of candidates to be retrieved during inference. Here, we denote the inference cost as the number of comparisons required for each method. We emphasize that this comparison theoretical and abstracts away implenentation-speciii factors. In Section 4.4 we report empirical inference times for dense retrieval, generative retrieval, and our hybrid method; however, these measurements are not meant to account for system-level optimizations, hardware acceleration, or software engineering factors, which remain beyond the scope of this work.
Comparison of Dense Retrieval, Generative Retrieual, and Our Hybrid Methods LIGER Across Different Costs. Here represents the total number of items, t denotes the total number of semantic IDs used by generative retrieval method, and K indicates the number of candidates retrieved during inference.
Table 1:
Dense Retrieval
Generative Retrieval
LIGER (Ours)
Learnable Embedding (Fized) Item Text Representation Inference Cost Cold-Start Item Generation
((N)
(())
O(t)
O (N)
O(N)
(NN)
O(tK) No
O(t))
Yes
Yes
In this section, we propose hybrid method, called LIGER, that combines the stnnttss of both approaches.
Our goal is to improve upon the existing generative retrieval method: enabling it to generate cold-start items and bridging the gap with dense retrieval. To achieve this, we integrate text represcntations into the sequential model training phase of the generative retrieval method. The associated costs of LIGER are detailed in the last column of Table 0
Formally, for each item i with semantic ID (f ... , ), we construct the input embedding for each semantic ID defined in Equation [ During training, the model is jointly trained on two objectives: the cosine similarity loss and the eext-token prediction loss the semantic IDs of the next item. The combined loss is formulated
exp (sim(B(()) (/77)))))
C(O, (e.). (o.") (e (6) = log Eiez exp (sim(E(()) of(r)/7))
Σ log P(s) 1
The first term in the loss function ensures that the model learns to align the encoder's output embedding with the text representation of the next item using softmax over cosine similarity. The second term corresponds the next-token prediction loss, where each token of the next item's semantic ID tuple is predicted sequentially in the decoder, conditioned on the historical input embeddings [E,.... ,E,]. Figure (Lower Right) provides detailed illustration of LIGER.

---

Published in Transactions on Machine Learning Research (06/2025)
We adopt encoder-decoder Transformer for our method. During inference, the decoder retrieves K can- didate items using beam search. Given that cold-start items are relatively sparse compared to in-set items, we supplement the retrieved candidates with them to ensure their inclusion. The final set is then ranked using the encoder's output embeddings. This approach accounts for the periodic introduction new items in recommendation systems, where cold-start items may otherwise be underrepresented. Details are shown in Figure (left).
4 Experimental Setup and Results
In this section, we present the experimental results across various datasets and baseline methods, showcasing the performance on both in-set and cold-start items. Specifically, assess the cold-start performance by testing on items that are unseen during training, which is determined by the dataset statistics.
4.1 Experimental Setup
Datasets. We evaluate LIGER on four datasets, preprocessing them using the standard 5-core filtering method (Zhang et al. 2019 Zhou et al., 2020 This process removes items with fewer than 5 users and users with fewer than 5 interactions. Additionally, we truncate sequences to a maximum length of 20, retaining the most recent items, following the setup in Rajput et al. [024].. Detailed statistics of the resulting datasets are provided in Appendix B.3
Amazon Beauly, Sports, and Toys (He & McAuley 2016): We use the Amazon Review dataset (2014), focusing three categories: Beauty, Sports and Outdoors, and Toys and Games. For each item, we construct emboddings by incorporating four key attributes: title, price, category, and description.
Steam (Kang& & McAuley 2018b): The dataset comprises online reviews of video games, from which we extract relevant attributes to construct item embeddings. Specifically, we utilize the following attributes: tille, genre, specs, lags, price, and publisher. To reduce the dataset size and make it more manageable, we apply subsampling by selecting every 7th sequence, thereby retaining a representative subset of the data.
When generating the item text representations, the item attributes are processod using the sentence- T5 model Ni et al. 2022 (XXL).
Semantic ID Generation. Utilizing the text representations generated from the sentenco-T5 model, we employ 3-layer MLP for both the encoder and decoder in the RQ-VAB (Lee et al. 2022a). The RQ-VAE features three levels of learnable codebooks, each with a dimension of 128 and a cardinality of 256. We use the AdamW optimizer to train the RQ-VAE, setting the lcarning rate at 0.001 and the weight decay at 0.1. To prevent collisions (i.e., the same semantic ID representing different items), following Rajput et al. (2024) we append an extra token at the end of the ordered semantic codes to ensure uniqueness.
Sequential Modeling Architecture and Training Algorithm. For the generative model, we utilize the T5 (Raffel et al.) 2020) encoder-decoder model, configuring both the encoder and decoder with 6 layers, an embedding dimension of 128, 6 heads, and feed-forward work hidden dimension of 1024. The dropout rate is 0.2. The dense retrieval model designed in Section 2.2 employs only the T5-encoder with 6 layers, while maintaining the same arper-paraaeters. We use the AdamW optimizer with a learning rate of 0.0003, weight decay parameter of 0.035, and a cosine learning rate scheduler. Additional details are presented Appendix B.1
Evaluation Metrics. We assess the model's performance using Normalized Discounted Cumulative Gain (NDCG)@11 and Recall@10. 10. For dataset splitting, we adopt the leave-one-out strategy following (Kang & McAuley 2018b; Zhou et al. 2020 Rajput et al. 2024), designating the last item as the test label, the preceding item for validation, and the remainder for training. During training, early slopping is applied based on the in-set NDCG@10 validation metric. For LIGER, which comprises two components: (A) the semantic ID prodiction head and (B) the output embedding head, we implement early stopping based on the performance of component (B). To ensure fair evaluation of cold-start items, we exclude them from the RQ-VAR training to avoid data contamination.
8

---

Published in Transactions on Machine Learning Research (06/2025)
Baselines. We compare our method against five state-of-the-art Item-ID-based dense retrieval methods:
(a) SASRec (Kang & McAuley 201 8018;); fealure-informed methods, which incorporate additional item or user attributes beyond interaction history: (b) FDSA (Zhang et al. 2019), (c) S S (Zhou et al. 2020 and modality-basee methods, which loverage text as additional modal content to enrich item representations: (d) UniSRec (Hou al.) 2022b), (e) Recformer (L.i el al. 2023a). Descriptions and implementation details for these baselines are provided Appendix B.4 We also compare LIGER against TIGER (Rajput et al. 2024), semantic ID-based generative retrieval method. Although subsequent works have built upon this paradigm using large language models (LLMs) (Zheng et al. 2023
Cao et al., 2024b), they rely on pre- pre-trained
LLMs, which are outside the scope of our comparisons.
4.2 Effect of Candidate Retrieval on LIGER
To demonstrate the efficacy of LIGER, we evaluate how its performance varies with the number of candidates K retrieved by the generative retrieval. For clearer insights, we present the normalized performance gap (NPG) in Recall@10 across various datasets. Specifically, let r(K) denote the Recall@10 performance of LIGER with K candidates retrieved by generative retrieval, while TTiGER and Tdense represent the Recall@10 performance of TIGER and dense retrieval, respectively. The NPG then defined as:
(PG(K)= = (r(K) (r(()) (remm Prim))
Norma ized Performance Gap (Recal1810)
A Algorithm 1: Inference Process
100%
Beauty Sports
Toys
Input Interaction sequence (E1, Ez,..., E.), Cold-start items C, Beam size K
Output: Ranked list of itemss I
1. Beam search retrieve top-K candidates:
50%
Theam TF([E1, E2...., E,]; K);
2. Combine with cold-start items:
Tcomb Ibeam UC;
3. Rank Candidates with encoder's output É:
2 = topk(sim(E.) (aaissimm,,eee e,oa)), Vi E Icomb);
10
100 # Candidates Retrieved by Gen. Ret., K
50
N
Figure 5: Inference Process and LIGER's Performance in Bridging the Gap as the Number of Retrieved Candidales Increnses. The left panel illustrates the inference process LIGER, detailing how candidate items are retrieved and ranked. In the algorithm, TF(,, K') denotes the Transformer generating K candidates using beam search based the input sequence. The right panel shows the normalized performance gap between generative and dense retrieval models across several datasets (Beauty, Sports, Toys, Steam) for the in-set Recall@ metric. In this normalization, 0% represents the performance the generative retrieval model, while 100% corresponds to the performance of the dense retrieval model. The figure highlights how LIGER progressively bridges the performance gap the number of candidates retrieved by the generative model increases.
We plot the NPG values for each dataset in Figure (right) with varying K. The results show a consistent interpolation between TIGER and the dense retrieval approach: as the number of candidates K retrieved by the generative model increases, the likelihood of including the correct items in the candidate set grows. Consequently, LIGER progressively improves its performance on the Recall@10 metric, narrowing the gap with the dense retrieval method. In the next section, we will demonstrate the effectiveness of our method across various datasets and baseline methods.
4.3 Experimental Results
We present the results from the various datasets in Table 2 where the mean and standard deviation are calculated across three random seed runs. Traditional item- item-ID-based methods, such as SASRec exhibit poor in-sct performance compared semantic-ID-basee models. However, when attribute information included, models like FDSA and s -R-Rec show improved in-set performance. Nevertheless, their performance on cold- start items remains subpar due to the static nature of item embeddings. Tn contrast, models that utilize

---

Published in Transactions on Machine Learning Research (06/2025)
Table 2: Performance Comporison Across Baseline Methods on Amazon Beauty, Sports, Toys, and Steam Datasets. The best performance is highlighted in bold, and the second-best performance is underlined. Our method consistently achieves either the best or second-best performance across all datasets, closely followed by modality-based baselines (UniSRec RecFormer). We report LIGER's results where generative retrieval is used to relrieve 20 items, followed by the sequential dense retrieval.
NDCG@101 (%)
Recall@101 (%)
Methods
Inference Cost
In-set
Cold
In-set
Cold
SASRee
O(N)
2.179 1 0.023
0.0 0.0
5.109 1. 0.042
0.0 0.0
FDSA
O(N)
2.244 + 0.135
0.0 + 0.0
4.530 + 0.357
0.0 + 0.0
Beanty
S*-Roc
O(N)
2.279 + 0.058
0.0 0.0
5.226 0.229
0.0 0.0
UniSRee
O(N)
3.346 0.057
1.422 0.128
6.937 0.110
3.704 0.000
RecFormer
O(N)
2.880 0.085
1,95 0.433
6.265 0.196
4.733 0.943
TIGER
O(tK)
3.216 + 0.084
0.0 0.0
6.009 0.204
0.0 0.0
LIGER (Ours)
O(tK)
4.020 + 0.044
3.800 0.523
7.447 = 0.111
10.082 1.285
SASRee
O(N)
1.160 + 0.038
0.0 +: 0.0
2,696 + 0.102
0.0 + 0.0
FDSA
O(N)
1.391 + 0.162
0.0 0.0
2.699 + 0.312
0.0 0.0
Sports
S*-Rec
O(N)
L097 + 0.033
0.0 + 0.0
2.537 0.034
0.0 0.0
UniSRee
O(N)
0.244
3.753
1,559
RecFormer
O(N)
1.318 0.053
1.797 0.000
2.921 0.167
TIGER
O(tK)
1,999 + 0.085
0.064 0.056
3.822 0.109
0.195 0.169
LIGER (Ours)
O(tK)
2.430 + 0.075
2.731 0.229
4.400 0.127
5.848 0.292
SASHee
O(N)
2.756 + 0.079
0.0 + 0.0
6,314 + 0.178
0.0 0.0
FDSA
O(N)
2.375 + 0.277
0.0 + 0.0
4.684 + 0.483
0.0 - 0.0
Toys
S*-Rec
O(N)
2.942 + 0.071
0.0 0.0
6.659 0.135
0.0 0.0
UniSRoc
O(N)
3.622 1 0.056
1.090 0.084
7.472
2.477 0.195
RecFormer
O(N)
3.697 1 0.052
4.432 0.094
7.971 0.170
10.023 0.516
TIGER
O(tK)
2.949 + 0.049
0.0 0.0
5.782 0.163
0.0 0.0
LIGER (Ours)
O(IK)
3.756 + 0.151
5.231
0.531
7.135 0.244
13.063 0.516
SASRee
O(N)
14.763 0.061
0.0 + 0.0
18.259 = 0.035
0.0 = 0.0
FDSA
O(N)
8.236 1 0.132
0.0 1. 0.0
14.773 0.234
0.0 0.0:10 0.0
Steamm
S--Ree
O(N)
14.437 0.127
0.0 0.0
18.025 0.222
0.0 0.0
UniSRoc
O(N)
RecFormer
O(N)
14.034 0.123
0.120
0.037
17.042
0.275
0,319 0.110
TIGER
O(IK) 15.034 + 0.064
0.0 0.0
18.980
0.135
0.0 0.0
LIGER (Ours)
O(IK)
14.951 0.158
0.512 0.047
19.049 0.234
1.466 0.110
text representations and pre-training, such as UniSRec and RecFormer, demonstrate enhanced capabilities in handling cold-start item scenarios. The inclusion of text embeddings during pre-training enables these models better handle unseen items. TIGER, which a semantic-ID-based generative retrieval model, outperforms Cem-ID-basee methods in terms of in-set performance but still struggles with cold-start item generation.
Our model, LIGER, builds upon TIGER by using comantic-ID-basee inputs and combining dense retrieval with semantic ID generation as outputs. This approach significantly improves upon the TIGER method and enables effective generation of cold-start items. Across all datasets, our method consistently achieves either the best or second-best performance, closely followed by modality-based baselines such as UniSRec and RecFormer. We adopt a hybrid approach for our reporting, where we use generative retrieval to retrieve 20 items and then rank them with cold-start items using dense retrieval. Comprehensive result including performance of LIGER with different number of retrieved items from generative retrieval is presented in Table Additional ablation study each component of LIGER is present in Appendix D
4.4 Retrieval Efficiency at Scale: Dense vs. Generative
In this section, we evaluate and compare the inference cost of dense retrieval, generative retrieval (TIGER), and our hybrid method (LIGER) as the total number of candidate items increases. This analysis comple- ments our earlier theoretical discussion in Section 3 by examining how these methods behave under both small-scale academic settings and large-scale scenarios where memory and compute constraints become criti-
10

---

Published in Transactions on Machine Learning Research (06/2025)
cal. All experiments are conducted on single NVIDIA RTX 4090 GPU with 24GB of memory, which allows us to test up to approximately 1.2 million items fully loaded into GPU memory.
As shown in Figure 6 under the academic benchmark we examined where, after 5-core filtering, the total number of items is x 12,000-dense retrieval (leftmost triangle) achieves faster inference than TIGER. This is primarily due to the highly optimized nature of inner product computations in dense retrieval, whereas beam search decoding in generative retrieval is less efficient in current implementations.
However, as increase the number of items up to million, we observe clear linear relationship be- tween inference time and the number of candidates N in dense retrieval. By fitting a linear trend to the observed runtime and extrapolating to N ze 16 million, we estimate that the inference time would exceed 1000 seconds. In contrast, the generative re- trieval module maintains a relatively constant run- time, as its decoding cost depends only on the num- ber of semantic IDs generated. Under our current implementation, the semantic ID space supports up to 256 x 16 million items, indicated by the orange dashed line in Figure G
10a
Seconds
(SID)
10a
K=20
K-401
K-00
K=
Km 100
10
5
10 15
20
items, N (x 1M)
Figure 6: Inference cost as a function of the total number of itemss We compare the inference time of dense retrieval with semantic IDs, generalive retrieval (TIGER), and our hybrid model (LIGER) under varying values K, as the total number of items increases. In theory, the semantic ID space used in our experiments can represent up to approx- imately 16 million items (indicated by the orange dashed line). In contrast, the inference cost of dense retrieval scales linearly with the number of items, surpassing that of gen- erative methods once the candidate pool exceeds 1 million.
We also report. the inference cost of our hybrid method, LIGER, across diffcrent values of where K denotes the number of candidates generated by the generative module and subsequently re-ranked by the dense module. As expected, the runtime in- creases approximately linearly with K; however as shown previously in Figure this increase accompanied by notable performance gains. Overall, these results highlight a key trade-off: while dense retrieval is more efficient smaller scales, generative retrieval offers better scalability in memory-limtted settings.
5 Discussion
Addressing Cold-Start Items with Hybrid Retrieval Models. Generative retrieval method's struggle with cold-start items primarily stems from prioritizing familiar semantic IDs during training, as discussed in Section 2.4 To mitigate this issue, LIGER efficiently combines dense retricval with generative retricval. Specifically, LIGER first generates a small set of K candidates (where K < N) using generative retrieval, which is then augmented with the cold-start item set. This approach leverages the efficiency of generative retrieval to reduce the candidate pool while ensuring cold-start items are represented. The dense retrieval component further enhances cold-start performance by leveraging item text embeddings as prior information. As shown in Table 4 this integrat ion ensures that when generative retrieval retrieves fewer than N items, the model maintains robust performance in cold-start scenarios, comparable to dense retrieval only.
Comparative Performance with Current Dense Retrieval Methods. While LIGER demonstrates competitive performance against existing baselines, its primary objective is to strike balance between the gencrative and dense retricval frameworks. As discussed in previous soctions, there are obscrved perfor- mance differences between these two methods on the datasets we tested, even when using the same input information and model architecture. The results presented in this work aim to shed light potential future directions for integrating these approaches, paving the way for the development of more robust and efficient recommendation systems.
Observed Performance Differences are Contextual to Small-Scale Datasets. We want to note that the performance differences obscrved in this study are inffuenced by various factors, including dataset size, implementation details, and the distribution of data collected under specific paradigms. Additionally, we are aware of industry-scale implementations of generative retrieval paradigms (Singh et. al. 2023 Zheng et al.
11

---

Published in Transactions on Machine Learning Research (06/2025)
2025 that outperform dense retrieval approaches in real-world settings. In this work, our goal is not to assert definitive performance gap between the two paradigms. Instead, we focused on aligning the two hods as closely as possible within the scope of academic benchmarks, which typically use small-scale datasets. Our findings are mcant to provide insights specific to this academic sctting and should not be extrapolated to large-scale, real-world applications without further investigation. Understanding whether dense retrieval methods maintain their ad advantages industry scale remains an important open question and beyond the scope of the current study.
6 Conclusion
In this work, we conducted a comprehensive comparison between dense retrieval methods and the emerg- ing generative retrieval approach. Our analysis revealed the limitations of dense retrieval, including high computational and storage requirements, while highlighting the advantages of generative retricval, which uses semantic IDs and generalive models to enhance efficiency and semantic understanding. Furthermore, we have identified the challenges faced by generative retrieval, particularly in handling cold-start items and matching the performance of dense retricval. To address these challenges, model, LIGER, that combines the strengths of both approaches. Our findings demonstrate that hybrid model surpasses existing models in handling cold-start scenarios and achieves advanced overall performance on benchmark datasets.
introduced a novel hybrid
Looking ahead, the fusion dense and generative retrieval methods holds tremendous potential for advancing recommendation systems. Our research provides a foundation for further exploration into hybrid models that capitalize on the strengths of both retrieval types. As these models continue to evolve, they will become increasingly practical for real-world applications, enabling more personalized and responsive user experiences.
12

---

Published in Transactions on Machine Learning Research (06/2025)
References
Michele Bevilacqua, Giuseppe Ottaviano, Patrick Lewis, Wen tau Yih, Sebastian Riedel, and Fabio Petroni. Autoregrossive search engines: Generating substrings as document identifiers. ArXiv, abs/2204.10623 2022. URL http:://app...emartiiccchhh
wrwp:orpps/coo.Cc.ooooo.o 248366293
Yuwei Cao, Nikhil Mehta, Xinyang Yi, Raghunandan Keshavan, Lukasz Heldt, Lichan Hong, Ed Chi, and Maheswaran Sathiamoorthy. Aligning large language models with recommendation knowledge. arXiv preprint ar ar140:24440002/5 2024a.
Yuwei Cao, Nikhil Mchta, Xinyang Yi, Raghunandan H. Keshavan, Lukasz Hcldt, Lichan Hong, Ed H. Chi, and Maheswaran Sathiamoorthh. Aligning large language models with recommendation knowledge. ArXiv, abs/2404.002245, 2024b. URL petps://ap1. htt.aap.ssmaatiiieccm semanticscholar org/CorpusID :28811968777
Jiangui Chen, Ruqing Zhang, J. Guo, M. de Rijke, Wei Chen, Yixing Fan, and Xueqi Cheng. Continual learning for generative retrieval over dynamic corpora. Proceedings of the 32nd ACM International Con- ference on Information and Knowledge Management, 2023. URL https: semantcsscooaar.... org/
Corpus1D:2212770653
Nicola De Cao, Gautier Izacard, Sebastian Riedel, and Fabio Petroni. Autoregressive entity retrieval. arXiv preprint ar Xrv:20000. 00904, 2020.
Gabriel de Souza. Pereira Moreira, Sara Rabhi, Jeong Min Lee, Ronay Ak, and Even Oldridge. Transform- recommendation. In Proceedings of the 15th ACM conference on recommender systems, pp. 143 153, 2021.
ers4rec: Bridging the gap between nlp and
Yijie Ding, Yupeng Hou, Jiacheng Li, and Julian McAuley. Inductive generative recommendation via retrieval-based speculation. ArXio, abs/2410.02939, 2024. URL https://api www.nticccoolaa...o.m
CorpusID:27316300
Chao Feng, Wu Li, Defu Lian, Zheng Liu, and Enhong Chen. Recommender forest for efficient retrieval. In Neural Information Processing Systems, 2022. URL https://ppp..eeenttccccoaaaaa wtt.:oooaaa..rg/r/rrsssssii 258509354]
Jibril Frej, Marta Knezevic, and Tanja Käser. Graph reasoning for explainable cold start recommendation. ArXio, abs/2406.074220, 2024. URL https://api. wemanticscholar http://ccoooooooooooooooooooaaaa
Ruining He and Julian McAuley. Ups and downs:
Modeling the visual evolution of fashion trends with one-class collaborative filtering. Proceedings of the 25th International Conference on World Wide Web, 2016. URL http://app...mmmancccch.ol.oeeee wttp://scccooaa...oooooooooooaia 1964279
Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. Session-based recommenda- tions with recurrent neural networks. arXiv preprinl ar orXic:1511.06993 2015.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural Computation, 9:1735-1780,
1997. URL hhttps://ppt:: www.nticsccooaaa.re....ea.. http::::/ppp...enaaaccccocoooooo http:///..a.aaaaaao.aooooooaraaa
Yupeng Hou, Zhankui He, Julian McAuley, and Wayne Xin Zhao. Learning vector-quantized item represen- tation for transferable sequential recommenders. Proceedings of the ACM Web Conference 2023, 2022a.
URL http://///appseemaaaccccccchalaa http::/eaaattttta..o.oooooooooaa www/CorppsCo..a...oooooo 253098091
Yupeng Hou, Shanlei Mu, Wayne Xin Zhao, Yaliang Li, Bolin Ding, and Ji rong Wen. Towards universal sequence representation learning for recommender systems. Proceedings of the 28th ACM SIGKDD Con- ference on Knowledge Discovery and Data Mining, 2022b. URL https://api. semanticcccctt org/
2029U51D:249625889
Yupeng Hou, Shanlei Mu, Wayne Xin Zhao, Yaliang Li, Bolin Ding, and Ji-Rong Wen. Towards universal sequence representation learning for recommender systems. In Proceedings of the 28th ACM SIGKDD Conference Knowledge Discovery and Data Mining, pp. 585 593, 2022c.
13

---

Published in Transactions on Machine Learning Research (06/2025)
Feiran Huang, Zhen Yang, Junyi Jiang, Yuan-Qi Qi Bei, Yijie Zhang, and Hao Chen. Large language model interaction simulator for cold-start item recommendation. ArXiv, abs/2442.09117, 2024. URL. https:
//api.semanticschol... wwwp:/opoo//oo.or...... 2676677788
Hong Jun Jeon, Songbin Liu, Yuantong Li, Jie Lyu, Hunter Song, Ji Liu, Peng Wu, and Zheqing Zhu. Epinet for content cold start. arXiv preprint arXiv:2412.044 2024.
Mengqun Jin, Zexuan Qiu, Jieming Zhu, Zhenhua Dong, and Xiu Li. Contrastive quantization based semantic code for generative recommendation. ar preprint 1xxt:2444.147774 2024.
Wang-Chenn Baang-Cheg Kang and Julian McAuley. Self-attentive sequential recommendation. In 2018 IEEE interna- tional conference on data mining (ICDM), pp. 197-206. IEEE, 2018a.
"rang-Cheng Kang and Julian J. McAuley. Self-attentive sequential recommendation. In IEFE International Conference on Data Mining, ICDM 2018, Singapore, November 17-20, 2018, pp. 197-206. IEEE Computer Society, 2018b. doi: 10. /TTT///DD....00..00.0
Varsha Kishore, Chao gang Wan, Justin Lovelace, Yoav Artzi, and Kilian Q. Weinberger. Inedsi: Incremen- 2023. URL
tally updatable document retrieval. ArXiv,
arg//orppusD::25991122
Doyup Lee, Chiheon Kim, Sachoon Kim, Minsu Cho, and Wook-Shin Han. Autoregressive image generation using residual quantization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11523 11532, 2022a.
Hyunji Lee, Jaeyoung Kim, Hoyeon Chang, Hanseok Oh, Sohee Yang, Vladimir Karpukhin, Yi Lu, and Minjoon Seo. Contextualized generative retrieval. ArXiv, abs/2210.02068, 2022b. URL https://api. semanticscholar.oom htt.:/ccccooaarr..oooooooopaaa/a 252715634
Hyunji Lee, Jaeyoung Kim, Hoyeon Chang, Hanseok Oh, Sohee Yang, Vladimir Karpukhin, Yi Lu, and
Minjoon Seo. Nonparametric decoding for generative retrieval. In Annual Meeting of the Association for Computational Linguistics, 2022c. URL https: htttp:://pp..seeeeeeee http:/ppp...aaancccccchcooorr www.://oo//CooooCD/ro : 258959550
Jiacheng Lá, Ming Wang, Jin Li, Jinmiao Fu, Xin Shen, Jingbo Shang, and Julian McAuley. Text is all you need: Learning language representations for sequential recommenaation. Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Dula Mining, 2023a. URL https:
7/api //api.semanticscholar orgg/CoppssD:: /ttp///rrSSI::2884118888
Jiacheng Li, Ming Wang, Jin Li, Jinmiao Fu, Xin Shen, Jingbo Shang, and Julian McAuley. Text is all you need: Learning language representations for sequential recommendation. In Proceedings of the 29th ACM SIGKDD Conference Knowledge Discovery and Data Mining, pp. 1258 1267, 2023b.
Yongqi Li, Nan Yang, Liang Wang, Furu Wei, and Wenjie Li. Learning rank in generative retrieval. In Proceedings of the AAAI Conference Artificial Intelligence, volume 38, pp. 8716-8723, 2024.
Han Liu, Yin wei Wei, Xuemeng Song, Weili Guan, Yuan-Fang Li, and Liqiang Nic. Mmgrec: Multimodal generative recommendation with transformer model. ArXiv, abs/2404.16555, 2024a. URL https::/app.
semanticscholar, wwwp:acchhlaaaaoooooooirrooe ott.://ooopusDD2222222999
Yang Liu, Yitong Wang, and Chenyue Feng. Unirec: A dual enhancement of uniformity and frequency in sequential recommendations. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, pp. 1483-1499 2024b.
Sanket Vaibhav Mehta, Jai Gupta, Yi Tay, Mostafa Dehghani, Vinh Q. Tran, Jinfeng Rao, Mare Najork, Emma Strubell, and Donald Metzler. +si+ Updating transformer memory with new documents, ArXiv,
abs/2222009944 2022. URL https://api... http:/ppp...maaiiicccccchhllaa htmpaaatieesslll.....oooooooopao
:2548552999
Jianmo Ni, Gustavo Hernandez Abrego, Noah Constant, Ji Ma, Keith B Hall, Daniel Cor, and Yinfci Yang. Sentenco-t5: Scalable sentence encoders from pre-trained text-to-text models. arXio preprint
XrXiv:2108.08877 2021.
14

---

Published in Transactions on Machine Learning Research (06/2025)
Fabian Paischer, Liu Yang, Linfeng Liu, Shuai Shao, Kaveh Hassani, Jiacheng Li, Ricky Chen, Zhang Gabriel Li, Xialo Gao, Wei Shao, Xue Feng, Nima Noorshams, Sem Park, Bo Long, and Hamid Eghhalzadeh. Preference discerning with llm-enhanced generative retrieval. ArXiu, abs/2412.08604, 2024. URL https:
//apl.semanticscholl. wttp/oorpss1D:2446588288
Haohao Qu, Wenqi Fan, Zihuai Zhao, and Qing Li. Tokenree: Learning to tokenize id for Ilm-based blm-baaadgeenrrativ recommendation. ArXiv, abss2440.104450, 2024. URL https://api www.nticschhollrr httpp:://pp..eeeanttccccoollllll
org/CorpusID: ;
270560438
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Πyya Sutskever. Language models are unsupervised multitask learners. 2019. htt.:ptss/app...eetttcccccooooot ott.p//ccc...aooooooooooooooaaaa
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, (1140))1-66, 2020.
Shashank Rajput, Nikhil Mehta, Anima Singh, Raghunandan Hulikal Keshavan, Trung Vu, Lukasz Heldt, Lichan Hong, Yi Tay, Vinh Tran, Jonah Samost, et al. Recommender systems with generative retrieval. Advances in Neural Information Processing Systems, 36, 2024.
Kaushik Rangadurai, Yiqun Liu, Siddarth Malreddy, Xiaoyi Liu, Piyush Maheshwari, Vishwanath Sangale, and Fedor Borisyuk. Nxtpost: User to post recommendations in facebook groups. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 3792-3800, 2022.
Scott Sanner, Krisztian Balog, Filip Radlinski, Benjamin D. Wedin, and Lucas Dixon. Large language models are competitive near cold-start recommenders for language- and ilem-based preferences. Proceedings of the 17th ACM Conference on Recommender Systems, 2023. URL https: www.nticscholeese........ org/ Corpus ID: 260164942
Anima Singh, Trung Vu, Nikhil Mchta, Raghunandan Keshavan, Maheswaran Sathiamoorthy, Yilin Zheng, Lichan Hong, Lukasz Heldt, Li Wei, Devansh Tandon, et al. Better generalization with semantic ids: A case study in ranking for recommendations arXiv preprint arXiv:2306. 08121, 2023.
Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. Bertárec: Sequential recommendation with bidirectional encoder representations from transformer. In Proceedings of the 28th ACM international conference information and knowledge management, pp. 1441 1450, 2019.
Weiwei Sun, Lingyong Yan, Zheng Chen, Shuaiqiang Wang, Haichao Zhu, Pengjie Ren, Zhumin Chen, Dawci Yin, Maarten Rijke, and Zhaochun Ren. Learning tokenize for generative retrieval. Advonces in Neural Informalion Processing Systems, 36, 2024.
Yi Tay, Vinh Q. Tran, Mostafa Dehghani, Jianmo Ni, Dara Bahri, Harsh Mehta, Zhen Qin, Kai Hui, Zhe Zhao, Jai Gupta, Tal Schuster, William W. Cohen, and Donald Metzler. Transformer memory as a differentiable search index. ArXiu, abs/2202.0699) 2022. URL https://api. semanticscholar. org/
1orpusID:2468633
Hai Dang Tran and Andrew Yates. Dense retrioval with entity views. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management, pp.
2022.
Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. Advances in neural information processing systems, 30, 2017.
Ashish Vaswani, Noam M. Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Iliia Polosukhin. Attention is all you need. In Neural Information Processing Systems, 2017. URL httpp://api. semanticscholart wtt.://ccoo..ooooooooooooiaaaaaa
Jianling Wang, Haokai Lu, Jamos Caverlee, Ed H Chi, and Minmin Chen. Large language models as data augmenters for cold-start item recommendation. In Companion Proceedings of the ACM Web Conference 2024, pp. 226-729, 2024.
15

---

Published in Transactions on Machine Learning Research (06/2025)
Yujing Wang, Ying Hou, Hong Wang, Ziming Miao, Shibin Wu, Hao Sun, Qi Chen, Yuqing Xia, Chengmin Chi, Guoshuai Zhao, Zheng Liu, Xing Xie, Hao Sun, Weiwei Deng, Qi Zhang, and Mao Yang. A neural cor- pus indexer for document retrieval. ArXiv,
2022. URL https///app..eeantiicchhh.....
org/CorpusID. (tt.pppu1--2229955554 249395549
D.H. Wolpert and W.G. Macready, No free lunch theorems for optimization. IEEE Transactions on Evolu- tionary Computation, 111):67-82, 1997. doi: 10.1109 00.1109/42355 /4235.585893.
Xinyang Yi, Ji Yang, Lichan Hong, Derek Zhiyuan Cheng, Lukasz Heldt, Aditee Kumthekar, Zhe Zhao, Li Wei, and Ed H. Chi. Sampling- correcere neural modeling for large corpus item recommen- dations. Proceedings of the 13th ACM Conference on Recommender Systems, 2019. URL https:
lapi.semanticscholaar weeemaniiiiccoolar..rrr/rprppppe wrtpCorppsID:20000000 202639719
Tingting Zhang, Pengpeng Zhao, Yanchi Liu, Victor S Sheng, Jiajie Xu, Deqing Wang, Guanfeng Liu, Xiaofang Zhou, et al. Feature-level deeper self-attention network for sequential recommendation. In IJCAI, pp. 4320-4326, 2019.
Yin Zhang, Derek Zhiyuan Cheng, Tiansheng Yao, Xinyang Yi, Lichan Hong, and Ed H. Chi. A model of two tales: Dual transfer learning framework for improved long-tail item recommendation. Proceedings of the Web Conference 2021, 2020. URL http::///ap..sssntissssseee
org/ CorpussDD:222211758 226221746
Yin Zhang, Ruoxi Wang, Derek Zhiyuan Cheng, Tiansheng Yao, Xinyang Yi, Lichan Hong, James Caverlee, and Ed H. Chi. Empowering long-tail item recommendation through cross docoupling network (cdn). Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Dala Mining, 2022. URL
https://api. www.sssesanniccoole......
CCOrppuII::23111770
Bowen Zheng, Yupeng Hou, Hongyu Lu, Yu Chen, Wayne Xin Zhao, and Ji rong Wen. Adapting large language models by integrating collaborative semantics for recommendation. 2024 IEEE 40th International Conference on Data Engineering (ICDE), pp. 1435-1448, 2023. URL https://api semantccshoola.r
org//orpusID:225211199
Bowen Zheng, Yupeng Hou, Hongyu Lu, Yu Chen, Wayne Xin Zhao, Ming Chen, and Ji-Rong Wen. Adapting large language models by integrating collaborative semantics for recommendation. In 2024 IEEE 40th International Conference on Data Engineering (ICDE), pp. 1435- 1448. IEEE, 2024.
Carolina Zheng, Minhui Huang, Dmitrii Pedchenko, Kaushik Rangadurai, Siyu Wang, Gaby Nahum, Jie Lei, Yang Yang, Tao Liu, Zutian Luo, et al. Enhancing embedding representation stability in recommendation systems with semantic id. arXic preprint arXiv:2504.0219 021387, 2025.
Kun Zhou, Hui Wang, Wayne Xin Zhao, Yutao Zhu, Sirui Wang, Fuzheng Zhang, Zhongyuan Wang, and Ji-Rong Wen. S3-rec: Self-supervised learning for sequential recommendation with mutual information maxinization. In Proceedings of the 29th ACM international conference on information & knowledge. management, pp. 1993-1902,-111 2020.
A Related Work
Generative Retrieval. The concept of generative retrieval was first proposed by Tay et al. (002)) within the domain of document retrieval. This paradigm shifts from traditional scarch and retrieval methods by en- coding document information directly into the weights of a Transformer model. Subsequent studies (De Cao
et al., 2020 Bevilacqua et al.) 2022 Feng et al. 2022 have expanded
this foundation, enhancing docu-
ment retrieval through improvements in indexing [Lee et al.
2022b
Wang et al. 2022), and the efficient
continual database updates (Mehta et al. 2022 Kishore et al. 2023
Chen et al. 2023).
In the realm of sequential recommendation systems, Rajput et al. 2024 is the first work to leverage the gencrative retricval techniques. The target item is directly generated given a user's interaction history, rather than selecting top items by ranking all relevant user-item pairs. A key challenge in generative retrieval is striking balance between memorization and generalization when encoding items. To address this, semantic
16

---

Published in Transactions on Machine Learning Research (06/2025)
IDs have been proposed by leveraging RQ- VAE models (Lee et al. 2022a Van Den Oord et al. 2017 These models encode content-based embeddings into compact, discrete semantic indexer that captures the hierar- chical structure of concepts within an item's 's content, proving to be scalable in industrial applications (Singh
ct al. 2023). Recent developments (Hou et al. 2022a) have expanded semantic-ID-basee generative retrieval
to include contrastive learning (Jin et al. 2024], multimodal integration (Liu et al. (024a), tokenization techniques (Sun et al. 2024.. learning-to-rar methods (Li al. 2024), and improved RQ- VQ- training (Qu
et al. 2024).
Sequential Dense Recommendation. Traditional sequential dense recommender models follow the paradigm of learning representations of users, items, and their interactions with multimodal data. Early work (Hidasi et al. 2015) proposed architectures based on traditional Recurrent Neural Networks (RNNs), while later studies (Kng & McAuley 2018a Sun et al. 2019 de Souza Pereira Moreira et al. 2021 have shifted towards the Transformer architecture to enhance performance. Besides capturing the user-item in- teraction history pattorn with the sequential modeling, extra features such as item attributes (Zhang et al., 2019 Zhou et al. 2020] has been utilized to further improve the performance. With the recent. advancements in Large Language Models (LLMs), several works have explored using these models as the backbone for rec- ommender systems, aligning item representations with LLMs to improve recommondation performance (Li ct al. 2023b Hou et al. 2022c Cao et al. 2024a Zheng et al. 2024p. In this work, we aim to merge the sequential dense recommendation approach with generative retrieval techniques, assessing performance gaps and resource efficiency, and proposing a hybrid method that combines the strengths of both paradigms.
Cold-start Problem. Traditional challenges such as long-tail and cold-start items continue to hinder rocommendation systems. The long-tail items issue arises from skewed distributions where a few popular items dominate user interactions (Zhang et al. 2022 2020), while the cold-start problem arises when new items are introduced without any historical interaction data. This problem is particularly prevalent in industrial settings (Rangadurai et al.] [2022] Jeon et al. 2024). where systems must frequently incorporate new items under tight latency and scale constraints. Recent studics (Hou et al.) 2022c Li ct al.) 2023b) have shown that textual embeddings can provide a robust prior for tackling the cold-start issue, and further improvements have been achieved by integrating pretrained LLMs (Huang et al.) 2024 Sanner et al. 2023 Wang et al. 2024h and knowledge graphs (Frej al.) 2024). In this work, we explore the cold-start problem within the context of generative retrieval and propose hybrid method that combines dense retrieval with textual embeddings to effectively mitigate this issue. A concurrent work by [yin et al. (2024) also investigates the cold-start issue in generative retrieval and proposes using dense retrieval as a drafter model to facilitate cold-start item generation.
B Experimental Details
B.1 Implementation Details of Dense Retrieval
In Section 4.1 we provide the TIGER implementation details. Here, we describe the implementation atio details of the dense retrieval method we developed.
The dense retricval method uses the same T5-encoder architecture, configured with layers, an embedding dimension of 128, 6 attention heads, and a feed-forward network hidden dimension of 1024. For content embeddings, use the same model as TIGER, Sentence-T5-XX: which generates text embeddings in R 768, To integrate these embeddings into the Transformer, we project them down to a 128-dimensional space using a Linear layer.
Following the notation in Section 2.2 once E, is collected for each item, it is passed through a LayerNorm layer, followed by a Dropout layer with a rate of 0.5. For the cosine similarity loss calculation, we set the temperature parameter 7 == 0.07. During training, we use the same learning rate (0.0003), cosine learning rate scheduler, optimizer (AdamW); and weight decay (0.035) employed in TIGER's training.
17

---

Published in Transactions on Machine Learning Research (06/2025)
Table 3: Dataset statistics after applying 5-core filtering to both users and items. The first three datasets (Beauty, Sports, and Toys) are subsets of the Amazon review dataset.
Datasct
# users
# items
# actions # cold-start items
Beauty
22,363
12,101
198,502
43
Toys and Games
19,412
11,924
167,597
56
Sports and Outdoors
35,598
18,357
296,337
81
Steam
47,761
12,012
599,620
400
B.2 Implementation Difference Between UniSRec and our Dense Model
Although the dense retrieval model we implemented is inspired by UniSRec (Hou et al, 2022c we made several modifications to simplify the model and align it with the best-tuned TIGER architecture. Specifically:
1. We use the same content model as TIGER: T5-XII, whereas UniSRec uses BERT.
We adopt the same encoder architecture as TIGER, which consists of T5 encoder layers with 6 atlention heads, an embedding dimension of 128, and a leed-forward network hidden dimension of 1024. In contrast, UniSRec uses custom Transformer block with 2 layers, 2 attention heads, an embedding size of 300, hidden size of 256, and different implementations for LayerNorm and positional embeddings compared to the T5 model
We replace UniSRec's mixture-of-experl layer and whitening layer with a simple Linear layer.
4. We train the dense retrieval model from scratch, whereas UniSRec relies on pretraining using the Amazon 2018 datasets.
B.3
Data Statistics
In Table we present the statistics of the datasets used in evaluation.
B.4 Baselines
We compare our methods with five state-of-the-art Hem--D-based dense retrieval methods, including:
1. SASRec (Kngg & McAulcy 2018b). A self-attention based sequential recommendation model that learns to predict the next item ID based on the user's interaction history.
FDSA (Zhang
2019) [feature-informed]] This method extends SASRec by incorporating item features into the sell-attention model, allowing it to leverage prior information about cold-start items through their attributes.
3. S3-Rec Zhhou et al. 2020) [feature-informeed] self-attention based model that utilizes data corre- lation to create self-supervision signals, improving sequential recommendation through pre-training,
4. UnisRec (Hou ct al. 2022b [modality-based]] A model that learns universal item representations by utilizing associated description text and lightweight encoding architecture that incorporates parametric whitening and a mixture-of-expee adaptor. We fine-tune the released pretrained model in the transductive sctting.
Recformer (Li et al. 2023a) (modality-based bidirectional Transformer-basee model that encodes item information using key-value attributes described by text. We fine-tune the pro-trained model on the downstream datasels.
18

---

Published in Transactions on Machine Learning Research (06/2025)
C
Full Experimental Result
present the full results the benchmark, where method with different number of retrieved candidates from generative retrieval are shown.
In Table 4
D Ablation Study
LIGER combines dense retrieval and generative retrieval paradigms into a unified framework. As described in Section and illustrated in Figure 2 for cach item i with semantic ID (st, , ,,"),, construct the input embedding for each semantic ID as:
E e + ei e + e
+ e +ep
where o₂ is the learnable embedding for the of semantic ID, e is the item's text representation, ot is the positional embedding for the item, and e the positional embedding for the semantic ID. The final embedding for item i then represented as: E, = [,,
During training, the model optimized with two objectives: the cosine similarity loss and the next-token prediction loss the semantic IDs of the next item: The cosine similarity loss ensures that the model learns to aligu the encoder's output embedding with the text representation of the next item, and the next-token prediction loss supervise on the next item's semantic ID tuple prediction performance. To summarize, the LIGER framework is structured as follows:
Input: Semantic ID (SID) and item's text representation;
2. Output: Predictions for:
(a) The next item's semantic ID through the SID head.
(b) The next item's lext representation through the embedding head.
The ablation study investigates the impact of each component, as shown in Figure 7
LIGER (detach) detaches the gradient updates from the SID head in LIGER to examine the impor- tance of multi-objective optimization through the SID head;
TIGER (T) removes the embedding head from LIGER, focusing solely on the SID head and the text representation as input:
TIGER further simplifies TIGER (T) by removing the item text representation input, reducing the model to the generative retrieval method described in Section 2.1
Dense (SID) removes the SID head from LIGER, retaining only the dense retrieval mechanism with SID as input;
Dense replaces the SID input with ID in Dense (SID), reducing the model to the dense retricval method transductive setting, as detailed in Section 2.2
Figure presents the ablation results in terms of Recall@10 10 across four datasets: Beauty, Sports, Toys, and Steam. The performance is analyzed with respect to the number of candidates retrieved by the SID head (K).
First, comparing TIGER to TIGER(T), we observe that TIGER(T) consistently performs the same or slightly better, demonstrating the positive impact of incorporating the item's text representation as input. However, the improvement is modest, indicating that while text representation is helpful, its contribution alone does not significantly enhance the model's performance.
19

---

Published in Transactions on Machine Learning Research (06/2025)
TIGER
LIGER
Dense (SID) +
Text Rep.
SID
Seq. Model
SID
Text
SID
See Model
Text Rep.
SID
Text
SID
Model
Text Rep.
TYGER
LIGER (ietaah)
Dense
SID
Seq. Model
SID
Text Rep.
SID
Seq Model
+
Text Rep.
SID
Text Rep.
Mode
Seg.
Text Rep.
Figure 7: Overview of our Ablation Study. This study examines the effects of different components within LIGER (lop middle), which inlegrates TIGER and semantie ID (SID)-based dense retrieval in an transductive setting. LIGER takes both the semantic ID and item lext representation as inputs, predicting the SID and generating embeddings, We perform the following ablations to evaluate the impact of specific components: (1) To assess the effect of multi- objective optimization, we detach gradient updates from STD head (bottom middle).
(2) To study the role of the embedding head, remove it (top left). (3) To evaluate the contribution of the item text representation input in (2), we remove it, reducing the model to TIGER (bottom left). (4) To analyze the effect the SID head, we remove it (top right). (5) Finally, replace the SID with item IDs in (4), reducing the model to standard dense retrieval in transductive setting (bottom right).
TIGER (T) +
LIGER
Dense (SID) +
TIGER
LIGER (detach)
Dense
Beauty
Sports
Toys
steam
0.055
0.060
0.33
0.205
0.090
0.055
0.10
Recall@
080
0.050
0.09
0.075
0.045
0.08
0.185
0.065
0.060
0.035
0.06
10
50
100
10
50
100
10
50
100
N
30
50
100
N
Candidates Retrieved by Gen. Ret., K
Figure 8: Ablation Results on Recall@ 10 across Datasets.
Second, when comparing Dense retrieval with Dense (SID), the results similar, suggesting that the primary limitation of the dense retrieval method is not due to the type of representation used (ID vs. SID). This highlights that the bottleneck contributing to the performance difference lies elsewhere, possibly in the learning of SID.
LIGER, which combines TIGER and dense retrieval paradigms, exhibits a smooth interpolation between the performance of TIGER and Dense. This suggests that LIGER effectively leverages the strengths of both approaches to achieve robust performance across datasets. Notably, when the gradient update from the SID head is detached (IIGERddetac)) the model still performs comparably to the standard LIGER, with the most significant drop observed on the Steam dataset. This result implies that the SID head's learning signal is crucial for the Steam dataset, which is of larger scale compared to the Amazon datasets.
Overall, these results demonstrate that LIGER strikes a balance between TIGER and dense retrieval meth- ods while retaining flexibility through multi-objective optimization. However, the importance of the SID hcad's gradient signals appears datasct-dependent, possibly also influenced by datasct scales, as highlighted by the performance gap on Steam.
20

---

Published in Transactions on Machine Learning Research (06/2025)
Table 4: Performance Tuble for Amazon Beauty, Sports, Toys, and Steam Dalaseis Across Various Baseline Methods. In this table, present method with different number of retrieved candidates K from generalive retrieval.
NDCG@101 (%)
Recall@111 (%)
Datasets
Methods
Inference Cost
In-set
Cold
In-set
Cold
SASRec
O(N)
2.179 + 0.023
0.000 + 0.000
5.109 + 0.042
0.000 + 0.000
FDSA
((N)
2.244 0.135
0.000 = 0.000
4.530 = 0.357
0.000 0.000
S*-Rec
((N)
2.279 0.058
0.000 0.000
5.226 0.229
0.000 0.000
Beauty
UniSRec
O(N)
3.346 + 0.057
1.422 + 0.128
6.937 = 0.110
3.704 + 0.000
RecFormer
O(N)
2.880 0.085
1.955 0.433
6.265 0.196
4.733 0.943
TIGER
O(tK)
3.216 + 0.084
0.000 + 0.000
6.009 + 0.204
0.000 + 0.000
Ours (K =0)
O(t))
1.020 0.04
3.800 + 0.523
7.447 + 0.111
10.082 1.285
Ours (K 40)
O(tK)
4.230 0.067
2.815 = 0.525
7.985 = 0.033
7.613 1.285
Ours (K 60)
о(K)
4.328 1 0.113
2.609 T. 0.343
8.207 T 0.101
7.202 T. 0.943
Ours (K 80)
O(L))
4.392 1 0.097
2.550 1 0.425
8.343 1 0.115
7.202 0.943
Ours [K- 100)
O(tK) 4.430 + 0.101
2.469 + 0.381
8.447 = 0.126
7.202 + 0.943
Ours (K N)
O(N)
4.738 0.151
1.225 0.256
9.210 0.247
3.704 0.617
SASRec
O(N) 1.160 0.038
0.000 0.000
2.696 0.102
0.000 0.000
FDSA
O(N)
1.391 + 0.162
0.000 + 0.000
2.699 = 0.312
0,000 + 0.000
S
O(N)
1.977 = 0.033
0.000 = 0.000
2.537 = 0.034
0.000 = 0.000
Sports
UniSRec
O(N)
1.814 0.041
0.676 + 0.244
3.753 + 0.106
1.559 0.447
RecFormer
O(N)
1.318 0.053
1.797 T. 0.000
2.921 1 0.167
3.801 1 0.000
TIGER
O(t))
1.989 0.085
0.064 0.056
3.822 0.109
0.195 0.169
Ours [K 20)
O(tK)
2.430 + 0.075
2.731 + 0.229
4.400 = 0.127
5.848 + 0.292
Ours 40)
O(K))
2.594 0.063
1.814 0.052
4.789 = 0.067
4.191 0.338
Ours (K 60)
O(tK)
2.672 + 0.067
1.517 + 0.068
4.987 + 0.098
3.411 + 0.338
Ours (K 80)
O(tK)
2.714 0.064
1.270 + 0.096
5.071 0.094
2.921 0.506
Ours (K 100)
O(tK) 2.744 0.055
1.175 = 0.059
5.141 = 0.087
2.729 0.169
Ours (K- N)
O(N)
2.962 0.053
0.581 1 0.238
5.641 1 0.071
1.365 1 0.447
SASRec
O(N) 2.756 0.079
0.000 = 0.000
6.314 = 0.178
0.000 = 0.000
FDSA
O(N) 2.375 0.277
0.000 T 0.000
4.684 1 0.483
0.000 0.000
S9-Rec
O(N)
2.942 1 0.071
0.000 1 0.000
6.659 1 0.135
0.000 1 0.000
Toys
UniSRec
O(N)
3.622 0.056
1.090 + 0.084
7.472 + 0.058
2.477 0.195
RecFormer
O(N)
3.697 + 0.052
4.432 + 0.094
7.971 + 0.170
10.023 + 0.516
TIGER
О(К) 2.949 0.049
0.000 = 0.000
5.782 = 0.163
0.000 = 0.000
Ours (K =20)
O(K) 3.756 0.151
5.231 + 0.531
7.135 + 0.244
13.063 + 0.516
Ours (K =10)
О(К)
1.021 +: 0.157
3.574 + 0.262
7.859 + 0.264
9.459 + 0.585
Ours (K -60)
O(t))
4.163 0.140
3.173 0.149
8.222 0.194
8.559 0.516
Ours (K 80)
O(t)) 4.245 = 0.130
2.861 + 0.139
8.375 = 0.187
7.770 + 0.338
Ours 100)
O(K) 4.288 0.120
2.783 0.108
8.468 1 0.172
7,658 0.195
Ours (K N)
O(tK)
4.680 + 0.086
2.149 + 0.202
9.482 + 0.117
6.081 + 0.676
SASRec
O(N) 14.763 + 0.051
0.000 + 0.000
18.259 + 0.055
0.000 + 0.000
FDSA
O(N) 8.236 0.152
0.000 0.000
14.773 0.234
0.000 0.000
S--Rec
O(N)
14.377 0.127
0.000 0.000
18.025 0.222
0.000 0.000
Steam
UniSRec
O(N)
RecFormer
O(N)
14.034 0.123
0.120 0.037
17.042 0.275
0.319 0.110
TICER
O(IK)
15.034 1 0.064
0.000 1. 0.000
18,980 1 0.135
0.000 1 0.000
Ours (K 20)
O(t))
14.951 0.158
0.512 + 0.047
19.049 0.234
1.466 0.110
Ours (K 40)
O(tK)
15.138 + 0.110
0.298 + 0.068
19.302 + 0.180
0.829 + 0.110
Ours (K =60)
O(K))
15.236 = 0.074
0.258 + 0.109
19.455 = 0.154
0.701 + 0.292
Ours (K -80)
O(K))
15.281 0.059
0.226 + 0.105
19.522
0.143
0.637 0.292
Ours (K -100)
о(K)
15.318 - 0.049
0.222 + 0.109
19.566 0.132
0.637 + 0.292
Ours (K .N)
O(tK)
0.006 15.431
0.175 0.082
19.736 0.086
0.510 0.221
21


Reviews:
[{"review_id": "RjY0QI5BSj", "reviewer": "Reviewer_419q", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "* **Strengths:** This paper is well written. The motivation is clearly articulated, and the authors provide a detailed discussion of the strengths and weaknesses of two well-known retrieval approaches. Moreover, the experimental results on multiple datasets further demonstrate the effectiveness of the proposed method.\n* **Weaknesses:** I do not see any major weaknesses. However, certain aspects of the writing could be improved in terms of clarity and presentation.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The primary contribution of this paper lies in the proposal of a unified model, LIGER, which combines the strengths of dense retrieval and generative retrieval models. The authors compare these two approaches under controlled conditions on academic benchmarks and observe performance differences: dense retrieval demonstrates superior ranking performance, whereas generative retrieval offers greater resource efficiency. In addition, the authors conduct extensive experiments to validate the effectiveness of their proposed method.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "* In the Introduction and abstract, \"Despite their effectiveness, these approaches require ...\" would be more appropriate. \n* In Section 2.3, the claim \"There are notable differences between the two implemented methods: ...\" uses (1) and (2) for enumeration. I strongly recommend that the authors avoid this numbering style, as it can easily be confused with equation numbering. For instance, when the authors mention \"To analyze the effect of (1), we modify ...\", I initially thought the following \"(1)\" referred to the equation for input embedding for each semantic ID, which caused some confusion. \n* In Table 2, would it be better to present the results using percentage signs? What I mean is that using percentage values could improve the readability and overall aesthetics of the table. \n* Regarding inference cost, the authors currently provide only theoretical results in Table 2, such as $O(N)$ and $O(tK)$. To better demonstrate the time efficiency of the proposed method, including a comparison of inference cost against other baselines would make the evaluation more convincing.", "broader_impact_concerns": "There are no ethical concerns associated with this work that would require adding a Broader Impact Statement.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "63AmchiD7A", "reviewer": "Reviewer_Cqaa", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n(S1) The advantages and disadvantages of the dense retrieval and the generative retrieval approaches were analyzed and empirically verified.\n\n(S2) A hybrid model LIGER was proposed by integrating the text representation based cosine similarity loss into generative retrieval approaches.\n\n(S3) Experimental results showed that this hybrid model could achieve similar performance as the dense retrieval when the number of candidates increases, and this model outperformed baselines on several benchmarks.\n\n\nWeaknesses:\n\n(W1) The comparison between the dense retrieval and the generative retrieval approaches can be further illustrated as follows.\n- How is the \"best-tuned TIGER architecture\" is obtained?\n- What is the impact of model architectures on the findings in the model comparison? Do the empirical observations hold for any other model architecture?\n- Can the comparison results be applied to other generative retrieval and dense retrieval approaches summarized in related works.\n- Can dense retrieval approaches outperform generative retrieval methods on large-scale datasets?\n\n(W2) It is unclear how the added cosine similarity loss in LIGER affects the training efficiency and model convergence.\n\n(W3) Experimental results are not well explained. \n- LIGER approximates the results of dense retrieval only when $K$ significantly increases. The gap between LIGER and dense retrieval still exists in real-world scenarios where $K$ is small. This phenomenon needs more explanation.\n- In some cases where $K=10$, the Normalized Performance Gap can be negative. Does it imply that LIGER achieves inferior performance than TIGER when $K$ is small?\n- It is more convincing to include the results of dense retrieval methods in Table 2.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper provided the empirical comparison between the dense retrieval and the generative retrieval approaches. It identified the advantages and disadvantages of these two approaches. Specifically, it then introduced a new hybrid model named LIGER to combine the strengths of both approaches. Experimental results showed that this hybrid model achieved better performance than baseline models on several benchmarks.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "(1) It will be more convincing to clarify whether the comparison results hold generally for other model architecture, other recent dense/generative retrieval approaches, and large-scale benchmarks.\n\n(2) The effectiveness of LIGER needs to be further justified in the experiments, especially when the number of candidates $K$ is small.", "broader_impact_concerns": "No ethical concerns are involved in this work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "DE6AbZcWFg", "reviewer": "Reviewer_1GZy", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The **strengths** are outlined in the contributions and can be summarized as follows:\n- A clear and concise introduction to the background information.\n- An in-depth analysis of two popular paradigms, with insightful conclusions regarding which factors contribute to performance differences. These findings may provide valuable guidance for future research.\n- Experimental results demonstrate that the proposed method, LIGER, effectively combines the strengths of both paradigms.\n\n**weakness**:\n1. **Novelty**. The proposed method—particularly its input structure (SID and item representation) and output (next item and item text representation)—is largely inspired by the two existing paradigms. As a result, the novelty of the approach is somewhat limited. (That said, since TMLR does not strictly require high novelty, this is not a serious drawback.)\n2. **Practicality**. The paper does not discuss how LIGER could be applied in real-world online recommendation systems. For example, inference time is a critical constraint in online systems. While LIGER may have lower storage complexity and inference latency compared to sequential dense retrieval, the potential impact on online applications is not analyzed. Additionally, the frequency and importance of cold-start items in practical settings are not addressed. Including some online analysis or deployment-focused experiments would significantly enhance the practical value of the work.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "1. **Background Information**: The author clearly outlines two popular recommendation paradigms and compares their respective strengths and weaknesses. The background information is well-organized, clearly presented, and aligns with the prevailing consensus in the recommendation systems community.\n&nbsp;\n\n2. **Motivaton and priliminary study**: \n- The author identifies three key differences between \"Sequential Dense Retrieval\" and \"Generative Retrieval\" and investigates which factor contributes most significantly to the performance gap. The findings \"semantic indexing effectively preserves the information contained in item IDs\" and \"item’s text representation only yields marginal improvements\" are both insightful and non-trivial, potentially offering valuable guidance for future research.\n- The analysis and experiments presented in Section 2.4 and Figure 4 convincingly explain why generative retrieval methods struggle with cold-start items.\n\n3. **Solution (LIGER)**: The author proposes LIGER, a novel approach that combines the strengths of both paradigms. The model’s input structure and loss function are clearly articulated through precise mathematical formulations.\n\n4. **Experiments**: LIGER is evaluated against several strong baselines, demonstrating superior performance. Additionally, the ablation study in Appendix D provides detailed insights into the effects of input heads and multi-objective optimization.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "**Currently, I recommend acceptance of the paper, even if the authors make no changes. However, I believe the following tips could further improve the quality and impact of the work.**\n\n1. In Section 2.4, the author demonstrated that generative retrieval models fail to handle cold-start items. But I believe, this issue can also arise--albeit to a lesser extent--in sequential dense retrieval models. I recommend adding further analysis here. For instance, by examining the distribution of \"next item representations\" (i.e., the outputs of the transformer), one could investigate whether these representations occupy a concentrated region in the embedding space, and whether cold-start items deviate from this distribution.\n\n2. Consider adding some online analyses or deployment-oriented experiments, as discussed in Weakness 2. Such additions would greatly enhance the practical relevance of the work.\n\n3. Some terminology should be reviewed for clarity and consistency. The limitations of sequential dense retrieval are generally related to both storage and computational costs. When mentioning the limitation, the author sometimes only include \"storage\" (Abstract line 4), sometimes only include \"computation\" (Abstract line 13), sometimes mention them both. They should be unified. Additionally, the term \"overfitting\" is used to explain why the model fails to handle cold-start items. Typically, overfitting refers to a model fitting noise or non-generalizable patterns in the training data. Since cold-start items do rarely appear in training, the issue may be more accurately described as under-representation or data sparsity rather than overfitting--unless \"overfitting\" is a term already established in this context by prior literature.", "broader_impact_concerns": "No.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The manuscript was well reviewed by three referees. After revision, two reviewers vote for acceptance, while one for weak reject. Summarizing the reviews, the manuscript is thought to be sufficiently strong in motivation, methodology, and experiments. The exploration of the two paradigms of dense retrieval and generative retrieval is intriguing, although the combination nature makes the novelty less solid. The manuscript can be further improved by strengthening the performance when $K$ is small.
Decision: Accept (poster)