Sparse Meets Dense: Unified Generative Recommendations with Cascaded Sparse-Dense Representations
Authors: Yuhao Yang
Zhi Ji
Zhaopeng Li
YI LI
Zhonglin Mo
Yue Ding
Kai Chen
Zijian Zhang
Jie Li
shuanglong li
LIU LIN
Venue: NeurIPS 2025 poster ()
Keywords: Generative Recommendation
Primary area: applications

Abstract: Generative models have recently gained attention in recommendation systems by directly predicting item identifiers from user interaction sequences. However, existing methods suffer from significant information loss due to the separation of stages such as quantization and sequence modeling, hindering their ability to achieve the modeling precision and accuracy of sequential dense retrieval techniques. Integrating generative and dense retrieval methods remains a critical challenge. To address this, we introduce the Cascaded Organized Bi-Represented generAtive retrieval (COBRA) framework, which innovatively integrates sparse semantic IDs and dense vectors through a cascading process. Our method alternates between generating these representations by first generating sparse IDs, which serve as conditions to aid in the generation of dense vectors. End-to-end training enables dynamic refinement of dense representations, capturing both semantic insights and collaborative signals from user-item interactions. During inference, COBRA employs a coarse-to-fine strategy, starting with sparse ID generation and refining them into dense vectors via the generative model. We further propose BeamFusion, an innovative approach combining beam search with nearest neighbor scores to enhance inference flexibility and recommendation diversity. Extensive experiments on public datasets and offline tests validate our method's robustness. Online A/B tests on a real-world advertising platform with over 200 million daily users demonstrate substantial improvements in key metrics, highlighting COBRA's practical advantages.

Sparse Meets Dense: Unified Generative Recommendations with Cascaded Sparse-Dense
Representations
Yuhao Yang, Zhi Jł, Zhaopeng Li, Yi Zhonglin Mo, Yue Ding, Kai Chen, Zijian Zhang, Jie Li, Shuanglong Li, Lin Liu
Baidu Inc..
Beijng. (yangyuhao01, jizhi bizhi)@baidu.com com
Beijing, China
Abstract
Generative models have recently gained attention in recommendation systems by directly predicting item identifiers from interaction sequences. However, exist- ing methods suffer from significant information loss due to the separation of stages such as quantization and sequence modeling, hindering their ability to achieve the modeling precision and accuracy of sequential dense retrieval techniques. Inte- grating generative and dense retrieval methods remains critical challenge. To address this, we introduce the Cascaded Organized Bi-Represented generAtive retrieval (COBRA) framework, which innovatively integrates sparse semantic IDs and dense vectors through a cascading process. Our method alternates between generating these representations by first generating sparse IDs, which serve conditions to aid in the generation of dense vectors. End-to-end training enables dynamic refinement of dense representations, capturing both semantic insights and collaborative signals from user-item interactions. During inference, COBRA employs coarse-to to-fine strategy, starting with sparse ID generation and refining them into dense vectors via the generative model. We further propose BeamFusion an innovative approach combining beam search with nearest neighbor scores enhance inference flexibility and recommendation diversity. Extensive experiments on public datasets and offline tests validate our method's robustness. Online A/B tests on a real-world advertising platform with over 200 million daily users demon- strate substantial improvements key metries, highlighting COBRA's practical advantages.
1 Introduction
Recommendation systems are vital components of modern digital ecosystems, providing personalized item suggestions that align with user preferences across e-commerce platforms, streaming services, and social networks [+3]. Recent advancements have focused on sequential recommendation methods, which leverage the sequential nature of user interactions to enhance recommendation performance 7. Notable models like SASRec [ and BERT4Rec [9] have demonstrated the
effectiveness of sequence models in capturing user behavior patterns.
The emergence of generative models has further expanded the capabilities of recommendation sys- tems 12) Unlike traditional sequential recommendation methods, generative models can directly predict target items based on user behavior sequences [5)15]. These models handle complex user-item interactions and offer emerging abilities such as reasoning and few-shot learning, which significantly improve recommendation accuracy and diversity [6-[8]. Among these, TIGER [I9
Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

TIGER
COBRA (Ours)
ID=(3, 24, 55)
Coarse
Fine Rep.
Title, Brand, Category....
Tatla, Brand, Category...
Densersa
Next Item 733
Transformer Decoder
ID=(7, 28, 67)
Transformer Decoder
Bidirectional Transformer Encoder
134
Denavass
1Da7a
Densegre
3.3
24
t.55
L11
t,31
t,79
Coarse ID
Fine Rep.
Coarse ID
Fine Rep.
Item 134 ID=(3, 24, 55)
Item 278
ID=(11, 31, 79)
Item 134
Item 278
Figure I: Comparison of generative recommendation paradigms. The left side illustrates traditional generative retrieval approaches, exemplified by TIGER, which uses a sequence of sparse IDs as input within Transformer recoder-decoder architecture to directly predict the sparse ID of the next item. The right side depicts the proposed COBRA( (OBRA(Caseaded Organized Bi-Repree Bi-Represented generAtive retrieval), which integrates sparse IDs for coarse semantics and dense vectors for fine details. The cascaded representation is processed by Transformer decoder that sequentially predicts the sparse ID followed by the dense vector.
pioneering approach in generative retrieval for recommendation systems. As depicted in Fig- ure Lower Left), TIGER leverages Residual Quantized Variational AutoEncoder (RQ-VAE) [2) to encode item content features into hierarchical semantic IDs, allowing the model to share knowledge across semantically similar items without the need for individual item embeddings. Beyond TIGER, several other methods have been proposed further explore the integration of generative models with recommendation systems. LC-Rec [21] aligns semantic and collaborative information using RQ- IQ-VAE with series of alignment tasks. ColaRec [22] combines collaborative filtering signals with content information by deriving generative identifiers from a pretrained recommendation model. IDGenRee 23 leverages large language models generate unique, concise, and semantically rich textual identifiers for recommended items, showing strong potential in zero-shot settings.
Despite these innovations, existing generative recommendation methods still face several challenges compared to sequential dense retrieval methods 24 25 Sequential dense retrieval methods, which rely on dense embeddings for each item, offer high accuracy and robustness but require substan- tial storage and computational resources. In contrast, generative methods, while efficient, often struggle with fine-grained similarity modeling [26). To effectively leverage the strengths of both retrieval paradigms, we propose Cascaded Organized Bi-Represented generAtive retrieval(COBRR framework that synergizes generative and dense retrieval. Figure Right illustrates the cascaded sparse-dense representations in COBRA. The proposed method introduces cascaded generative retrieval framework alternating between generating sparse IDs and dense vectors. This approach mitigates the information loss inherent in ID-based methods. Specifically, COBRA's input is a sequence of cascaded representations composed of sparse IDs and dense vectors corresponding to items in the user's interaction history. During training, dense representations are learned through contrastive learning objectives in end-to-end manner. By first generating the sparse IDD and then the dense representation, COBRA reduces the learning difficulty of dense representations and promotes mutual learning between the two representations. During inference, COBRA employs coarse-to- tonrse-o--fine generation process, starting with sparse ID that provides a high-level categorical sketch capturing the categorical essence of the item. The generated ID is then appended to the input sequence and fed back into the model to predict the dense vector that captures the fine-grained details, enabling more precise and personalized recommendations. To ensure flexible inference, we introduce BeanFusion, sampling technique combining beam search with nearest neighbor retrieval scores, ensuring con- trollable diversity in the retrieved items. Unlike TIGER, which relies solely on sparse IDs, COBRA harnesses the strengths of both sparse and dense representations.
Figure[[Right)
Our main contributions are follows:

---

Cascaded Bi-Represented Retrieval Framework: We introduce COBRA, framework that alternates between generating sparse semantic IDs and dense vectors. It addresses information loss in ID>-based methods and reduces the difficulty of representation learning by using sparse IDs as conditions for generating dense vectors.
Learnable Dense Representations via End-to-End Training: COBRA uses the original item data as input to generate dense representations through end-to-end training. Unlike static embeddings, COBRA's dense vectors are dynamically learned, capturing semantic information and fine-grained details.
During inference, COBRA first generates sparse IDs, which are then fed back into the model to produce refined dense representations, enhancing the granularity of the dense vectors. We also introduce BeamFusion for more diverse recommendations.
Coarse-to-Fine Generation Process:
Comprehensive Empirical Validation: Extensive experiments on benchmark datasets show COBRA surpasses current methods in recommendation accuracy, proving its effectiveness in balancing precision and diversity.
2 Related Work
Sequential Dense Recommendation. Early sequential recommendation systems leveraged RNNS and CNNs to model user behavior sequences 27 28
The introduction of Transformer- based methods, such SASRec [ and BERT4Rec [9, greatly improved the capability capture complex user dynamics. Recent models focus on cross-domain transferability and the integration of textual features through contrastive learning [19
Generative Recommendation. Generative approaches have shifted the field from discriminative ranking to directly generating item identifiers [32 19 33). Some models treat recommendation as a language modeling task [33]. while others generate semantically meaningful or structured identifiers 19
34 Hybrid methods, such as LIGER 26). combine generative and dense retrieval to overcome the limitations of each approach. However, how to more flexibly integrate these
paradigms remains an open problem. A more comprehensive review is provided Appendix
3 Methodology
This section introduces the Cascaded Organized Bi-Re Bi-Represented generAtive Retrieval (COBRA) framework, which integrates cascaded searse-dense representations and coarse-to -ine generation to enhance recommendation performance. Figure illustrates the overall framework of COBRA.
3.1 Sparse-Dense Representation
Sparse Representation. COBRA generates sparse IDs using a Residual Quantized Variational Autoencoder (RQ-VAE), inspired by the approach in TIGER [19. For each item, extract its attributes to generate a textual description, which embedded into dense vector space and quantized to produce sparse IDs. These IDs capture the categorical essence of items, forming the basis for subsequent processing. For the sake of brevity, the subsequent methodology descriptions will assume that the sparse ID consists of single level. However, it should be noted that this approach can be easily extended to accommodate scenarios involving multiple levels.
Dense Representation. To capture nuanced attribute information, we develop end-to-end trainable dense encoder, encoding item textual contents. Each item's attributes are flattened into a text sentence, prefixed with [CLS] token, and fed into Transformer-based text encoder Encoder. The dense representation V, is extracted from the output corresponding to the [CL.) token, capturing fine-grained details of the item's textual content. As illustrated in the lower part of Figure 2 we incorporate position embeddings and type embeddings to model the positional and context of tokens within the sequence. These embeddings are added the token embeddings, enhancing the model's ability to distinguish between different tokens and their positions in the sequence.
Cascaded Representation. The cascaded representation integrates sparse IDs and dense vectors within unified generative model. Specifically, for each item, we combine its sparse ID ID, and dense vector v, to form cascaded representation (TDr, vz). This approach leverages the strengths of both representations, providing more comprehensive characterization of items: sparse IDs provide
3

---

Eeparse
Lieene
Feus
P4Ll
Transformer Decoder
Item Pos Emb
Ps
Par-t
Par
Pares
Pare
Item Type Emb
Type
Typeoeue
Typew
Tapra
Typtu
Type Jurza "
Type
Type
Type:p
Typeemo
Item Emb
Pii
ID
11
ID:
10;
10,
V:
Pisi
Cascaded Representation
Dense Vector
Sparse ID
Appsi
Bu,,
Aa
Puyx
Pug
Auu
Bidirectional Transformer Encoder
Encoder
Decoder
Token Emb
Wi
Wrss
Wrss
Wr
Wiss
W''s
Pos Emb
Type Emb
Type
Typeeraass
Type
Typrorad
cadebook
(CLS]
category text:..
title text:
brand text
Pretrained Quantization Model
item
The architecture of COBRA. The model employs cascaded sparse-dense representation approach, where sparse IDs are generated via Residual Quantization and dense vectors are produced by trainable Transformer Encoder. These representations serve inputs Transformer Decoder, which alternates between predicting sparse IDs and dense vectors. The predicted outputs are used compute the loss functions Lypurse and Ldense- For the sake of simplicity, the figure illustrates example with a single level of sparse ID.
Figure 2:
stable categorical foundation through discrete constraints, while dense vectors maintain continuous feature resolution, ensuring that the model captures both high-level semantics and fine-graine details.
3.2 Sequential Modeling
Probabilistic Decomposition. The probability distribution modeling of the target item is factorized into two stages, leveraging the complementary strengths of sparse and dense representations. Specifi- cally, instead of directly predicting the next item $141 based on the historical interaction sequence S COBRA predicts the sparse ID ID241 and the dense vector VT+1 separately:
Pe Pi St:t)
(1)
where P(ID++1 (1.)) represents the probability of generating the sparse ID ID++1 based the historical sequence Si.,. capturing the categorical essence the next item. ( ID+41, represents the probability of generating the dense vector Vi+1 given the sparse ID ID++1 and the historical sequence S1.t, capturing the fine-grained details of the next item. This decomposition allows COBRA to leverage both the categorical information provided by sparse IDs and the fine-grained details captured by dense vectors.
Sequential Modeling with Unified Generative Model. For sequential modeling, we utilize unified generative model based on the Transformer architecture to effectively capture sequential dependencies in user-item interactions. The Transformer receives input sequence of cascaded representations, with each item represented by its sparse ID and dense vector.
The sparse ID, denoted as DD, is transformed into dense vector space through embedding layer: e; Embedd (T)).. This embedding e; is concatenated with the dense vector v, to form the model's input at each time step: h, [e;; vzl.

---

Our Transformer Decoder model comprises multiple layers, each featuring self-attention mechanisms and feedforward networks. As depicted in the upper part of Figure [ the input sequence to the Decoder consists of cascaded representations. To enhance modeling of sequential and contextual information, these representations are augmented with item position and type embeddings. For brevity, mathematical formulations in the following sections focus on the cascaded sequence representation, omitting explicit notation for position and type embeddings. The Decoder processes this enriched input to generate contextualized representations for predicting the subsequent sparse ID and dense vector.
Sparse ID Prediction. Given history interaction sequence S1.t, to predict the sparse ID I DD++1 the Transformer imput sequence is: S1:; [hi h2... h,] ee,. V1, C2, V....., Ct Vel. where each h, is a concatenation of the sparse ID embedding and the dense vector for the i-th item. The Transformer processes this sequence to generate contextualized representations, subsequently used to predict the next sparse II) and dense vector. Specifically, the Transformer decoder processes the sequence S1:t. producing sequence of vectors y; TransformerDecoderr(SS The logits for sparse ID prediction are derived as: Z Sparsellead(y:). where Z++1 represents the logits for the predicted sparse ! Dr+1.
Dense Vector Prediction. For predicting the dense vector Vr+1,, the Transformer input sequence can be represented as: S = [S C++1 = [e1, V1, e2, V...... C₁, Vz, C++1]. The Transformer decoder processes S1: to output the predicted dense vector: Vi+1 = TransfonerDDecdee(S::.)))
3.3 End-to-End Training
In COBRA, the end-to-end training process is designed to optimize both sparse and dense representa- tion prediction jointly. The training process is governed by composite loss function that combines losses for sparse ID prediction and dense vector prediction.
Sparse ID Loss. The sparse ID prediction loss, denoted as Liparse, ensures the model's proficiency in predicting the next sparse ID based on the historical sequence SIz:
exp(2 ID++1)
Caparse
Σ log t=1
E exp(2(41)
where T is the length of the historical sequence, ID the sparse ID corresponding to interacted
item at time step & + 1, "t+1 ID++1 represents the predicted logit of ground truth sparse ID / DD+11 at time step & 1, generated by the Transformer Decoder, and C denotes the set of all sparse IDs.
Dense vector Loss. The dense vector prediction loss Laesse focuses on refining the dense vectors, enabling them to discern between similar and dissimilar items. The loss is defined as:
T
exp(coo(ViHt) Vi+1))
Ldanse Σ 1=1
log
(3)
Eitec exp(cos(V1+t (V.41, Vitem;))
where v, is the predicted dense vector, v, is the ground truth dense vector for the positive item, and Vitemy represents the dense vectors of items within the batch. The term cos(Vr+1 V++1) represents the cosine similarity between the predicted and ground truth dense vectors. The dense vectors are generated by an end-to-end trainable encoder denoted by Encoder, which is optimized during the training process. This ensures that the dense vectors dynamically refined and adapted to the specific requirements of the recommendation task.
Overall Loss.
The overall loss function is formulated as: L Csparse Ldease The dual-objective loss function enables a balanced optimization process, where the model dynamically refines dense vectors guided by sparse IDs. This end-to-end training approach captures both high-level semantics and feature-level information, optimizing sparse and dense representations jointly for improved performance.
3.4 Coarse-to-Fine Generation
During the inference phase, COBRA implements the coarse-to-fine fine generation procedure, involving the sequential generation of sparse IDs followed by the refinement of dense vectors in a cascaded manner, as illustrated in Figure 3 The coarse-to-fin generation process in COBRA is designed to capture both the categorical essence and grained details of user-item interactions. This process involves three main stages:
5

---

Top Retrieved Ads
BeamFusion
Generative Model
d
Beam Fusion
Final Score
Beam Search
Candidate Dense Vectors
Nearest Neighborhood
ID
9
Beam Score
NN Score
Figure 3: Illustration of the Coarse-to to-Fie Generation process. During inference, M sparse IDs generated via Beam Search, and appended to the sequence. Dense vectors are then generated and used in ANN obtain candidate items. BeamFusion combines beam scores and similarity scores rank candidates, from which the top K items are selected.
Sparse ID Generation. Given a user sequence SI.7, we utilize the ID probability distribution modeled by the Transformer Decoder, 1Dr+1 ~ ((ir+)(S::r) and employ the BeamSearch algorithm to derive the top M IDs. The formulation is as follows:
{I) T+1kk= MM BeamSearch( CaannSeerch(TTannsiin TraanforrerDeeoooeee Decoder(S1.r).. M)
where & E (1, MJ. Each generated ID is associated with a beam score & Ib
Dense Vector Refinement. Each generated sparse ID T+1 is subsequently converted into embedding and appended to the previous cascaded sequence embedding ST.T. Then the corresponding dense vector v++1 generated:
T+1 = TrausfornneeDeeccc((SS.. Embed (+1))
(5)
After that, we employ Approximate Nearest Neighbor (ANN) search retrieve the top N candidate items:
A = ANNNID T+1,
k where ((I))+1) ) is the set of candidate items associated with sparse ID ID 1D7+1. and N represents the number of top items be retrieved.
Beam Fusion Mechanism. order to achieve balance between precision and diversity, we devise globally comparable score for items corresponding to cach sparse ID. This score is capable of reflecting both the differences among different sparse IDs and the fine-grained difference among items under the same sparse ID. To accomplish this, we propose the BeamFusion mechanism:
(Vr 1D a) Softmax(Tó )x Softmax cos (++1.a) a))
where represents the candidate item, and are coefficients, and ΦID denotes the beam score obtained during the beam search process. Finally, rank all candidate items based on their BeamFusion Scores and select the top items as the final recommendations:
U क, K
R = Topk
(8)
k=1
where R denotes the set of final recommendations, and TopK(-) represents the operation of selecting the top items with the highest BeamFusion Scores. For detailed algorithmic description, please refer to the pseudocode provided in Appendix [
3.5 Theoretical Justification
our framework, each item characterized by hybrid sparse-dense representation (ID, v). where ID denotes the sparse ID and v represents the dense vector. A critical consideration is how to model the joint conditional distribution P(TD, v|5) given user sequence This distribution can be modeled in two primary ways:

---

Independent Modeling
This approach assumes that ID and v are predicted independently given
S:
V(ID.vIS)= P(ID))) P(vIS).
Cascaded Modeling (Ours) This approach, which we adopt, factorizes the joint distribution by first predicting / D, and subsequently predicting v conditioned on both ID and S:
P(ID. P(IDIS) P(DDI)) P(v|ID,S).
(10)
We posit that the cascaded formulation is theoretically superior as it explicitly captures the dependency between ID and given S. We formalize this advantage in terms of information entropy.
Theorem 1 (Superiority of Cascaded Modeling). Let Hindep and Heaseaded denote the total entropies of the probability distributions defined by the independent (Eq. 2 and cascaded (Eq. TOp formulations, respectively. Then.
Heascuded (IDeeaddeaID,W))) (ID, S Hindep (ID.v)))I
(11)
with equality holding if and only v and TD conditionally independent given S.
Theorem demonstrates that the cascaded modeling consistently produces a joint distribution with lower information entropy. This implies a more compact and informative representation that facilitates model learning. The detailed proof is provided in Appendix
Experiment
This section presents a comprehensive evaluation of the COBRA framework using both public and industrial datasets. Our experiments focus assessing COBRA's ability to improve recommendation accuracy and diversity, while also validating its practical effectiveness through offline and online evaluations.
4.1 Public Dataset Experiments
Datasets. In experiments, evaluate the performance of COBRA using the Amazon Product Reviews dataset [35] (3). Our analysis focuses on three specific subsets: "Beauty." " "Sports and Outdoors," " and "Toys and Games." For each subset, construct item embeddings leveraging attributes such as title, price, category, and description. To ensure data quality, we apply a 5-core filtering process, eliminating items with fewer than five user interactions and users with fewer than five item interactions. Detailed statistics of the datasets are presented in Appendix C..
Evaluation Metrics. For the evaluation of recommendation accuracy and ranking quality, we employ Recall@K and NDCG@K K, specifically K and K 10. These metrics provide insights into the system's s ability to accurately recommend relevant items and maintain a high-quality ranking order.
Implementation Details. In approach, adopt method for generating semantic IDs similar to the one used in [19] However, unlike [9]. which uses a different configuration, we employ a 3-level semantic ID structure, where each level corresponds codebook size of 32. These semantic IDs are generated using the T5 model. COBRA is implemented with lightweight architecture, featuring I-layer encoder and 2-layer decoder.
Baselines. To comprehensively evaluate the performance of our proposed COBRA method, we compare it with the following recommendation methods (which are described briefly in Appendix [B: P5 33 Caser 28 HGN (37, GRU4Rec (27). SASRec g FDSA 38. BERT4Rec 9. S -Rec 39
and TIGER 119.
Results. COBRA consistently surpasses all baseline models across various metrics, presented in [ On the "Beauty" dataset, COBRA achieves Recall @ of 0.0537 and Recall 10 of 0.0725, exceeding the previous strongest baseline model (TIGER) by 18.3% and 11.9%, respectively. For the "Sports and Outdoors' dataset, COBRA records Recall @5 of 0.0305 and an NDCG @5 of 0.0215, outperforming TIGER by 15.5% and 18.8% respectively. On the "Toys and Games" dataset, COBRA attains a Recall@ 10 of 0.0462 and an NDCG@ 10 of 0.0515, surpassing TIGER by 24.5% and 19.2%. %6 respectively.

---

Table Performance comparison on public datasets. The best metric for each dataset is highlighted in bold, while the second- best is underlined.
Method
R@5
N@5
@ 10
N@ 10
P5
0.0163
0.0107
0.0254
0.0136
Caser
0.0205
0,0131
0.0347
0.0176
HGN
0,0325
0,0206
0,0512
0.0266
Beauty
GRU4Rec BERT4Rec
0.0164
0.0099
0,0283
0.0137
0.0203
0.0124
0.0347
0.0170
FDSA
0.0267
0.0163
0.0407
0.0208
SASRec
0.0387
0,0249
0.0605
0.0318
S -Ree
0.0387
0.0244
0.0647
0.0327
TIGER
0.0454
0.0321
0.0648
0.0384
COBRA
0.0537
0.0395
0.0725
0.0456
P5
0.0061
0,0041
0,0095
0.0052
Caser
0.0116
0.0072
0.0194
0.0097
HGN
0.0189
0.0120
0.0313
0.0159
GRU4Rec
0.0129
0.0086
0.0204
0.0110
Sports
BERT4Rec
0.0115
0.0075
0.0191
0.0099
FDSA
0.0182
0.0122
0.0288
0.0156
SASRec
0.0233
0.0154
0.0350
0.0192
$ -e-Rec
0.0251
0.0161
0.0385
0.0204
TIGER
0.0264
0.0181
0.0400
0.0225
COBRA
0.0305
0.0215
0.0434
0.0257
P5
0.0070
0.0050
0.0121
0.0066
Caser
0,0166
0.0107
0.0270
0.0141
HGN
0.0321
0.0221
0.0497
0.0277
GRU4Rec
0.0097
0.0059
0.0176
0.0084
Toys
BERT4Rec
0.0116
0.0071
0.0203
0.0099
FDSA
0.0228
0,0140
0,0381
0.0189
SASRec
0.0463
0,0306
0,0675
0.0374
RReRec
0.0443
0.0294
0.0700
0.0376
TIGER
0.0521
0,0371
0.0712
0.0432
COBRA
0.0619
0.0462
0.0781
0.0515
Ablation Study. To validate the neces- sity of COBRA'S key components and un- derstand their individual contributions, we compare the full model against three vari- ants. COBRA w/o ID removes sparse IDs, relying solely on dense vectors. COBRA w/o Dense removes dense vectors, using only sparse IDs. COBRA w/o BeamFusion removes the BeamFusion module during
Table 2: Ablation study on public datasets (Recall @ 10).
Method
Beauty
Sports
Toys
COBRA
0.0725
0.0434
0.0781
COBRA w/o Dense
0.0656
0.0331
0.0713
COBRA w/o ID
0.0681
0.0365
0.0653
COBRA w/o BeamFusion
0,0714
0.0413
0.0769
inference, using top-1 sparse ID and nearest-neighbor retrieval for top-k results. As shown Table 2 the removal of any key component leads consistent performance drop. COBRA w/o Dense shows a significant decline, highlighting the limitations of using only discrete sparse IDs, which fail capture fine-grained semantic nuances. COBRA w/o ID also underperforms, demonstrating the importance of sparse IDs in offering a structural framework that supports coarse-to-fine generation. COBRA w/o BeamFusion also exhibits performance drop.
4.2 Industrial-scale Experiments
Dataset. To comprehensively evaluate the proposed COBRA method, we utilize a large-scale industrial dataset from major information feed platform, which contains 5 million users and million advertisements across diverse recommendation scenarios. Advertisements are represented via attributes such as title, industry labels, brand, and campaign text, encoded into two-level sparse IDs

---

Table 3: Performance comparison industrial dataset
Method
R @@
@ 100
R@200 200
R@500
R@800
COBRA
0.1180
0.1737
0.2470
0.3716
0.4466
COBRA w/o ID
0.0611
0.0964
0.1474
0.2466
0.3111
COBRA w/o Dense
0.0690
0.1032
0.1738
0.2709
0.3273
COBRA w/o BeamFusion
0.0856
0.1254
0.1732
0.2455
0.2855
and dense vectors to capture multi-granularity semantic information. A more detailed description of the dataset can be found Appendix C.2
Evaluation Metries. For offline evaluation, we employ Recall @ K as the evaluation metric, testing with K E {50, 100, 200, 500, 800). This metric provides a measure of the model's 's ability to accurately retrieve relevant recommendations at various thresholds.
Implementation Details. COBRA is built upon Transformer- architecture. In this framework, the text encoder processes advertisement text into sequences, which are then handled by the sparse ID head to predict 2-Ievel semantic IDs configured as 32 x 32.
Results. For further analysis on the industrial dataset, we also compare COBRA against the variants defined in the previous section, i.e.. COBRA w/o ID. w/o Dense, and w/o Beam- Fusion. Notably, the COBRA w/o Dense vari- ant employs iner-grained 3-level semantic IDs (256 x 256 x 256) to ensure its sufficient fine- grained modeling capacity, compensating for its lack of dense vectors. As shown in Table COBRA consistently outperforms all its vari- ants across all evaluated metries. At K 500, COBRA achieves @ecall@ 500 of 0.3716, rep- resenting 42.2% improvement over the CO- BRA w/o Dense variant. When K = 800, CO- BRA attains Recall 800 of 0.4466, reflecting a 43.6% improvement over the COBRA w/o ID variant and a 36.1 1 % enhancement compared to COBRA w/o BeamFusion. At relatively smaller values of K, the absence of dense or ID representations results in more pronounced performance declines, underscoring the importance of cascaded representations for achieving granularity and precision. Conversely, as the recall size K increases, the performance advantages associated with BeamFusion become increasingly evident, demonstrating its effectiveness in practical industrial recall systems. The results further underscore the contributions of specific components: Excluding sparse IDs leads to a recall reduction ranging from 26.7% to 41.5%, highlighting the critical role of semantic categorization. The removal of dense vector results in performance drop between 30.3% and 48.3%, underscoring the importance of fine-graing fine-grained modeling. Eliminating BeamFusion results in a recall decrease of 27.5% to 36.1% ,, emphasizing the significance of fusion strategy.
CCS SETI Enfamers COBRA 1th
66:00
4225
1208
-100
Figure 4: Cosine Similarity Matrix (full compari-
son in Appendix 6
4.3 Further Analysis
Analysis of Representation Learning. The heatmap in Figure Ademonttrates COBRA's strong intra- ID cohesion and inter-ID separation, indicating effective capture of both item-specific features and categorical semantics. Quantitative verification through difference analysis is provided in Appendix D Further validation of COBRA's embeddings is achieved through visualizing the distribution of advertisement embeddings in a two-dimensional space using t-SNE. By randomly sampling 10,000 advertisements, distinct clustering centers for various categories are observed. Figure revells that advertisements are effectively clustered by category, indicating strong cohesion within categories. The clusters in purple, teal, light green, and dark green correspond primarily to advertisements for novels, games, legal services, and clothing, respectively. This demonstrates that the advertisement representations effectively capture semantic information.

---

brencty
Moss
40
35
30
Diversit
0.57
0.96
10
0.6
0.8
1.0
1.2
1.4
(a) Embedding Visualization using t-SNE
(b) Recall-Diversity Trade-off
Figure (a) t-SNE visualization of 10,000 randomly sampled advertisement embeddings. (b) Recall @ 2000 and diversity metrics under different values.
Recall-Diversity Equilibrium. analyze the trade-off between accuracy and diversity in COBRA, we examine recall-diversity curves, which depict how Recall 2000 and diversity metrics evolve with the coefficient in the BeamFusion mechanism, while keeping & fixed. depicted in Figure 5b increasing т generally leads to decrease in diversity. COBRA achieves optimal balance between 0.9. this point, the model maintains high accuracy while ensuring sufficiently diverse set of retrieved items. Specifically, when M 50, compared to 1.0, setting т 0.5 results in 4. 4.99% decrease in recall, but brings more than double the diversity. Meanwhile, 0.9 leads to 0.12% increase in recall and an 18.80% relative improvement in diversity. This fine-gres theeggraied control over 7 and ó allows for adjusting the emphasis on accuracy or diversity according to specific business objectives. Platforms prioritizing exploration reduce to enhance diversity. This flexibility distinguishes COBRA from models with fixed retrieval strategies, making it adaptable various recommendation scenarios.
recall and diversity
Online Results. To validate COBRA's real-world effectiveness, we conducted online A/B tests major information feed platform. The experiment covered 10% of user traffic, ensuring statistical significance. The primary evaluation metrics (ARPU), which directly reflect user engagement and economic value. Within the traffic segment exposed our proposed strategy, COBRA achieved a 3.60% increase in conversion and 4.15% increase in ARPU. These results demonstrate that COBRA's hybrid architecture not only improves recommendation quality in offline evaluations but also drives measurable business outcomes in production environments.
conversion and Average Revenue Per User
5 Conclusion
In this work, we introduced COBRA, generative recommendation framework that combines sparse and dense representations for enhanced accuracy and diversity. COBRA employs coarse-to- generation process, starting with sparse to capture the categorical essence of item and refining with dense vector. Our extensive experiments that COBRA achieves superior performance over state-of-the-art methods, delivering high accuracy with controllable diversity. These gains are further validated through online A/B tests, confirming method's s practical applicability. In the future, we intend to incorporate more multi-domain and multi-modal information to further enhance framework's effectiveness. Additionally, will explore performance optimizations in the generative approach to improve its efficiency and scalability.
public and industrial datasets demonstrate
References
[1] H. Guo, R. Tang. Y. Ye, Z. Li, and X. He, "Deepfim: factorization- machine based neural network for ctr prediction," arXiv preprint ar0iv:1703. 170::1703.04247, 2017.
10

---

[2] H.-T. Cheng, L. Koc, J. Harmsen. T. Shaked, T. Chandra, H. Aradhye, G. Anderson, G. Corrado, W. Chai, M. Ispir el al., "Wide deep learning for recommender systems," in Proceedings the 1st workshop on deep learning for recommender systems, 2016, pp. 7-10.
[3] G. Penha, A. Vardashi, F. Palumbo, M. De Nadai, and H. Bouchard, "Bridging search and recommendation in generative retrieval: Does one task help the other?" in Proceedings of the 18th ACM Conference on Recommender Systems, 2024, pp. 340-349.
[4] W. Song, C. Shi, Z. Xiao, Z. Duan, Y. Xu, M. Zhang, and J. Tang. "Autoint: Automatic feature interaction learning via self attentive neural networks." Proceedings of the 28th ACM international conference on information and knowledge management, 2019, pp. 1161-1170.
[5] Q Chen, Zhao, W. Li, P. Huang, and W. Ou, "Behavior sequence transformer for e-commerce recom- mendation in alibaba," in Proceedings of the 1st international workshop on deep learning practice for high--immnnsoonl sparse data, 2019, pp. 1-4.
[6] G. Zhang. Y. Hou, H. Lu, Y. Chen, W. X. Zhao, and J.-R. Wen, "Scaling law of large sequential recom- mendation models," in Proceedings of the 18th ACM Conference on Recommender Systems, 2024, pp. 453-
[7] H. Fan, M. Zhu, Hu, H. Feng, Z. He, H. Liu, and Q. Liu, "Tim4rec: An efficient sequential recommen- dation model hased on time-aware structured state space duality model," arXiv preprint arXiv:2409. 16182, 2024.
[8] W.-. Kang and J. McAuley, "Self-attentive sequential recommendation," " in 2018 IEEE international conference on data mining (ICDM). IEEE, 2018, pp. 197-206.
[9] F. Sun, J. Liu, J. Wu, C. Pei, X. Lin, W. Ou, and P. Jiang, "Bert4rec: Sequential recommendation with bidirectional encoder representations from transformer," in Proceedings of the 28th ACM international conference information and knowledge management, 2019, pp. 1441 -1400.
[10] J. Zhai, L. Liao, X. Liu, Y. Wang, R. Li, X. Cao, L. Gao, Z. Gong, F. Gu, M. He et al., "Actions speak louder than words: Trillion-parann parameter sequential transducers for generative recommendations! arXiv preprint 24Xiv:240..171522 2024.
[11] S. Xu, W. Hua, and Y. Zhang, "Openp5: An source platform for developing, training, and evaluating lim-based recommender systems," in Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2024, pp. 386-394.
[12] J. Chen, L. Chi, B. Peng, and Z. Yuan, "Hllm: Enhancing sequential recommendations via hierarchical large language models for item and user modeling," arXiv preprint arXiv:2409.122 12740, 2024.
[13] Y. Deldjoo, Z. Нe, J. McAuley, A. Korikov, S. Sanner. A. Ramisa, R. Vidal, M. Sathiamoorthy, A. Kasirzadch, and S. Milano, "A review of modern recommender systems using generative models (gen-recsys)"" in Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2024, pp. 6448 6458.
[14] J. Zhu, M. Jin, Q. Liu, Z. Qiu, Z. Dong, and X. Li, "Cost: Contrastive quantization based semantic tokenization for generative recommendation"" in Pruceedings of the ACM Conference on Recommender Systems, 2024, pp. 969-974.
[15] A. Singh, T. Vu, N. Mehta, R. Keshavan, Sathiamoorthy, Y. Zheng. L. Hong. L. Heldt, L. Wei. D. Tandon et al., "Better generalization with semantic ids: A case study in ranking for recommendations" in Proceedings of the 18th ACM Conference on Recommender Systems, 2024, pp- 1039-1044.
[16] Chen, C. Gao, S. Yuan, Liu, Q. Cai, and Jiang. "Dlcrec: A novel approach for managing diversity in llm-based recommender systems," arXiv preprint arXiv:2408. /2470, 2024.
[17] Z. Wan, B. Yin. J. Xie, F. Jiang, X. Li. and W. Lin, "Larr: Large language model aided real-time scene recommendation with semantic understanding." " in Proceedings of the 18th ACM Conference on Recommender Systems, 2024, pp. 23-32.
[18] S. Wang, B. Xie, I.. Ding, X. Gao, J. Chen, and Y. Xiang, "Secor: Aligning semantic and collaborative representations by large language models for next-point-of-interee recommendations, in Proceedings of the 18th ACM Conference on Recommender Systems, 2024, pp. 1-11.
[19] S. Rajput, N. Mehta, A. Singh, R. Hulikal Keshavan, T. Vu, L. Heldt, L. Hong. Y. Tay, V. Tran, J. Samost et al., "Recommender systems with generative retrieval," Advances in Neural Information Processing Systems, vol. 36, pp. 10 1199-10315,,20 2023. 2023.

---

[20] D. Lee. C. Kim, S. Kim. M. Cho. and W.S. Han, "Autoregressive image generation using residual quantization." " in Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, 2022. pp. 11 523-11 532.
[21] B. Zheng. Y. Hou, H. Lu, Y. Chen, W. X. Zhao, M. Chen, and J.-R. Wen, "Adapting large language models by integrating collaborative semantics for recommendation," in 2024 IEFE 40th International Conference on Data Engineering (ICDE). IEEE, 2024, pp. 1435-1445,
[22] Y. Wang. Z. Ren, W. Sun, J. Yang, Z. Liang, X. Chen, R. Xie, S. Yan, X. Zhang, P. Ren et al., "Content- based collaborative generation for recommender systems," in Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, 2024, pp. 2420-2430.
[23] J. Tan, S. Xu, W. Hua, Y. Ge, Z. Li, and Y. Zhang, "Idgenrec: Lem-recsys alignment with textual id learning"" in Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2024, pp. 355-364.
[24] Y. Li, X. Lin, W. Wang, Feng, L. Pang. W. Li, L. Nie, X. He, and T.-S. Chua, "A survey of generative search and recommendation in the era large language models," arXiv preprint arXiv:2404.16924 2024.
[25] Y. Deldjoo, Z. Hc, J. McAuley, A. Korikov, S. Sanner, A. Ramisa, R. Vidal, M. Sathiamoorthy, A. Kas- rizadch, S. Milano et al., "Recommendation with generative models," arXiv preprint arXiv:2409.15173, 2024.
[26] L. Yang. F. Paischer, K. Hassani, J. Li, S. Shao, Z. G. Li, Y. He, X. Feng, N. Noorshams, S. Park et al., "Unifying generative and dense retrieval for sequential recommendation," arXiv preprint arXiv:2411. 188814, 2024.
[27] B. Hidasi, "Session-based recommendations with recurrent neural networks," arXiv preprint arXiv: 1.69999999 2015.
[28] J. Tang and K. Wang. "Personalized top-n sequential recommendation via convolutional sequence embed- ding," " Proceedings of the eleventh ACM international conference on web search and data mining, 2018, pp. 565-573.
[29] Li, M. Wang, L. Fu, X. Shen, J. Shang, and J. McAuley, "Text is all you need: Learning language representations for sequential recommendation," in Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2023, pp. 1258-1267.
[30] H. Ding, Y. Ma, A. Deoras, Y. Wang, and H. Wang, "Zero-shot recommender systems," " arXiv preprint arXiv:2105.0 08318, 2021.
[31] Y. Hou, S. Mu, W. X. Zhao, Y. B. Ding, J.-R learning for recommender systems," in Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2022. pp. 585-593.
J.-R. Wen, "Towards universal sequence representation
[32] J. Ji, Z. Li, S. Xu, W. Hua. Y. Ge. J. Tan, and Y. Zhang, "Genree: Large language model for generative recommendation." in European Conference on Information Retrieval. Springer, 2024, pp. 494-502.
[33] S. Geng. S. Liu, Z. Fu, Y. Ge, and Y. Zhang, "Recommendation language processing (rlp): A unified pretrain, personalized prompt & predict paradigm (p5)." in Proceedings of the 16th ACM Conference on Recommender Systems, 2022, pp. 299-315.
[34] Z. Si, Z. Sun, J. Chen, G. Chen, X. Zang, K. Zheng, Y. Song, X. Zhang, J. Xu, and K. Gai, "Generative retrieval with semantic tree-structured identifiers and contrastive learning." in Proceedings of the 2024 Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pucific Region, 2024, pp. 164-163.
[35] R. He and J. McAuley. "Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering." " proceedings of the 25th international conference on world wide web, 2016, pp. 507-517.
[36] J. McAuley, C. Targett, Q. Shi, and A. Van Den Hengel, "Image-based recommendations on styles and substitutes," in Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval, 2015, pp. 43-52.
[37] C. Ma, P. Kang, and X. Liu, "Hierarchical gating networks for sequential recommendation." in Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining. 2019, pp. 825-833.
12

---

[38] T. Zhang, P. Zhao, Y. Liu, V. S. Sheng. J. Xu, D. Wang, G. Liu, X. Zhou et al., "Feature-level deeper self-attention network for sequential recommendation. " in IJCII, 2019, pp. 4320-4326
[39] K. Zhou, H. Wang. W. X. Zhao, Y. Zhu, S. Wang, F. Zhang, Z. Wang, and J.-R. Wen, "S3-rec: Self- supervised learning for sequential recommendation with mutual information maximization." in Proceedings of the 29th ACM international conference on information knowledge management, 2020, pp. 1802-1902.
[40] V. A. Tran, G. Salha-Galvan, B. Sguerra, and R. Hennequin, "Attention mixtures for time-aware sequential recommendation." " in Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2023, pp. 1821-1826
[41] R. Qiu. Z. Huang, H. Yin, and Wang. "Contrastive learning for representation degeneration problem sequential recommendation." in Proceedings of the fifteenth ACM international conference on web search and data mining, 2022, pp. 813-823.
[42] Z. Fan, Z. Liu, Y. Wang. A. Wang, Z. Nazari, L. Zheng, H. Peng, and P. S. Yu. "Sequential recommendation via stochastic self-attention." in Proceedings of the ACM web conference 2022, 2022, pp.
[43] M. Li, Z. Zhang, X. Zhao, W. Wang, M. Zhao, R. Wu, and R. Guo, "Automlp: Automated mlp for sequential recommendations" in Proceedings of the ACM web conference 2023, 2023, 1190- 1198.
[44] W. Zaremba, "Recurrent neural network regularization," arXiv preprint arXiv: 20l.:39999222 2014.
[45] L. Zheng, V. Noroozi, and P. Yu, "Joint deep modeling of users items using reviews recommen- dation," in Proceedings of the tenth ACM international conference on web search and data mining, 2017,
pp. 2344..4.
[46] N. Pancha, A. Zhai, J. Leskovec, and C. Rosenberg, "Pinnerformer: Sequence modeling for user represen- tation at pinterest," in Proceedingy of the 28th ACM SIGKDD conference on knowledge discovery and data mining, 2022, pp. 3702-3712
[47] Z. Liu, Hou. and J. McAuley, "Multi-behavior generative recommendation," in Proceedings of the 33rd ACM International Conference Information and Knowledge Management, 2024, pp. 1575-1585.
[48] Y. Li, X. Zhai, M. Alzantot, K. Yu, I. Vulić, A. Korhonen, and M. Hammad, "Calrec: Contrastive alignment of generative Ilms for sequential recommendation." in Proceedings the 18th ACM Conference on Recommender Systems, 2024, pp. 422 432.
[49] Y. Wang, J. Xun, M. Hong. J. Zhu, Jin, W. Lin, H. Li, Xia, Z. Zhao et al., "Eager: Two-stream generative recommender with behavior-semanne collaboration," in Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2024, pp. 3245-3254.
[50] K. Bao, J. Zhang, W. Wang. Y. Zhang, Yang. Y. Luo, C. Chen. Feng, and Q. Tian, "A bi-step grounding paradigm for large language models in recommendation systems," ACM Transactions on Recommender Systems, 2023.
[51] X. Lin, W. Wang, Y. Li, F. Feng. S.-K. Ng. and T.S. Chua, "A multi-face paradigm to bridge large language model and recommendation," arXiv preprint 2rXiv:2310.06499 vol. 3, 2023.
[52] H. Wang, J. Lin, X. Li, B. Chen, C. Zhu, R. Tang, W. Zhang, and Y. Yu, "Flip: Fine- alignment between id-based models and pretrained language models for etr prediction," in Proceedings of the 18th ACM Conference Recommender Systems, 2024, pp. 94-104.
[53] S. Kim, H. Kang. K. Kim, J. Kim, D. Kim, M. Yang. K. Oh, J. McAuley, and C. Park, "Lost in sequence: Do large language models understand sequential recommendation"" arXiv preprint arXiv:2502. 2025.
[54] T. M. Cover, Elements of information theory. John Wiley & Sons, 1999.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The paper clearly demonstrates the performance improvement of the method on recommendation tasks, and the experimental section provides ample support for these claims. Therefore, the main assertions in the abstract and introduction accurately reflect the contributions and scope of the paper.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: our paper, we mention that our proposed method currently only handles text modality, which implicitly acknowledges a limitation of our current work.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-tex system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best
14

---

judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: All relatively complex mathematical formulas are accompanied by detailed derivations and transformations.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: The paper provides detailed description of each stage of the experiment, including the source and preprocessing of the dataset and the model's parameter settings. This information allows other researchers to replicate the experiment and achieve similar results.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
15

---

(a) If the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g.. a large language model), then there should either be way to access this model for reproducing the results way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors welcome deseribe the particular way they provide for reproducibility. the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Due the submission requirements of the commercial company, unable to include the code with our submission. However, we provide complete and detailed parameter settings and pseudocode for by other researchers.
Justification:
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co ec/
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed run reproduce the results. See the NeurlPS code and data submission guidelines (https btttsss.cc/pulligguudeee CoodeSubmm wttpsstmmmssssipPee.eeeeoooo Psioniolicy for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question:
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
The paper provides detailed description of the dataset used in the experiments, the data preprocessing methods, and the hyperparameter settings. These details help readers understand the process by which the experimental results were generated.
Justification:
16

---

Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: Due to the significant differences observed in the experimental results of our study, we believe that conducting statistical significance tests this context less necessary. However, we will consider adding them in future work. We have ensured the rigor the experimental process and the reliability of the results by validating the stability of the model's performance through multiple experiments. Additionally, the comparison with other methods provides valuable insights.
Guidelines:
The answer NA means that the paper does include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (c.g. negative error rates).
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [No]
The focus of this study is on theoretical innovation and methodological exploration of the algorithm, rather than specific engineering implementation and resource optimization. We are primarily concerned with the structural design of the model and the logical flow of the algorithm. At this stage, believe that the innovativeness and effectiveness of the algorithm are the more critical factors to consider.
Justification:
Guidelines:
17

---

The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported paper (e.g., preliminary or failed experiments that didn't It make it into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics http:///wwrr..ccooooo/oc/ccaccaa
Answer: [Yes]
Justification: The authors strictly adhered to the NeurIPS Code of Ethics throughout their research process.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: There is no societal impact of the work performed.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
18

---

11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
The paper includes detailed annotations for the code, datasets, and other assets used.
Justification:
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, patnesswpppssiiii theode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information not available online, the authors are encouraged to reach out to the asset's is creators.
13. New assets
Question:
Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
19

---

Justification:
The paper provides detailed documentation for the newly created dataset and model, including the data collection and preprocessing methods, the model architecture, and the training process.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
Our research did not involve human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
20

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: LLM is not the central component of our method; our core contribution is generative framework.
Guidelines:
The answer NA means that the method development in this research does not involve L.MSs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should be described.
21

---

A Related Work
Sequential Dense Recommendation. Sequential dense recommendation utilize user interaction histories to learn representations for personalization [8] 39 29], capturing both long-term and short- term preferences [40 GRU4Rec [27) first applied RNNs [44] to model sequential behavior, effectively handling temporal dependencies. Caser [28] adopted CNNs [45] extract local sequential patterns from behavior matrices. Tramsformer-bases models, such SASRec (8] and BERT4Rec [9), leverage self-attention to model complex user behavior, with SASRec focusing on autoregressive predictions and BERT4Rec adopting bidirectional context encoding. Advanced architectures like PinnerFormer [46 and FDSA (38] further enhance user modeling by integrating multi-source fea- tures and capturing long-range dependencies. Recent efforts (e.g., ZESRec [30], UniSRec [3I], RecFormer (2)) have moved towards cross-domain recommendation and richer feature integration,
often via contrastive learning and unified language-sequence modeling.
Generative Recommendation. With the emergence of large generative models, recommendation is increasingly framed as a generation task B2] 47451]. P5 [3] reformulates various recommendation tasks language generation, enabling unified modeling and prompting strategies. TIGER [19] applies residual quantized autoencoders to produce semantic item identifiers, which transformers then generate from user sequences. LC-Rec [1] aligns these semantic IDs with collaborative signals, while IDGenRec [23] leverages large language models for unique, dense textual identifiers. SEATER (34] and ColaRec [22] focus on aligning semantic and collaborative spaces maintaining semantic consistency via structured indexes. Despite their advantages, generative models relying on discrete IDs may lose fine-grained preference information [52], and natural language generation may be less aligned with recommendation-sppenne generate sparse IDs and dense vectors, narrowing the gap between retrieval paradigms. Nevertheless. challenges remain in achieving optimal flexibility and representation granularity.
signals [53] Hybrid approaches like LIGER (26) jointly
B Baselines
To rigorously evaluate the effectiveness of our proposed COBRA method, we benchmark against range of recommendation methods.
P5 This innovative approach transforms recommendation tasks into natural language se- quences, utilizing the capabilities of language models to unify various recommendation scenarios.
Caser [28): Caser employs convolutional layers to effectively capture sequential user behavior patterns. modeling high-order Markov Chains.
The Hierarchical Gating Network(HGN) is designed to capture both long-term and short-term user interests through a sophisticated gating architecture, facilitating personalized recommendations.
HGN 37
As pioneering RNN-based approach, GRU4Rec leverages gated recurrent units to model user behaviors sequential recommendation tasks, paving the way for subsequent developments in the field.
GRU4Rec 27::
SASRec 8 : As Transformer-based model, SASRec focuses long-term dependencies in user interactions, employing self-attention mechanisms for precise sequential recommendations.
FDSA 38
By integrating item features with embeddings, the Feature-level Deeper Self- AttentionFDDSA) Network enriches the input sequence, leveraging self self-attentive mechanisms to enhance recommendation quality.
BERT4Rec g Utilizing a bidirectional self-attention framework with cloze task objective, BERT overcomes the limitations of traditional uni-directional models, offering robust recommendation capabilities.
Ree :9]: S² -Rec leverages contrastive learning to bolster the recommendation process, employing self-supervised techniques to enhance sequential recommendation performance.
TIGER 19
TIGER employs RQ-VAE for encoding item content features and leverages Transformer for generative retrieval, showcasing novel approach to incorporating content features in recommendation tasks.
22

---

These methods collectively represent the forefront of recommendation technology. embodying diverse methodologies from sequential and dense recommendation generative approaches.
C Dataset Statistics
C.1 Public Datasets.
Table 4: Summary of dataset statistics for three real-world benchmarks.
Dataset
Users
Items
Avg. Length
Med. Length
Beauty
22,363
12,101
8.87
6
Sports and Outdoors
35,598
18,357
8.32
6
Toys and Games
19,412
11,924
8.63
which spans user reviews and product information from May 1996 to July 2014. To comprehensively explore the effectiveness of recommendation methods, selected three categories: "Beauty." "Sports and Outdoors," and "Toys and Games." " Table provides a concise summary of these datasets. During data preprocessing, we constructed users' historical item interaction sequences based on review timestamps, excluding users with fewer than five reviews. For evaluation, we adopted the widely-used leave-one-out strategy: the last item in each user's sequence served the test sample, the second-to-lan as the validation sample, and the remaining items as training data. The "Beauty" dataset contains 22,363 users and 12,101 items, featuring an average sequence length of approximately 8.87, with median of 6. The "Sports and Outdoors" dataset comprises 35,598 users and 18,357 items, with an average sequence length of 8.32 and median of 6. Similarly, the "Toys and Games" dataset includes 19,412 users and 11,924 items, with an average sequence length of about 8.63 and median of 6.
Our study utilizes the Amazon Product Reviews dataset 335 36
C.2 Industrial Dataset Details.
To thoroughly evaluate the proposed COBRA method, we employ large-scale industrial dataset derived from user interaction logs on major information feed platform. This dataset covers multiple recommendation scenarios, including list-page, dual-column, and short-video recommendations, and contains approximately 5 million users and 2 million advertisements, providing a comprehensive reflection of real-word woal-world user behaviors and ad advertising content.
Advertisers and advertisements described by attributes such as title, industry labels, brand, and campaign text. To effectively capture both coarse- coarse-grained grained and fine- ine-grained semantic information, these attributes encoded into two-level sparse IDs alongside dense vector representations. This dual encoding enables COBRA to model user preferences and item characteristics more accurately.
The dataset is divided into two parts: the training set D Durain and the test set D The training consists of user interaction logs collected over the first 60 days, covering recommendation content and user behaviors during this period. The test set is constructed from logs recorded on the day immediately following the training period and serves as a benchmark for model performance evaluation. This chronological split ensures the temporal consistency of training and testing processes, improving the reliability of the evaluation.
D Supplementary Similarity Analysis
The COBRA model exhibits significant intra-ID cohesion and inter-ID separation, as demonstrated in the top heatmap of Figure 4 This suggests that COBRA's dense embeddings proficiently capture detailed item characteristies while preserving semantie consistency within categories. Conversely, the model variant without sparse IDs (Figure ba) shows weaker category separation, underscoring the importance of sparse IDs in maintaining semantic structure. The difference matrix in Figure 6b) quantitatively confirms that incorporating sparse IDs enhances both cohesion and separation.
23

---

6thh bitween COBAA
EITS betweeee COIRA a5s
200
-333
430
-3.6
Without ID (COBRA w/o ID)
(b) Difference Matrix
Figure 6: Complete similarity matrix comparison: (a) Weaker separation without ID, (b) Quantitative
improvement from sparse IDs
E Pseudocode for Coarse-to-Fine Generation
Algorithm Inference with BeamFusion
1: Input: input_seq SI:T, beam_ size M, nn_num N, recall _um K, candidate items
2: Output: Top K recommendations R
3:
4: procedure FORWARD(--.)
5: Sparse ID Generation:
6: for each ID hierarchy do
7:
Run transformer forward pass
8:
Compute ID logits and scores (ID T+1 MM
9:
Update beam scores and decoder input
end for
11:
Dense Vector Refinement:
Obtain final decoder output v
13:
Compute similarity scores cos( v.l. 1, a)
14:
Filter logits using generated ID
15:
BeamFusion Mechanism:
16:
Combine beam scores ID with similarity scores cos( TT11 a)
17:
Select top K candidates based on fused scores
Prepare Results:
19:
Collect final ID sequences, retrieved items, and scores
end procedure
F Proof of Theorem 1
We calculate the information entropy (for discrete variable ID) and differential entropy (for continu- ous variable v) under both probability distributions given by Eq, and Eq. 10 We denote the discrete entropy by H-)) and the differential entropy by h(-).
Independent Modeling (Eq.P) 9)
Hindep
EPinde [lgg P(ID, v())
EPPinee Dog (P(()))))) P(v[S))]
--EPinde [log P(ID[S)] Err,e [log P(v
HH(D)S)) + h(v
(12)
Hence, under independent modeling, the total entropy decomposes additively ID and v.

---

Cascaded Modeling (Eq. 10::
H eascaaded
Ep Poossadee oddded log P(TD.v|S)
-BP Poosedded log(P((D)S) P(v
--Epascaded log P(ID[S)] Pareeedd log P(v
H +(v
(13)
Hence, cascaded modeling captures the full joint uncertainty between ID and v.
Comparison: By the property of conditional entropy 54
h(v[S) h(v
(14)
Equality holds if and only if v and ID are conditionally independent given S.
Substituting this inequality into Eq. 12 and 13 yields:
H Hindep Hoascaded
(15)
which confirms Theorem
G Computational Cost Analysis
We define as the sequence length, T as the number of tokens per item and D as the embedding dimension.
Table 5: Training Complexity. (Please add citations for TIGER/SASRec)
Component
COBRA
TIGER 19
SASRec
Item Feature Encoder
O + I T..TDD) O(L2 D + L- D²)
N/A
N/A
Sequential Model
O D D2)
O D+ L D2)
Table 6: Inference Complexity. (Please add citations for TIGER/SASRec)
Component
COBRA
TIGER [19
SASRec 8
Item Feature Encoder
N/A O L. D2) O D + L. D2) O D + L- D2)
((1) (cached)
N/A
Sequential Model
To ensure practical efficiency, employ several key techniques during implementation. These include sequence Packing and FlashAttention to minimize computational waste. Furthermore, encoder caching is utilized to decouple the encoder computation, significantly speeding up the inference process. Thanks to these optimizations, COBRA achieves over 30% Model FLOPs Utilization (MFU), and has been successfully deployed, serving over 200 million daily users.
25


Reviews:
[{"review_id": "K8wVAzRtgz", "reviewer": "Reviewer_XgPc", "summary": "This paper proposes COBRA, a new model which integrates generative and dense retrieval method by formulating sparse ID generation and dense vector prediction through a cascading manner in sequential recommendation. COBRA is trained via optimizing both the CE loss of next token prediction for sparse IDs and the CE loss with in-batch negatives for dense vectors. During inference, COBRA leverages beam search to generate several distinct IDs and dense vectors, builds the candidate item set by combining items corresponding to these IDs and items retrieved by dense vectors through MIPS, and the final result is generated by selecting top-K items in the candidate set via a non-linear combination of beam search scores and cosine similarity scores. Experiments on public and private datasets demonstrate the effectiveness of COBRA.", "questions": "1. Experiment results do not verify the necessity and importance of the cascade bi-represented formulation in COBRA, which raises doubts about the novelty of the paper.  It seems the performance improvement comes from ensembling generative retrieval and dense retrieval methods. A natural question is, what is the performance of combining retrieved results of existing GR and DR methods through BeamFusion or other simple fusion strategy like linear combining beam search and NN scores?\n2. As COBRA leverages multiple dense embeddings for NN search, current dense retrieval methods with multiple embeddings should also considered as baseline for fair comparison. Examples includes MIND[1] and ComiRec[2].\n3. What is the ablation study result in the public dataset?\n4. What is the value of $M$ corresponding to Table 1&2? \n\n[1] Multi-Interest Network with Dynamic Routing for Recommendation at Tmall\n[2] Controllable Multi-Interest Framework for Recommendation", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strength:\n1. Writing is good and the paper is easy to follow\n2. Experimental results seems impressive\n\nWeakness:\n1. The evaluation of major contributions (cascade bi-represented formulation, coarse-to-fine generation) is not thorough. \n2. Lack of analysis of time complexity of training and inference, which is important for retrieval models.", "was_revised": true, "final_justification": "The authors' response has addressed most of my concerns and I raise my rating accordingly.", "rebuttal": "We sincerely appreciate your constructive feedback and insightful comments on our paper. Thank you for acknowledging the clarity of our presentation and the strength of our experimental results. We have thoroughly addressed your concerns regarding contribution evaluation, baseline comparisons, time complexity, and implementation settings. Below is our response incorporating additional analyses and experiments.\n\n---\n\n### **For Weakness 1 & Question 1: Enhanced Evaluation of Core Contributions**\n\nTo rigorously validate the necessity of our cascaded bi-represented formulation—which extends beyond a simple combination of generative retrieval (GR) and dense retrieval (DR)—we conducted comprehensive experiments on public datasets. Our experimental design compares two variants:\n\n| Variant                  | Architecture                                       |\n|--------------------------|---------------------------------------------------|\n| **Ensembling-Linear**    | Combining GR and DR using linear fusion strategy  |\n| **Ensembling-BeamFusion**| Combining GR and DR using BeamFusion              |\n| **COBRA (Full)**         | Complete cascaded architecture                    |\n\nRecall@10 performance across public datasets:\n\n| Method                | Beauty   | Sports & Outdoors | Toys & Games |\n|-----------------------|----------|-------------------|--------------|\n| Ensembling-Linear     | 0.0670   | 0.0379            | 0.0720       |\n| Ensembling-BeamFusion| 0.0664   | 0.0380            | 0.0722       |\n| **COBRA (Full)**     | **0.0725** | **0.0424**        | **0.0781**   |\n  \nCOBRA outperforms ensembling approaches in Recall@10 metrics. For example:\n- On *Beauty*: Ensembling-BeamFusion (0.0664) vs. COBRA (0.0725, +$9.2\\\\%$)\n- On *Sports & Outdoors*: Performance gap of $10.6\\\\%$ between Ensembling-Linear and COBRA\n\nThis demonstrates that cascaded generation—where sparse IDs explicitly condition dense vector refinement—is fundamentally more effective than ensembling separate GR and DR methods. The joint training ensures dense vectors are semantically anchored to sparse IDs, avoiding ambiguous retrievals.\n\n---\n\n### **For Weakness 2: Time Complexity Analysis**\n\nWe appreciate this important concern. For our complexity analysis, we define $L$ as the sequence length, $T$ as the number of tokens per item, and $D$ as the embedding dimension.\n\n**Training Complexity**\n| Component | COBRA | TIGER[1] (GR) | SASRec[2] (DR) |\n| :---------------------- | :------------------------------------------- | :--------------------------- | :--------------------------- |\n| Item Feature Encoder | $\\mathcal{O}(L \\cdot T^2 \\cdot D + L \\cdot T \\cdot D^2)$ | N/A | N/A |\n| Sequential Model | $\\mathcal{O}(L^2 \\cdot D + L \\cdot D^2)$ | $\\mathcal{O}(L^2 \\cdot D + L \\cdot D^2)$ | $\\mathcal{O}(L^2 \\cdot D + L \\cdot D^2)$ |\n\n**Inference Complexity**\n| Component               | COBRA                     | TIGER                     | SASRec                    |\n|-------------------------|---------------------------|---------------------------|---------------------------|\n| Item Feature Encoder    | $\\mathcal{O}(1)$ (cached) | N/A                       | N/A                       |\n| Sequential Model        | $\\mathcal{O}(L^2 \\cdot D + L \\cdot D^2)$ | $\\mathcal{O}(L^2 \\cdot D + L \\cdot D^2)$| $\\mathcal{O}(L^2 \\cdot D + L \\cdot D^2)$|\n\nTo ensure practical efficiency, we employ several key techniques during implementation. These include sequence Packing  and FlashAttention to minimize computational waste. Furthermore, caching is utilized to decouple the encoder computation, significantly speeding up the inference process. Thanks to these optimizations, COBRA achieves over 30% Model FLOPs Utilization, surpassing typical recommendation benchmarks[3]. This efficiency has enabled successful deployment, serving over 200 million daily users.\n\n---\n\n### **For Question 2: Comparison with Multi-Interest Retrieval Methods**\n\nWe appreciate your suggestion to include multi-interest retrieval baselines. To provide a more comprehensive evaluation beyond sequential paradigms (TIGER for GR, SASRec for DR), we conducted experiments comparing COBRA against state-of-the-art multi-interest methods. Following literature findings that ComiRec[2] generally outperforms MIND[1], we focus our comparison on ComiRec:\n\n| Method   | Recall@5 | Recall@10 | NDCG@5 | NDCG@10 |\n|----------|----------|-----------|--------|---------|\n| ComiRec  | 0.0205   | 0.0445    | 0.0105 | 0.0183  |\n| COBRA    | **0.0537** | **0.0725** | **0.0395** | **0.0456** |\n\n**Key Observations**\n1. Consistent Superiority:  \n   COBRA demonstrates significant performance advantages over ComiRec across all metrics, with particularly notable gains in early-position ranking (Recall@5 and NDCG@5).\n\n2. Architectural Distinction:  \n   While ComiRec generates multiple interest embeddings per user, COBRA's cascaded paradigm fundamentally differs by:\n   - Explicitly conditioning dense vector generation on sparse IDs\n   - Enabling mutual refinement of both representations through end-to-end training\n\n---\n\n### **For Question 3: Ablation Studies on Public Datasets**\n\nWe initially conducted ablation studies primarily on industrial datasets due to their direct relevance to our business objectives. To address the reviewer's concern about generalizability, we have now performed additional ablation analyses on the \"Beauty,\" \"Sports & Outdoors,\" and \"Toys & Games\" public datasets. The Recall@10 results are summarized below:\n\n| Method | Beauty | Sports & Outdoors | Toys & Games |\n| :---------------------- | :------- | :---------------- | :------------- |\n| COBRA w/o Dense | 0.0656 | 0.0331 | 0.0713 |\n| COBRA w/o ID | 0.0681 | 0.0365 | 0.0653 |\n| COBRA w/o BeamFusion | 0.0714 | 0.0413 | 0.0769 |\n| **COBRA (Full)** | **0.0725** | **0.0424** | **0.0781** |\n\nThe results on public datasets align with those from the industrial setting, demonstrating that the absence of any key component (Dense, ID, or BeamFusion) leads to a performance drop. This confirms the robustness and generalizability of COBRA's proposed components across different scales and data characteristics.\n\n---\n\n### **For Question 4: Experimental Settings Clarifications**\n\n**Beam Size ($M$) Configuration**:\n- **Public Datasets** (Table 1): $M=20$  \n  Optimal for smaller item pools ($11K$-$18K$ items), balancing diversity and efficiency. \n- **Industrial Dataset** (Table 2): $M=50$  \n  Necessary for large-scale scenarios to ensure broad interest coverage. \n\n---\n\n### Conclusion\n\nWe have demonstrated COBRA's performance gains stem from its **cascaded sparse-dense generative paradigm**, not simply GR/DR ensembling. The consistent performance gap across datasets validates each component's necessity. Our comparison with multi-interest methods highlights the architectural uniqueness of our approach, while complexity analyses confirm its practical scalability.\n\n---\n\n### Revision Plan:  \nWe will incorporate all modifications below in the final version:\n1. Complete ablation tables (public + industrial datasets)\n2. Expanded complexity analysis section\n3. Multi-embedding Retrieval baseline comparisons\n4. Engineering optimization details\n\nThank you for your valuable feedback, which has significantly strengthened our work.\n\n---\n\n### References\n[1] Li et al., \"Multi-Interest Network with Dynamic Routing\", CIKM 2019  \n[2] Cen et al., \"Controllable Multi-Interest Framework for Recommendation\", KDD 2020  \n[3] Rajput et al., \"Recommender Systems with Generative Retrieval\", NeurIPS 2023  \n[4] Kang & McAuley, \"Self-Attentive Sequential Recommendation\", ICDM 2018  \n[5] Zhou et al., \"OneRec: Optimizing Recommendation Training Efficiency\", Technical Report 2023"}, {"review_id": "XhFr3HVWkk", "reviewer": "Reviewer_29r5", "summary": "This paper proposes COBRA, a generative recommendation framework that integrates sparse IDs for coarse-grained item semantics and dense vectors for fine-grained details, enabling precise and personalized recommendations. During inference, COBRA introduces a BeamFusion mechanism, combining beam search with nearest neighbor scores to enhance inference flexibility and recommendation diversity. Extensive experiments on both public and industrial datasets demonstrate the effectiveness, robustness, and scalability of COBRA.", "questions": "1. The paper claimed: \"However, unlike [19], which uses a different configuration, we employ a 3-level semantic ID structure, where each level corresponds to a codebook size of 32\" (Lines 227-228). What are the advantages of using the same codebook sizes across all levels, compared to varying codebook sizes?\n2. COBRA uses 3-level semantic IDs (32×32×32) for public datasets (Lines 227–228), while it uses 2-level semantic IDs (32×32) for industrial-scale datasets (Line 254). Does this imply that the number of levels is dataset-dependent? If so, the paper should provide empirical evidence and a guideline for choosing appropriate configurations under different conditions. This also raises concerns about the COBRA’s adaptability across different settings.\n3. In Section 4.2, COBRA uses 2-level semantic IDs (32×32), but its variant “COBRA w/o Dense” uses 3-level semantic IDs (256×256×256) (Line 260). What is the rationale for this difference? This setup may affect the fairness of the ablation.", "limitations": "See the above **Weaknesses**.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "**Strengths** \n\n1. This paper integrates dense vectors to capture fine-grained details of items through end-to-end training, effectively mitigating the information loss in ID-based methods.\n2. This paper presents a BeamFusion mechanism during inference, which balances recommendation accuracy and diversity.\n3. Experimental results validate the COBRA's superiority over compared baselines, with detailed quantitative results and analyses.\n4. This paper is well-organized and clearly written, with a clear introduction, related work, methodology, and experimental evaluation.\n\n**Weaknesses**\n\n1. While the paper incorporates dense vectors to enhance sequential recommendation accuracy over TIGER [1], it also introduces additional computational overhead. Therefore, providing a comparative analysis of model complexity and training/inference runtime across baselines would be valuable for assessing the effectiveness and efficiency of COBRA.\n2. The paper lacks essential implementation details, including the embedding dimension, detailed architectures of the encoder and decoder, optimizer choices, learning rate, batch size, and pretraining settings for the residual quantization model (Figure 2). These omissions raise concerns about the reproducibility of the experimental results.\n3. Several important baselines in the generative recommendation are missing, such as ColaRec [2], LC-Rec [3], and RPG [4]. Including comparisons with these methods would better highlight the effectiveness of the proposed COBRA.\n\n4. While ablation studies are conducted on industrial datasets (Section 4.2), additional ablation analysis on public datasets would further validate the robustness and generalizability of the proposed COBRA.\n5. Some typos are present. \n6. In Lines 108 and 174, \"encoder Encoder\" should be revised for consistency in formatting—either uniformly bolded or unbolded; In Figure 1, the Transformer encoder should be revised—it should be either shown or omitted on both sides to avoid misleading readers into thinking the proposed COBRA omits the encoder compared to TIGER; In Line 170, $\\bold{v}_{j}$ should be revised to $\\bold{v}_{{item}_j}$; In Lines 153–155, obtaining $\\bold{e}_{t+1}$ from predicted $ID_{t+1}$ via the embedding layer Embed (as defined in Line 136) should be explicitly described to ensure clarity in the methodology.\n\n**Reference**\n\n[1] Recommender Systems with Generative Retrieval, NeurIPS, 2023.\n\n[2] Content-Based Collaborative Generation for Recommender Systems, CIKM, 2024.\n\n[3] Adapting Large Language Models by Integrating Collaborative Semantics for Recommendation, ICDE, 2024.\n\n[4] Generating Long Semantic IDs in Parallel for Recommendation, KDD, 2025.", "was_revised": true, "final_justification": "The authors have addressed my comments, and I have maintained the original score (Borderline accept).", "rebuttal": "We thank the reviewer for their thorough feedback and constructive suggestions, which have significantly helped us improve the manuscript. We address each point below.\n\n---\n\n### **For Weakness 1: Computational Overhead**\n\nWe appreciate this important concern. For our complexity analysis, we define $L$ as the sequence length, $T$ as the number of tokens per item, and $D$ as the embedding dimension.\n\n**Training Complexity**\n| Component | COBRA | TIGER[1] (GR) | SASRec[2] (DR) |\n| :---------------------- | :------------------------------------------- | :--------------------------- | :--------------------------- |\n| Item Feature Encoder | $\\mathcal{O}(L \\cdot T^2 \\cdot D + L \\cdot T \\cdot D^2)$ | N/A | N/A |\n| Sequential Model | $\\mathcal{O}(L^2 \\cdot D + L \\cdot D^2)$ | $\\mathcal{O}(L^2 \\cdot D + L \\cdot D^2)$ | $\\mathcal{O}(L^2 \\cdot D + L \\cdot D^2)$ |\n\n**Inference Complexity**\n| Component               | COBRA                     | TIGER                     | SASRec                    |\n|-------------------------|---------------------------|---------------------------|---------------------------|\n| Item Feature Encoder    | $\\mathcal{O}(1)$ (cached) | N/A                       | N/A                       |\n| Sequential Model        | $\\mathcal{O}(L^2 \\cdot D + L \\cdot D^2)$ | $\\mathcal{O}(L^2 \\cdot D + L \\cdot D^2)$| $\\mathcal{O}(L^2 \\cdot D + L \\cdot D^2)$|\n\nTo ensure practical efficiency, we employ several key techniques during implementation. These include sequence Packing  and FlashAttention to minimize computational waste. Furthermore, caching is utilized to decouple the encoder computation, significantly speeding up the inference process. Thanks to these optimizations, COBRA achieves over 30% Model FLOPs Utilization, surpassing typical recommendation benchmarks[3]. This efficiency has enabled successful deployment, serving over 200 million daily users.\n\n---\n\n### **For Weakness 2: Missing Implementation Details**\n\nWe apologize for the omission of critical implementation details. We will add a \"Implementation Details\" table in the Appendix. Below is a comprehensive list of all key hyperparameters:\n\n| Module | Public Datasets | Industrial Datasets |\n| :-------------------- | :------------------------------------------- | :------------------------------------------- |\n| Embedding dimension | 128 | 768 |\n| Encoder layers | 1 | 6 |\n| Decoder layers | 2 | 8 |\n| Attention heads | 8 | 12 |\n| Optimizer | AdamW | AdamW |\n| Learning rate | $3\\mathrm{e}{-4}$  | $4\\mathrm{e}{-5}$  |\n| Batch size | 64 | 64 |\n| Dropout | 0.1 | 0.1 |\n| RQ-VAE pretrain | T5-base, $32 \\times 32 \\times 32$ | ERNIE, $32 \\times 32$ |\n| Training epoch | 10 | 1 |\n\n---\n\n### **For Weakness 3: Missing Baselines (ColaRec, LC-Rec, RPG)**\n\nWe thank the reviewer for highlighting these recent and important generative recommendation baselines. We agree that a comprehensive comparison is valuable for positioning our work within the evolving landscape of the field.\n\nRegarding RPG[4] , we note its publication on arXiv in June 2025, which falls after our submission deadline. For ColaRec[5] and LC-Rec[6], their core contributions lie in aligning semantic IDs with collaborative information, often leveraging pre-trained graph-based representations. While conceptually related to generative recommendations, their distinct methodological focus on explicit semantic-collaborative alignment differs from COBRA's emphasis on unified cascaded sparse-dense representations through end-to-end training. \n\nHowever, to ensure a more comprehensive comparison, we have now conducted experiments on public datasets with LC-Rec. The results on the \"Beauty\" dataset are as follows:\n\n| Metric | Recall@5 | Recall@10 | NDCG@5 | NDCG@10 |\n| :------- | :------- | :-------- | :------- | :-------- |\n| LC-Rec | 0.0482 | 0.0681 | 0.0327 | 0.0409 |\n| COBRA | **0.0537** | **0.0725** | **0.0395** | **0.0456** |\n\nCOBRA consistently outperforms LC-Rec across all metrics, with improvements ranging from 6.4% to 20.8%. This further validates the effectiveness of COBRA's cascaded sparse-dense representation approach for generative recommendation.\n\n---\n\n### **For Weakness 4: Ablation Studies on Public Datasets**\n\nWe initially conducted ablation studies primarily on industrial datasets due to their direct relevance to our business objectives. To address the reviewer's concern about generalizability, we have now performed additional ablation analyses on the \"Beauty,\" \"Sports & Outdoors,\" and \"Toys & Games\" public datasets. The Recall@10 results are summarized below:\n\n| Method | Beauty | Sports & Outdoors | Toys & Games |\n| :---------------------- | :------- | :---------------- | :------------- |\n| COBRA w/o Dense | 0.0656 | 0.0331 | 0.0713 |\n| COBRA w/o ID | 0.0681 | 0.0365 | 0.0653 |\n| COBRA w/o BeamFusion | 0.0714 | 0.0413 | 0.0769 |\n| **COBRA (Full)** | **0.0725** | **0.0424** | **0.0781** |\n\nThe results on public datasets align with those from the industrial setting, demonstrating that the absence of any key component (Dense, ID, or BeamFusion) leads to a performance drop. This confirms the robustness and generalizability of COBRA's proposed components across different scales and data characteristics.\n\n---\n\n### **For Weaknesses 5-6: Formatting and Typos**\n\nWe appreciate you pointing out these errors. We have thoroughly reviewed the manuscript and made the following corrections:\n* The notation of $\\text{Encoder}$ in line 108 have been changed to $\\mathbf{Encoder}$ for consistency throughout the paper.\n* In Figure 1, both sides now explicitly show the  encoder to prevent any misunderstanding regarding COBRA's architecture.\n* In Line 170, the formula have been revised to $v_{{item}_j}$. We have also explicitly clarified in the text that \"the sparse ID is mapped into a dense vector space through an embedding layer\" in Section 3.1 to ensure clarity in the methodology.\n\nAll these modifications will be incorporated into the final version of the paper.\n\n---\n\n### **For Question 1: Advantages of using the same codebook sizes across all levels**\n\nUsing a uniform codebook size (32) across all three levels for public datasets offers design simplicity and streamlines quantization implementation. Based on current empirical observations, this configuration yields satisfactory results. However, a detailed investigation into the impact of varying codebook sizes remains an avenue for future research.\n\n---\n\n### **For Question 2: Number of levels being dataset-dependent and COBRA's adaptability**\n\nThe optimal number of semantic ID levels is primarily driven by **Performance Requirements and Computational Cost**. In industrial scenarios, there's a critical demand for retrieving a large number of diverse items from vast candidate sets within strict latency constraints. Based on our practical experience, employing finer-grained ID hierarchies significantly increases the computational overhead of beam search, which can severely impact real-time performance. Conversely, for smaller datasets or scenarios with less stringent latency requirements, more hierarchical levels might be permissible to capture finer semantic distinctions if needed.\n\nCOBRA demonstrates its adaptability by successfully operating with different semantic ID structures while achieving strong performance. This flexibility indicates that COBRA's core architecture effectively integrates these representations, allowing for configurations that balance semantic granularity with the practical demands of the deployment environment.\n\n---\n\n### **For Question 3: Rationale for COBRA w/o Dense using 3-level semantic IDs vs. COBRA's 2-level IDs on industrial dataset**\n\nCOBRA's full model uses $32 \\times 32$ IDs because dense representations provide significant fault tolerance, allowing for a coarser ID design.\n\nIn contrast, \"COBRA w/o Dense\" **completely removes dense vectors**, necessitating **much finer-grained sparse IDs** to compensate for lost information. A $32 \\times 32$ ID set  would be insufficient for the 2 million industrial advertisements, leading to substantial information loss and consequently poor recall. Therefore, a $256 \\times 256 \\times 256$ configuration is used for \"COBRA w/o Dense\" to maximize its representational capacity and enable a fair baseline comparison against the full model. This highlights the synergistic benefit of COBRA's cascaded sparse-dense approach, achieving superior performance with a less complex sparse ID structure.\n\n---\n\n### References\n[1] Rajput et al., \"Recommender Systems with Generative Retrieval\", NeurIPS 2023  \n[2] Kang & McAuley, \"Self-Attentive Sequential Recommendation\", ICDM 2018  \n[3] Zhou et al., \"OneRec: Optimizing Recommendation Training Efficiency\", Technical Report 2023   \n[4] Hou et al., \"Generating Long Semantic IDs in Parallel for Recommendation\", KDD 2025  \n[5] Wang et al., \"Content-Based Collaborative Generation for Recommender Systems\", CIKM 2024  \n[6] Zheng et al., \"Adapting Large Language Models by Integrating Collaborative Semantics for Recommendation\", ICDE 2024"}, {"review_id": "sCwnsflMyV", "reviewer": "Reviewer_XoPo", "summary": "This paper presents COBRA (Cascaded Organized Bi-Represented generAtive retrieval), a unified framework for generative recommendation that integrates both sparse semantic IDs and dense vector representations in a cascaded manner. The model takes a user’s interaction history encoded as a sequence of sparse-dense pairs and uses a Transformer decoder to alternately predict the next item’s sparse ID followed by its dense vector. The dense representations are trained end-to-end using contrastive learning, and the inference adopts a coarse-to-fine generation strategy supported by a BeamFusion module for diversity control. Experiments on standard benchmarks show COBRA outperforms previous state-of-the-art generative recommenders in both accuracy and diversity.", "questions": "See above weakness", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths\n\n1. The use of sparse IDs as coarse-grained anchors helps reduce the learning difficulty of dense representations and improves controllability in inference.\n\n2. The introduction of BeamFusion adds flexibility and diversity to the generative recommendation process.\n\n3. Empirical results on public benchmarks demonstrate strong performance gains over existing generative recommendation models such as TIGER.\n\n\nWeaknesses\n\n1. The baselines primarily include sparse ID-based models (e.g., TIGER), but miss stronger or more comparable systems like Learnable Item Tokenization for Generative Recommendation, which also explore hybrid sparse-dense paradigms. The absence of such baselines makes it difficult to fully assess COBRA's relative effectiveness.\n\n2. The model architecture used in experiments is relatively small (1-layer encoder and 2-layer decoder), which raises concerns about scalability. Recent models like OneRec suggest that more expressive architectures are becoming standard, and it would strengthen the work to evaluate COBRA’s performance under larger-scale settings.\n\n3. The proposed method lacks sufficient theoretical analysis to justify why the cascaded structure improves performance. While empirical results are promising, scenarios like cold-start—where sparse IDs may be poorly learned—are not thoroughly discussed. The assumption that sparse IDs always enhance dense representations could break down in such cases. Further study on these edge cases would be valuable.", "was_revised": true, "final_justification": "The authors addressed some of my concerns in the rebuttal and demonstrated effectiveness in industrial scenarios, which led me to slightly raise my score. However, this paper is still largely engineering-focused with limited novelty. Moreover, in the rebuttal’s proof for “Weakness 3,” the authors claim that lower entropy facilitates learning, but in recommender systems, considering the need for diversity, lower entropy may not necessarily be beneficial. In addition, when addressing the scaling issue, the authors did not provide concrete improvements in real-world scenarios, which reduces the credibility of their claims.", "rebuttal": "We sincerely appreciate the reviewer's thorough evaluation and constructive suggestions. Below we provide a comprehensive response addressing all concerns.\n\n---\n\n### **For Weakness 1: Baseline Comparison with LETTER**\n\nWe appreciate the reviewer's suggestion to include Learnable Item Tokenization for Generative Recommendation (LETTER[1] ) as a baseline. We recognize LETTER's valuable contribution in proposing a learnable item tokenizer that incorporates regularization to capture hierarchical semantics, collaborative signals, and code assignment diversity within item identifiers. Our initial exclusion of LETTER as a direct baseline was primarily due to the distinct focus of our work. While LETTER optimizes the item tokenization process, enhancing the quality of discrete representations, COBRA introduces a cascaded sparse-dense paradigm. Our method's primary advancement lies in jointly modeling sparse ID generation and dense vector refinement, enabling end-to-end optimization and conditional generation where sparse IDs explicitly guide dense prediction.\n\nTo comprehensively address this point, we conducted comparisons between COBRA and LETTER's implementations (LETTER-TIGER and LETTER-LC-Rec) on the Amazon Beauty dataset:\n\n| Method          | Recall@5 | Recall@10 | NDCG@5 | NDCG@10 |\n|-----------------|----------|-----------|--------|---------|\n| LETTER-TIGER    | 0.0431   | 0.0672    | 0.0286 | 0.0364  |\n| LETTER-LC-Rec   | 0.0505   | 0.0703    | 0.0355 | 0.0418  |\n| COBRA (Ours)    | 0.0537   | 0.0725    | 0.0395 | 0.0456  |\n\nOur approach achieves performance improvements of **7.9-38.1%** over LETTER-TIGER and **3.1-11.3%** over LETTER-LC-Rec across all metrics. This demonstrates the effectiveness of COBRA's cascaded sparse-dense generative approach.\n\nFurthermore, we consider LETTER's optimization of the item tokenization process to be an effective plug-and-play enhancement for semantic IDs, which does not conflict with our core contribution. In future work, COBRA could incorporate designs from LETTER to improve the generation process of sparse IDs, potentially yielding even better results.\n\n---\n\n### **For Weakness 2: Model Scalability**\n\nFor public datasets, we primarily utilized a lightweight architecture (1-layer encoder, 2-layer decoder) to demonstrate efficiency. To address scalability concerns, we modified model parameters (2-layer encoder, 4-layer decoder) and conducted additional experiments on the Amazon Beauty dataset:\n\n| Model Scale         | Recall@5 | Recall@10 | NDCG@5 | NDCG@10 |\n|---------------------|----------|-----------|--------|---------|\n| COBRA-Base (1-enc, 2-dec)  | 0.0537   | 0.0725    | 0.0395 | 0.0456  |\n| COBRA-Large (2-enc, 4-dec) | 0.0535   | 0.0774    | 0.0384 | 0.0461  |\n\nGiven the smaller scale of public datasets, further deepening the network does not yield significant benefits. This aligns with scaling law research[2-3] and our industrial experience, indicating that model size increases must correspond with data volume for gains. As supporting evidence, in larger-scale industrial scenarios, we have deployed a COBRA model with a more substantial architecture (6-layer encoder + 8-layer decoder), achieving significant benefits in industrial applications. These findings demonstrate that COBRA's cascaded paradigm can flexibly scale according to data volume, exhibiting architectural flexibility and real-world scalability.\n\n---\n\n### **For Weakness 3: Theoretical Analysis and Cold-Start Robustness**\n\n#### **Theoretical Justification for Cascaded Design**\n\nThe reviewer raises an excellent point regarding the theoretical analysis of our cascaded design. Our approach addresses a key limitation of ID-based models like TIGER[4]  where discrete semantic IDs can limit expressiveness for fine-grained user intent. We augment each item with a dense representation to capture more comprehensive information, thus each item is represented by a (sparse, dense) pair. We theoretically demonstrate the superiority of our cascaded prediction modeling compared to an independent prediction approach.\n\nLet $S_{1:t}$ be a user's behavior sequence. For an item, $C$ represents a discrete random variable for its sparse ID, and $X$ represents a continuous random variable for its dense vector. We denote the information entropy of a discrete random variable as $H$ and the differential entropy of a continuous random variable as $h$. Our objective is to model $P(Item\\_{t+1} | S\\_{1:t})$ for predicting the next interacted item. We consider two distinct approaches for modeling $Item_{t+1}$:\n\n**Approach 1: Independent Prediction** Predict $C$ and $X$ separately based on the user's historical sequence $S\\_{1:t}$. The probability distribution for this approach is formulated as:\n    $$P(Item_{t+1} | S_{1:t}) = P(C | S_{1:t}) \\times P(X | S_{1:t}) \\quad (1)$$\n\n**Approach 2: Cascaded Prediction** First predict the sparse information of the $Item_{t+1}$ using the user's historical sequence $S\\_{1:t}$, and then utilize this sparse information along with the preceding sequence $S\\_{1:t}$ to predict the dense vector of the next item. The probability distribution for this approach is formulated as:\n    $$P(Item_{t+1} | S_{1:t}) = P(C | S_{1:t}) \\times P(X | C, S_{1:t}) = P(C, X | S_{1:t}) \\quad (2)$$\n\nWe demonstrate that the cascaded prediction modeling (Approach 2) consistently yields superior performance compared to independent prediction modeling.\n\n**Theorem 1: Superiority of Cascaded Modeling** \nLet $E_1$ be the information entropy of the independent prediction modeling probability distribution (Equation 1), and $E_2$ be the information entropy of the cascaded prediction modeling probability distribution (Equation 2). Then, it always holds that $E_2 \\leq E_1$, where equality holds if and only if the random variables $C$ and $X$ are conditionally independent given $S\\_{1:t}$.\n\n**Proof:**\nWe calculate the information entropy for both probability distributions given by equations (1) and (2).\n\nFor independent prediction modeling (Equation 1):\n$$\n\\begin{aligned}\nE\\_1 &= -\\sum\\_{c} \\int_{x} P(c|S_{1:t}) \\times P(x | S_{1:t}) \\log(P(c|S_{1:t}) \\times P(x | S_{1:t})) dx \\\\\\\\\n&= -\\sum_{c} \\int_{x} P(c|S_{1:t}) \\times P(x | S_{1:t}) \\log(P(c|S_{1:t})) dx \\\\\\\\\n&\\quad -\\sum_{c} \\int_{x} P(c|S_{1:t}) \\times P(x | S_{1:t}) \\log(P(x | S_{1:t})) dx \\\\\\\\\n&= H(C | S_{1:t}) + h(X | S_{1:t})\n\\end{aligned}\n$$\n\nHence, under independent modeling, the total entropy decomposes additively over $C$ and $X$.\n\nFor cascaded prediction modeling (Equation 2):\n$$\n\\begin{aligned}\nE_2 &= -\\sum_{c} \\int_{x} P(c|S_{1:t}) \\times P(x | c, S_{1:t}) \\log(P(c|S_{1:t}) \\times P(x | c, S_{1:t})) dx \\\\\\\\\n&= -\\sum_{c} \\int_{x} P(c, x|S_{1:t}) \\log(P(c, x|S_{1:t})) dx \\\\\\\\\n&= H(C, X | S_{1:t})\n\\end{aligned}\n$$\n\nHence, cascaded modeling captures the full joint uncertainty of $C$ and $X$. By the subadditivity of information entropy[5]: $$E_2 = H(C, X | S_{1:t}) \\leq H(C | S_{1:t}) + h(X| S_{1:t}) = E_1$$\nEquality holds if and only if random variables $C$ and $X$ are conditionally independent.\n\nTheorem 1 demonstrates that our cascaded modeling scheme always yields a probability distribution with lower information entropy. This implies that the resulting probability distribution is more concentrated, which is beneficial for model learning. Equality holds only when random variables $C$ and $X$ are conditionally independent, in which case $P(x|c, S_{1:t}) = P(x | S_{1:t})$. From equations (1) and (2), it is evident that both modeling approaches are equivalent under this condition. Consequently, cascaded modeling is generally superior to independent modeling.\n\n#### **Cold-Start Robustness Mechanisms**\n\nAs noted, cold-start presents a challenging problem in recommendation systems. However, COBRA can effectively handle cold-start scenarios to a considerable extent.\n\n **1) Content-Based Dense Representations for Cold-Start Items:** \n COBRA processes raw item attributes (e.g., title, description) to generate dense vectors. \n Crucially, this process operates independently of auto-incrementing system ID. \n This ensures that even for cold-start items with no interaction history, meaningful dense representations are generated based purely on their content.\n\n**2) Leveraging Semantic IDs for Cold-Start Items:**\nFor cold-start items, we generate sparse IDs using the same method as TIGER. As discussed in Section 4 of the TIGER[2] paper, compared to \"Random IDs\" used in traditional recommendation methods and KNN in semantic space, semantic IDs can effectively leverage content-based semantic information, ensuring that items still possess meaningful semantic ID even in cold-start scenarios.\n\n**3) Empirical Validation in Cold-Start Scenarios:** \nMoreover, according to our online A/B test results, cold-start advertisements achieved a **2.43% increase in conversion**, demonstrating the robustness of the COBRA method in cold-start scenarios.\n\n**4) Addressing Poorly Learned IDs:** \nThe reviewer's concern regarding \"poorly learned IDs\" representing an extreme edge case is valid. While our end-to-end training framework fosters mutual learning between sparse and dense representations, inherently reducing the likelihood of such scenarios, the performance under poorly learned sparse IDs could be further analyzed. We plan to explore the effects of such extreme edge cases and potential mitigation strategies (e.g., adaptive weighting of sparse/dense contributions based on ID quality confidence) as part of our future work.\n\n---\n\n### References\n[1] Wang et al., \"Learnable Item Tokenization for Generative Recommendation\", CIKM 2024  \n[2] Hoffmann et al., \"Training Compute-Optimal Large Language Models\", 2022   \n[3] Hou et al., \"Scaling Law of Large Sequential Recommendation Models\", 2023   \n[4] Rajput et al., \"Recommender Systems with Generative Retrieval\", NeurIPS 2023   \n[5] Cover & Thomas. Elements of Information Theory. Wiley-Interscience, USA, 2006"}, {"review_id": "Hle5UxebaF", "reviewer": "Reviewer_7Wfj", "summary": "This paper introduces COBRA, a generative recommendation framework that combines sparse semantic IDs and dense item vectors through a cascaded generation process. By leveraging a coarse-to-fine strategy and a novel BeamFusion inference method, COBRA improves both accuracy and diversity. Extensive evaluations on public and industrial datasets, including online A/B tests, demonstrate its effectiveness over strong baselines.", "questions": "1. Why is there such a large performance gap between GRU4Rec and SASRec across the three datasets?\n2. Why didn’t the authors include comparisons with the most recent methods in both traditional sequential recommendation and generative recommendation?\n3. I suggest that the authors revise the methodology section using more standard equation formatting practices.\n4. Although the paper presents corresponding experimental results, it fails to clearly illustrate the limitations of TIGER and explain why integrating sparse semantic IDs and dense vectors can effectively address the issues in TIGER.", "limitations": "Yes", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. The paper presents a well-motivated and novel cascaded representation learning strategy that tightly integrates sparse semantic IDs and dense vectors in both training and inference, addressing limitations of previous generative retrieval methods that rely solely on one modality.\n2. The model is not only validated through rigorous offline experiments but also deployed in a real-world recommendation platform.\nThe authors provide extensive quantitative evaluations, including ablations and diversity trade-off analysis, offering strong evidence for the effectiveness and robustness of their approach.\n\nWeaknesses:\n1. As far as I know, the only difference between GRU4Rec and SASRec is that the former uses RNN for sequential modeling, while the latter adopts attention mechanisms. While attention-based methods may have certain advantages, why is there such a significant performance gap between the two on all three datasets?\nI believe the baselines used in the experiments can be roughly divided into two categories: traditional sequential recommendation methods, such as GRU4Rec, and generative approaches, such as P5 and TIGER. However, the authors did not compare their method with the latest baselines from either category.\n2. The authors did not present each equation on a separate line with a corresponding number in the methodology section, which slightly deviates from standard academic writing conventions.\n3. While the empirical results are convincing, the paper lacks a formal theoretical analysis of why cascaded sparse-dense generation yields superior performance or how error propagation between stages is mitigated.", "was_revised": true, "final_justification": "Thanks author's response. I will maintain my original positive score.", "rebuttal": "We sincerely thank the reviewer for the constructive feedback and thoughtful questions. Below, we address the key concerns raised.\n\n---\n\n### **For Weakness1 & Question 1: Large performance gap between GRU4Rec and SASRec**\n\nWe appreciate the reviewer’s attention to this detail. Indeed, both GRU4Rec[1] and SASRec[2] are classical sequential recommendation models, but they differ significantly in their capacity to capture long-range dependencies and sequence context.\n\nGRU4Rec leverages RNNs, which are known to struggle with modeling long user behavior sequences due to vanishing gradient issues. In contrast, SASRec employs self-attention mechanisms that allow the model to directly attend to any previous item in the sequence, thereby capturing richer patterns and long-range dependencies more effectively. This architectural advantage explains SASRec’s consistent outperformance on all datasets. This observation is also consistently reported in other research, such as TIGER[3] and S$^3$-Rec[4].\n\n---\n\n### **For Weakness 1 & Question 2: Comparisons with the most recent methods**\n\nCOBRA's core contribution lies in its novel cascaded sparse-dense representation for items, which integrates the strengths of both generative and dense retrieval paradigms through a coarse-to-fine generation process. Our primary focus is on this paradigm shift. We contend that most existing generative recommendation improvements focus on optimizing sparse ID generation, which is orthogonal to our paradigm-level innovation. We believe that such improvements could potentially be integrated into COBRA's sparse ID generation.\n\nTo further validate COBRA's effectiveness, we conducted additional experiments on the Amazon Beauty dataset, comparing COBRA with recent representative approaches: LC-Rec[5] , GenRec[6] , and LETTER[7]. The results are as follows:\n\n| Method         | Recall@5 | Recall@10 | NDCG@5  | NDCG@10 |\n| :------------- | :------- | :-------- | :------ | :------ |\n| LC-Rec          | 0.0482   | 0.0681    | 0.0327  | 0.0409  |\n| LETTER           | 0.0505   | 0.0703    | 0.0355  | 0.0418  |\n| GenRec          | 0.0515   | 0.0641    | 0.0397  | 0.0439  |\n| COBRA (Ours)    | 0.0537   | 0.0725    | 0.0395  | 0.0456  |\n\nAs the results demonstrate, COBRA consistently achieves strong performance. Specifically, it shows a +4.3% improvement in Recall@5 compared to GenRec , a +13.1% improvement in Recall@10 over LC-Rec , and an +8.7% improvement in NDCG@10 compared to GenRec. While GenRec shows a slightly higher NDCG@5, COBRA generally achieves superior holistic performance, confirming the unique value of our cascaded sparse-dense paradigm.\n\n---\n\n### **For Weakness 2 & Question 3: Equation format**\n\nWe appreciate the reviewer’s suggestion regarding presentation. To improve clarity and align with academic writing standards, we will revise the methodology section by separating each key equation into its own line with proper numbering. This will be reflected in the final submission.\n\n---\n\n### **For Weakness 3 & Question 4: Formal theoretical analysis on sparse-dense cascading**\n\nThe reviewer raises an excellent point regarding the theoretical analysis of our cascaded design. Our approach addresses a key limitation of ID-based models like TIGER  where discrete semantic IDs can limit expressiveness for fine-grained user intent. We augment each item with a dense representation to capture more comprehensive information, thus each item is represented by a (sparse, dense) pair. We theoretically demonstrate the superiority of our cascaded prediction modeling compared to an independent prediction approach.\n\nLet $S_{1:t}$ be a user's behavior sequence. For an item, $C$ represents a discrete random variable for its sparse ID, and $X$ represents a continuous random variable for its dense vector. We denote the information entropy of a discrete random variable as $H$ and the differential entropy of a continuous random variable as $h$. Our objective is to model $P(Item\\_{t+1} | S\\_{1:t})$ for predicting the next interacted item. We consider two distinct approaches for modeling $Item_{t+1}$:\n\n**Approach 1: Independent Prediction** Predict $C$ and $X$ separately based on the user's historical sequence $S\\_{1:t}$. The probability distribution for this approach is formulated as:\n    $$P(Item_{t+1} | S_{1:t}) = P(C | S_{1:t}) \\times P(X | S_{1:t}) \\quad (1)$$\n\n**Approach 2: Cascaded Prediction** First predict the sparse information of the $Item_{t+1}$ using the user's historical sequence $S\\_{1:t}$, and then utilize this sparse information along with the preceding sequence $S\\_{1:t}$ to predict the dense vector of the next item. The probability distribution for this approach is formulated as:\n    $$P(Item_{t+1} | S_{1:t}) = P(C | S_{1:t}) \\times P(X | C, S_{1:t}) = P(C, X | S_{1:t}) \\quad (2)$$\n\nWe demonstrate that the cascaded prediction modeling (Approach 2) consistently yields superior performance compared to independent prediction modeling.\n\n**Theorem 1: Superiority of Cascaded Modeling** \nLet $E_1$ be the information entropy of the independent prediction modeling probability distribution (Equation 1), and $E_2$ be the information entropy of the cascaded prediction modeling probability distribution (Equation 2). Then, it always holds that $E_2 \\leq E_1$, where equality holds if and only if the random variables $C$ and $X$ are conditionally independent given $S\\_{1:t}$.\n\n**Proof:**\nWe calculate the information entropy for both probability distributions given by equations (1) and (2).\n\nFor independent prediction modeling (Equation 1):\n$$\n\\begin{aligned}\nE\\_1 &= -\\sum\\_{c} \\int_{x} P(c|S_{1:t}) \\times P(x | S_{1:t}) \\log(P(c|S_{1:t}) \\times P(x | S_{1:t})) dx \\\\\\\\\n&= -\\sum_{c} \\int_{x} P(c|S_{1:t}) \\times P(x | S_{1:t}) \\log(P(c|S_{1:t})) dx \\\\\\\\\n&\\quad -\\sum_{c} \\int_{x} P(c|S_{1:t}) \\times P(x | S_{1:t}) \\log(P(x | S_{1:t})) dx \\\\\\\\\n&= H(C | S_{1:t}) + h(X | S_{1:t})\n\\end{aligned}\n$$\n\nHence, under independent modeling, the total entropy decomposes additively over $C$ and $X$.\n\nFor cascaded prediction modeling (Equation 2):\n$$\n\\begin{aligned}\nE_2 &= -\\sum_{c} \\int_{x} P(c|S_{1:t}) \\times P(x | c, S_{1:t}) \\log(P(c|S_{1:t}) \\times P(x | c, S_{1:t})) dx \\\\\\\\\n&= -\\sum_{c} \\int_{x} P(c, x|S_{1:t}) \\log(P(c, x|S_{1:t})) dx \\\\\\\\\n&= H(C, X | S_{1:t})\n\\end{aligned}\n$$\n\nHence, cascaded modeling captures the full joint uncertainty of $C$ and $X$. By the subadditivity of information entropy[3]:$$E_2 = H(C, X | S_{1:t}) \\leq H(C | S_{1:t}) + h(X| S_{1:t}) = E_1$$\nEquality holds if and only if random variables $C$ and $X$ are conditionally independent.\n\nTheorem 1 demonstrates that our cascaded modeling scheme always yields a probability distribution with lower information entropy. This implies that the resulting probability distribution is more concentrated, which is beneficial for model learning. Equality holds only when random variables $C$ and $X$ are conditionally independent, in which case $P(x|c, S_{1:t}) = P(x | S_{1:t})$. From equations (1) and (2), it is evident that both modeling approaches are equivalent under this condition. Consequently, cascaded modeling is generally superior to independent modeling.\n\n---\n\n### Conclusion\n\nWe thank the reviewer again for their detailed and insightful comments. We will incorporate all of the above improvements into the final version to enhance clarity, completeness, and academic rigor.\n\n---\n\n### Reference\n1. Hidasi et al., \"Session-based Recommendations with Recurrent Neural Networks\"  \n2. Kang & McAuley, \"Self-Attentive Sequential Recommendation\", ICDM 2018   \n3. Rajput et al., \"Recommender Systems with Generative Retrieval\", NeurIPS 2023  \n4. Zhou et al., \"S^3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization\", CIKM 2020  \n5. Zheng et al., \"Adapting Large Language Models by Integrating Collaborative Semantics for Recommendation\", ICDE 2024   \n6. Cao & Lio, \"GenRec: Generative Sequential Recommendation with Large Language Models\", ECIR 2024  \n7. Wang et al., \"Learnable Item Tokenization for Generative Recommendation\", CIKM 2024"}]

Author rebuttal:

Meta-review: This work continues the recent important trend of generative sequential recommenders by combining both sparse and dense representations in a cascaded manner that the authors theoretically show to reduce entropy, which is claimed to yield a better model fit.  A reviewer questions whether lower entropy negatively impacts diversity; the authors provide further technical explanation in the discussion, which should be incorporated in the paper to clarify the technical claims.  The work empirically achieves SOTA results on public academic and industrial datasets.  

Overall, while the methodology seems somewhat incremental and highly engineered, SOTA results are achieved, theoretical justification is provided and this is a timely contribution to a nascent field of generative recommendation models.  After the extensive rebuttal discussion, all reviewers (weakly) recommend NeurIPS acceptance and the AC agrees with the decision to accept.
Decision: Accept (poster)