SynerGen: Contextualized Generative Recommender for Unified Search and Recommendation

## Abstract

Abstract The dominant retrieve-then-rank pipeline in large-scale recommender systems suffers from mis-calibration and engineering overhead due to its architectural split and differing optimization objectives. While recent generative sequence models have shown promise in unifying retrieval and ranking by auto-regressively generating ranked items, existing solutions typically address either personalized search or query-free recommendation, often exhibiting performance trade-offs when attempting to unify both. We introduce SynerGen , a novel generative recommender model that bridges this critical gap by providing a single generative backbone for both personalized search and recommendation, while simultaneously excelling at retrieval and ranking tasks. Trained on behavioral sequences, our decoder-only Transformer leverages joint optimization with InfoNCE for retrieval and a hybrid pointwise-pairwise loss for ranking, allowing semantic signals from search to improve recommendation and vice versa. We also propose a novel time-aware rotary positional embedding to effectively incorporate time information into the attention mechanism. SynerGen achieves significant improvements on widely adopted recommendation and search benchmarks compared to strong generative recommender and joint search and recommendation baselines. This work demonstrates the viability of a single generative foundation model for industrial-scale unified information access.

## 1 Introduction

Large-scale search and recommendation systems in e-commerce, short video, and food-delivery platforms are typically deployed as multi-stage cascades. A high-recall retriever (e.g., BM25 (Robertson et al., 2009), two-tower (Covington et al., 2016), or item–item collaborative filter (Linden et al., 2003)) narrows hundreds of millions of items to a few thousand candidates, which are then re-ordered by a compute-intensive ranker to optimize business metrics such as click-through rate (CTR), revenue, or watch-time. While pragmatic, this architecture forces each stage to optimize different losses, operate on disjoint features, and refresh on separate cadences—leading to redundant engineering and production misalignment.

Inspired by the success of large language models, recent work reframes recommendation as a generative problem: an autoregressive model directly outputs a ranked slate conditioned on user context, thereby aligning retrieval and ranking under a single objective. These generative recommenders have shown promising results in specific domains such as short-video feeds, food delivery, and even trillion-scale deployments.

However, existing systems typically target only one interaction mode. Query-aware search and query-free feed recommendation remain siloed: some models focus exclusively on search, while others excel in feed recommendation but ignore queries. Attempts at unification reveal an inherent trade-off: semantic query signals and collaborative behavioral signals compete within shared representations, so improving one task often degrades the other. To date, no published work demonstrates a single generative backbone that simultaneously excels at both retrieval and ranking and supports both query-aware search and query-free recommendation.

We address this gap with SynerGen, the first decoder-only generative recommender that unifies these four capabilities. Retrieval and ranking are jointly optimized within one backbone: retrieval with an InfoNCE loss (Oord et al., 2018) (using in-batch and mined hard negatives), and ranking with a hybrid pointwise–pairwise loss. Semantic, behavioral, and temporal signals are fused before the first Transformer layer, ensuring richer contextualization than late concatenation. Joint training allows semantic cues from queries to strengthen feed recommendation and collaborative signals to benefit search.

By demonstrating that a single generative foundation model can deliver state-of-the-art performance across both retrieval and ranking, in both search and recommendation, while meeting strict latency constraints, SynerGen positions generative recommendation as a universal backbone for industrial information access.

##### Contributions.

In summary, we make the following contributions:
1. We present the first decoder-only generative model that seamlessly supports both query-aware search and query-free recommendation, achieving superior results on standard benchmarks.
2. We show that retrieval and ranking can be jointly optimized within a single backbone without cross-stage misalignment, simplifying deployment while improving effectiveness.

Figure: Figure 1: Overview of SynerGen pretraining, illustrating unified retrieval and ranking tasks within a joint search and recommendation framework. A single forward pass processes a user behavior sequence of length $n$, with context and task tokens jointly encoded. Semantic embeddings cached from a pretrained language model are frozen during training.
Refer to caption: figures/model_arch.png

## 2 Background and Related Work

As large generative models emerge in recommendation and search, the focus of applications has shifted toward addressing long-standing challenges such as: unifying search and recommendation, modeling evolving user preferences, and incorporating temporal patterns into sequential prediction.

Unifying Search and Recommendation.
Traditional search and recommendation systems have been developed in silos—search engines retrieve items based on explicit queries, while recommenders predict user preferences from interaction histories. Such separation ignores the shared users and overlapping item spaces across tasks. Recent work has sought to unify search and recommendation into a single modeling framework (Zhao et al., 2025; Liu et al., 2024), enabling richer user representations, improved generalization, and reduced data sparsity through joint training. Notable examples include USER (Yao et al., 2021), which models heterogeneous sequences of queries and item interactions, and UnifiedSSR (Xie et al., 2023), which learns a shared representation of user behavior across both modalities. UniSAR (Shi et al., 2024) further refines this idea by explicitly modeling cross-modal transitions, using contrastive objectives and cross-attention to align and fuse behaviors from search and recommendation.

Generative Search and Recommendation.
The emergence of generative large language models (LLMs) has catalyzed a new paradigm for unifying search and recommendation in e-commerce. Unlike traditional retrieval methods that rely on sparse or dense vector matching, generative LLMs can directly generate item identifiers or embeddings in response to user queries or profiles, enabling a seamless integration of both search and recommendation within a single framework. Recent advances demonstrate two primary approaches: text-based generative inputs (e.g., HLLM Chen et al. (2024)) and ID-based inputs (e.g., HSTU Zhai et al. (2024) and MTGR Han et al. (2025)). Both have shown that generative models can bridge the gap between search (semantic relevance) and recommendation (collaborative filtering).

Beyond these, a growing set of generative recommenders explore collapsing the retrieval–ranking pipeline altogether. OneRec (Deng et al., 2025) applies an encoder–decoder MoE architecture to short-video feeds, surpassing strong two-tower baselines. MTGR (Han et al., 2025) extends this idea to food delivery, using a hierarchical backbone with dynamic masking to preserve cross-feature interactions. Actions Speak Louder than Words (Zhai et al., 2024) scales generative recommendation to the trillion-parameter regime with specialized fast-attention kernels, achieving double-digit online lifts. In contrast, GenR-PO (Li et al., 2024) targets e-commerce search but does not support feed recommendation, while GenRank (Huang et al., 2025) focuses on recommendation without query signals. More recent attempts such as UniSAR (Shi et al., 2024) and GenSAR (Shi et al., 2025) aim at unified modeling but report trade-offs between query-aware and query-free modes.

**Table 1: Comparison of representative generative recommender systems. Existing work typically supports only one mode (search or recommendation) or suffers from performance trade-offs when unifying them. SynerGen uniquely unifies retrieval and ranking across both modes without sacrificing effectiveness.**
| Model | Architecture | Query-Aware Search | Query-Free Recommendation |
| --- | --- | --- | --- |
| HLLM (Chen et al., 2024) | Two Tier Decoder | $\times$ | $\checkmark$ (short-video feeds) |
| OneRec (Deng et al., 2025) | Encoder–Decoder MoE | $\times$ | $\checkmark$ (short-video feeds) |
| HSTU (Zhai et al., 2024) | Causal Encoder on ID-inputs | $\times$ | $\checkmark$ (Social Media) |
| MTGR (Han et al., 2025) | HSTU + dynamic masking | $\times$ | $\checkmark$ (food delivery) |
| Actions Speak (Zhai et al., 2024) | Trillion-scale fast-attention | $\times$ | $\checkmark$ (internet-scale feeds) |
| GenR-PO (Li et al., 2024) | Decoder-only | $\checkmark$ (e-commerce search) | $\times$ |
| GenRank (Huang et al., 2025) | Decoder-only | $\times$ | $\checkmark$ (recommendation) |
| UniSAR (Shi et al., 2024) | Unified decoder | $\checkmark$ | $\checkmark$ (but trade-off) |
| GenSAR (Shi et al., 2025) | Unified decoder | $\checkmark$ | $\checkmark$ (but trade-off) |
| SynerGen (ours) | Decoder-only | $\checkmark$ | $\checkmark$ |

Time-Aware Sequence Modeling.
Incorporating temporal information into sequential recommendation allows models to better capture evolving user interests and short-term intent. The prevailing approach, used in seminal works such as SasRec Kang & McAuley (2018a) and Bert4Rec Sun et al. (2019), is to arrange event sequences chronologically. Beyond simple ordering, time can be injected directly through positional embeddings (absolute time) Makhneva et al. (2023) or through relative encodings in the attention mechanism. The latter has gained traction, with models such as HSTU Zhai et al. (2024) incorporating relative temporal bias into self-attention, enabling finer-grained modeling of temporal dependencies.

## 3 Method

In this section, we describe the proposed method in detail, beginning with the problem formulation and overall model overview.

### 3.1 Problem Formulation and Model Overview

We consider a user’s historical interaction sequence $H=\{x_{1},x_{2},\ldots,x_{T}\}$, where each timestamped event $x_{t}$ is represented as a tuple of four heterogeneous token types:

$$ $x_{t}=(\mathrm{ITEM}_{t},\mathrm{ACTION}_{t},\mathrm{TIME}_{t},\mathrm{QUERY}_{t}).$ $$

Here, $\mathrm{ITEM}$ denotes the product identity, $\mathrm{ACTION}$ specifies the interaction type (e.g., click, purchase, add-to-cart), $\mathrm{TIME}$ encodes the temporal context, and $\mathrm{QUERY}$ captures the associated search query or contextual signals.

Our objective is to jointly solve two core tasks of recommender systems within a single framework:
(1) Relevance estimation, which retrieves potentially relevant items from the entire catalog, and
(2) Click probability estimation, which ranks retrieved items by estimating the likelihood of user interaction.

SynerGen unifies these tasks in a generative paradigm by formulating them as conditional sequence generation problems. Specifically, retrieval corresponds to predicting the next relevant item given user history and intent, while ranking corresponds to estimating the probability of a specified action on a candidate item conditioned on history and query context.

### 3.2 Input Representation and Multi-Modal Fusion

##### Embedding Construction.

Each interaction token $x_{t}$ is mapped into a continuous embedding space via a hybrid design that integrates semantic, collaborative, and contextual features:

$$ $\mathrm{ITEM}=(e^{\mathrm{item}_{s}};e^{\mathrm{item}_{c}}),\quad\mathrm{ACTION}=e^{\mathrm{action}},\quad\mathrm{QUERY}=e^{\mathrm{query}}.$ $$

Semantic embeddings $e^{\mathrm{item}_{s}}$ and $e^{\mathrm{query}}$ are derived from a pretrained encoder $\mathrm{LM}$ (a 350M-parameter LLaMA-based model Touvron et al. (2023)), trained on item metadata, user queries, and query–item matching tasks with a joint masked language modeling and contrastive objective Zhang et al. (2025); BehnamGhader et al. (2024). This encoder produces $d$-dimensional semantic representations, where $e^{\mathrm{item}_{s}}$ encodes product attributes (e.g., title, brand, color, material) and $e^{\mathrm{query}}$ captures query semantics.

In parallel, collaborative embeddings $e^{\mathrm{item}_{c}}$ (item IDs) and $e^{\mathrm{action}}$ (interaction types) are randomly initialized and optimized during task-specific training. The combination of semantic signals (generalization) and ID-based signals (collaboration) ensures robustness to both frequent and long-tail items.

The final interaction representation $x_{t}$ is formed by concatenating all embeddings, which are projected to the model dimension via a fusion MLP, denoted as $\text{MLP}_{\text{fusion}}$.

### 3.3 Decoder Backbone Architecture

SynerGen adopts a decoder-only Transformer with $L$ layers, hidden dimension $d$, and $H$ masked self-attention heads. The left-to-right causal model processes the sequence $x_{t}$ in an auto-regressive manner, which produces the next token by conditioning on past timesteps.

A key novelty in our architecture is the design of task tokens, which explicitly signal whether the model should perform context modeling, relevance estimation, or click probability estimation. These tokens allow the backbone to seamlessly switch between retrieval and ranking within the same sequence.

##### Context Token.

The context token captures a user’s behavioral history in a query-free manner, serving as the foundation for feed-style recommendation. By masking out query signals and retaining only item and action embeddings, the model learns a pure representation of evolving user intent:

$$ $e^{\text{context}}=\text{MLP}_{\text{fusion}}([0;e^{\langle\text{item}\rangle};e^{\text{action}}]).$ $$

With the causal attetnion structure, this ensures that feed recommendation can be performed directly from behavioral dynamics, while also providing a clean state that query-aware retrieval and ranking can later condition on without leakage.

##### Retrieval Token.

The retrieval token is tailored for query-aware search. It takes the user’s query and action type as input, but masks the item identity:

$$ $e^{\text{rel}}=\text{MLP}_{\text{fusion}}([e^{\text{query}};e^{\langle\text{mask}\rangle};e^{\text{action}}]).$ $$

Masking the item using a trainable embedding $e^{\langle\text{mask}\rangle}$ prevents trivial copying of the ground truth and forces the model to infer relevant items consistent with the query context. This mirrors real search scenarios, where the system must generate relevant candidates from a large catalog conditioned on query signals but without direct knowledge of the clicked item.

##### Ranking Token.

The ranking token supports both modes by calibrating probabilities of interaction with specific candidate items. Unlike retrieval, the candidate embedding is explicitly included but the query can be chosen to be presented or not:

$$ $e^{\text{rank}}=\text{MLP}_{\text{fusion}}([e^{\text{query}};e^{\text{item}};e^{\text{action}}]).$ $$

One can choose query to be present of nor for ranking task. In query-aware search, this design enables fine-grained click-probability estimation for items surfaced by retrieval; in query-free feed recommendation, the same ranking token evaluates each candidate’s engagement likelihood purely from historical context. Explicitly supplying the candidate identity allows the model to perform calibrated probability estimation across both modes rather than mere sequence reconstruction. For the ranking token training, we include both positive and negative clicks: this balances the click vs. non-click label distribution and improves training efficiency, since mask-controlled attention lets positive and negative candidates share the same Transformer-encoded context while only the candidate representation changes.

##### Unified Training and Inference.

Retrieval and ranking are cast as conditional prediction problems: retrieval predicts relevant items given history, query, and action but without any item information, while ranking estimates interaction likelihood for given information on query, item, and history. During training, task tokens are inserted throughout sequences to provide dense supervision, exposing the backbone to both objectives. At inference, retrieval tokens score the entire catalog, while ranking tokens refine candidates with calibrated probabilities. This setup mirrors production pipelines, but within a single generative backbone, closing the gap between training and deployment.

### 3.4 Temporal Attention Mechanism

##### Task specific masking matrix

To regulate information flow among heterogeneous tokens, we introduce a task-specific masking matrix $M$ that modifies the standard self-attention mechanism. In order to maintain the causality for each task, $M$ enforces three types of constraints. First, temporal causality: context tokens are only allowed to attend to strictly earlier events, ensuring chronological consistency in behavior modeling. Second, session isolation: retrieval and ranking tokens may only access historical context from previous request groups, preventing leakage of information from the current query session. Third, cross-task alignment: ranking tokens are additionally permitted to attend to retrieval tokens corresponding to the same interaction event, allowing them to reuse relevance signals when estimating click probabilities. The exact mathematical specification of our masking matrix $M$ is provided in Appendix.

##### Modeling Time with RoPE

User behavior unfolds over real time, with irregular intervals that simple sequential encodings cannot capture. To address this, we adopt Rotary Positional Embeddings (RoPE) Su et al. (2021) applied directly to Unix timestamps. Unlike discrete bucket encodings or absolute positional embeddings Vaswani et al. (2017), this approach encodes temporal gaps as rotations in the attention space. The design provides three benefits: (i) fine-grained modeling of absolute and relative time, (ii) shift invariance with respect to session start, and (iii) extrapolation to unseen time intervals.
The full mathematical formulation of RoPE-based temporal attention, together with a worked example illustrating how absolute timestamps translate into relative rotations, is provided in Appendix.

### 3.5 Multi-Task Training Objectives

We jointly optimize retrieval and ranking objectives within a single training loop, ensuring that both coarse-grained relevance estimation and fine-grained click probability estimation are aligned within the same backbone.

##### Retrieval Loss.

The retrieval objective is formulated as an InfoNCE contrastive loss Oord et al. (2018), aligning retrieval token representations $h_{i}$ with their corresponding positive item embeddings $e^{+}_{i}$ while pushing them away from negatives:

$$ $\mathcal{L}_{\text{rel}}=\alpha\mathcal{L}_{\text{rel}}^{\text{easy}}+(1-\alpha)\mathcal{L}_{\text{rel}}^{\text{hard}}.$ $$

Each component follows the standard InfoNCE form:

$$ $\mathcal{L}_{\text{rel}}^{*}=-\sum_{i=1}^{N_{\text{rel}}}\log\frac{\exp(\text{sim}(h_{i},e^{+}_{i})/\tau)}{\exp(\text{sim}(h_{i},e^{+}_{i})/\tau)+\sum_{j}\exp(\text{sim}(h_{i},e^{-}_{j})/\tau)},$ $$

where $\text{sim}(\cdot)$ denotes dot-product similarity and $\tau$ is a temperature parameter.

We adopt two types of negatives:
(i) in-batch random negatives, which provide broad semantic contrast across items, and
(ii) impressed-but-not-clicked negatives, which are items displayed to the user but ignored, representing challenging decision boundaries.
This dual-negative strategy encourages both general discrimination (via easy negatives) and fine-grained preference modeling (via hard negatives).

##### Ranking Loss.

For ranking, we construct positive–negative pairs using clicked (positive) and impressed-but-not-clicked (negative) observed in the same context. This design ensures that the model learns to distinguish between items that were actually presented to the user, rather than arbitrary negatives.

We employ a hybrid objective that combines pointwise and pairwise components. The pointwise loss $\mathcal{L}_{\text{point}}$ is a binary cross-entropy applied independently to each ranking token:

$$ $\mathcal{L}_{\text{point}}=-\sum_{i=1}^{N_{\text{rank}}}\Big[y_{i}\log\sigma(s_{i})+(1-y_{i})\log\big(1-\sigma(s_{i})\big)\Big],\quad\mathcal{L}_{\text{pair}}=-\sum_{(i,j)\in\mathcal{P}}\log\sigma(s_{i}-s_{j}),$ $$

where $s_{i}$ is the logit score predicted by the ranking head for token $i$ upon the decoder module, $y_{i}\in\{0,1\}$ denotes the click label, and $\sigma(\cdot)$ is the sigmoid function. This objective encourages the model to output calibrated probabilities for click vs. non-click.

In addition, we apply a pairwise loss that directly enforces relative ordering between positive and negative candidates. For each positive–negative pair $(i,j)$, the loss takes the form as equation $\mathcal{L}_{\text{pair}}$,
where $\mathcal{P}$ is the set of observed positive–negative pairs. This term encourages the score of the positive item to exceed that of the negative by a large margin.

The overall ranking loss combines the two objectives:

$$ $\mathcal{L}_{\text{rank}}=\mathcal{L}_{\text{point}}+\lambda\mathcal{L}_{\text{pair}},$ $$

where $\lambda$ controls the trade-off between probability calibration and relative ranking accuracy.

##### Joint Optimization.

The final training objective combines retrieval and ranking losses:

$$ $\mathcal{L}=\mathcal{L}_{\text{rel}}+\mathcal{L}_{\text{rank}}.$ $$

This joint formulation tightly couples the two tasks: retrieval benefits from ranking’s fine-grained supervision, while ranking leverages retrieval’s catalog-level discrimination. Unlike conventional two-stage pipelines, our unified optimization ensures consistent representation learning and reduces system complexity.

## 4 Experiments

In this section, we present a comprehensive evaluation of SynerGen. We begin by describing the experimental setup and evaluation metrics, followed by comparative results against state-of-the-art baselines. We then analyze various model components through ablation studies to understand their individual contributions.

**Table 2: Performance of using Book Review dataset in recommendation task with all 686k Items as Candidates.**
| Method | R@10 | R@50 | R@200 | N@10 | N@50 | N@200 |
| --- | --- | --- | --- | --- | --- | --- |
| SynerGen-ID ^† | 7.33 | 17.49 | 30.39 | 3.91 | 6.11 | 8.09 |
| HLLM-1B-Scratch | 6.85 | 13.95 | 23.19 | 4.02 | 5.56 | 6.95 |
| HSTU-large ^† | 6.50 | 12.22 | 19.93 | 3.99 | 5.24 | 6.38 |
| SASRec ^† | 5.35 | 11.91 | 21.02 | 2.98 | 4.40 | 5.76 |
| SynerGen* | 9.91 | 19.29 | 30.48 | 5.45 | 7.15 | 8.43 |
| Bert4Rec* | 4.58 | 12.54 | 24.08 | 1.94 | 3.66 | 5.39 |
| HLLM-1B* | 9.28 | 17.34 | 27.22 | 5.65 | 7.41 | 8.89 |

**Table 3: Performance comparison on the eBook Search Sessions dataset across recommendation-only (left) and search-enhanced (right) settings using a candidate pool of 100 items (1 positive, 99 random negatives). Note that methods designed for only recommendation or only search are not applicable in the other setting and are thus omitted where appropriate.**
|  | Recommendation (no query) | Search (w/ synthetic query) |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Method | R@1 | R@5 | R@10 | N@5 | N@10 | R@1 | R@5 | R@10 | N@5 | N@10 |
| SynerGen | 31.36 | 59.36 | 70.72 | 46.13 | 49.83 | 63.35 | 85.25 | 90.30 | 75.41 | 77.10 |
| UnifiedSSR | 20.13 | 51.96 | 67.07 | 36.62 | 41.51 | 36.63 | 77.44 | 88.12 | 58.47 | 61.96 |
| UniSAR | 30.10 | 58.74 | 70.20 | 45.13 | 48.85 | 53.43 | 81.90 | 89.77 | 68.75 | 71.32 |
| USER | 23.61 | 54.41 | 68.54 | 39.64 | 44.22 | 41.23 | 76.31 | 86.97 | 60.00 | 63.48 |
| SESRec | 27.26 | 56.23 | 68.64 | 42.45 | 46.48 | – | – | – | – | – |
| BERT4Rec | 24.81 | 53.11 | 66.58 | 39.54 | 43.90 | – | – | – | – | – |
| SASRec | 20.59 | 52.95 | 67.72 | 37.47 | 42.25 | – | – | – | – | – |
| TEM | – | – | – | – | – | 40.52 | 81.69 | 90.51 | 63.03 | 65.87 |
| CoPPS | – | – | – | – | – | 31.17 | 66.16 | 77.07 | 62.81 | 65.70 |

### 4.1 Datasets and Evaluation Protocols

We highlight our evaluation for SynerGen on three representative datasets: Book Review, eBook Search Sessions, and Session-US. The statistics for each dataset are summarized in the appendix.

Preprocessing.
For Book Review and eBook Search Sessions, we follow standard practice Chen et al. (2024): users and items with fewer than five interactions are removed, and a leave-one-out strategy is used to split sessions into training, validation, and test sets. Baseline results are reproduced under identical protocols for fair comparison.

Book Review.
This dataset is used for sequential recommendation. Evaluation is performed over the full candidate set, with Recall@K and NDCG as metrics.

eBook Search Sessions.
Constructed from large-scale interaction data, this dataset supports joint search and recommendation. Synthetic queries are generated following (Ai et al., 2017; 2019; Si et al., 2023; Shi et al., 2024). Each ground-truth item is paired with 99 randomly sampled negatives to form a candidate pool of 100. We report Recall@1,5,10 and NDCG@5,10, averaged over 10 runs with different seeds.

Session-US.
A large-scale real-world e-commerce dataset derived from search logs (Wang et al., 2023). Interactions are chronologically ordered and segmented into weekly sub-sessions containing the final 100 events, with sessions of fewer than two actions discarded. Each record includes item IDs, timestamps, action types (clicks/purchases), and queries. The dataset exhibits realistic sparsity (average 60 interactions per session, long-tail item distribution). During training, the model processes roughly 75B relevance tokens and 150B ranking tokens per epoch.

Two benchmark tasks are defined:
1. Contextualized Recommendation — predict the next clicked or purchased item two days after training, without query information; evaluated by NDCG over 15 viewed items per impression.
2. Contextualized Search — predict the next click two days after training, with query information; evaluated by Recall over the full item set for retrieval and MRR over 7 viewed items for ranking. Both tasks employ $k$-nearest neighbor retrieval to assess the model’s ability to capture user preferences under different contexts.

**Table 4: Performance and ablation results across recommendation and search tasks on the Session-US dataset. For the Full model, all parameters are trained jointly; results are reported either by evaluating the ranking head or the retrieval head separately. All ablation experiments are evaluated via the retrieval head.**
|  | Retrieval (10M Pool) | Ranking tasks (Impressed pool) |  |  |  |
| --- | --- | --- | --- | --- | --- |
|  | Recommendation (query free) | Search | Search |  |  |
| Configuration | Click NDCG | Purchase NDCG | R@300 | MRR | R@1 |
| Full model (trained with all parameters) |  |  |  |  |  |
| Full model (evaluated via Ranking head) | – | – | – | 58.20 | 33.84 |
| Full model (evaluated via Retrieval head) | 48.79 | 49.44 | 72.96 | 57.30 | 31.57 |
| Ablations (evaluated via Retrieval head) |  |  |  |  |  |
| w/o Collaborative emb. | 48.21 | 47.44 | 61.59 | 56.68 | 30.83 |
| w/o Time-based RoPE | 48.56 | 48.12 | 70.56 | 57.05 | 31.25 |
| w/o Ranking head | 48.41 | 48.40 | 73.36 | 56.90 | 31.11 |
| w/o Target time info. | 48.66 | 49.29 | 71.36 | 57.19 | 31.54 |

### 4.2 Baselines

We compare SynerGen against three categories of baselines: (i) sequential recommendation models without search data, (ii) joint search–recommendation models, and (iii) personalized search models without recommendation data.

Sequential recommendation.
On the Book Review dataset, we consider canonical sequence models SASRec (Kang & McAuley, 2018b) and BERT4Rec (Sun et al., 2019), along with more recent architectures. HLLM (Chen et al., 2024) introduces a hierarchical encoder for item content and user history, while HSTU (Zhai et al., 2024) represents a state-of-the-art sequential transducer for large-scale streaming data.

Joint search and recommendation.
On the eBook Search Sessions dataset with synthetic queries, we include SESRec (Si et al., 2023), which leverages search interactions to improve recommendation; USER (Yao et al., 2021), which integrates queries and browsing into a single heterogeneous sequence; UnifiedSSR (Xie et al., 2023), which learns a shared representation of user history across both domains; and UniSAR (Shi et al., 2024), which models fine-grained transitions between search and recommendation via contrastive alignment and cross-attention fusion.

Contextualized search.
We also compare with TEM (Bi et al., 2020), which incorporates user preferences and context for relevance estimation, and CoPPS (Dai et al., 2023), which aligns user intent with search results using contrastive learning.

### 4.3 Implementation and Training Details

SynerGen is trained on 16 NVIDIA A100 GPUs with mixed precision. We use AdamW with learning rates of $0.001$ (dense) and $0.003$ (sparse), InfoNCE temperature $\tau=0.085$, and loss weights $\alpha=0.1$ (contrastive) and $\lambda=1$ (pairwise). Unless otherwise noted, evaluation uses the retrieval head output. Different from trending generative recommender like Chen et al. (2024), our SynerGen is not jointly trained with the pretrained encoder $\mathrm{LM}$.

Architectural and training hyperparameters are scaled to dataset size. For Book Review and eBook Search Sessions, we use a compact 100M-parameter decoder backbone; for Session-US, a larger 17B-parameter configuration is adopted, dominated by the collaborative embedding table. Additional dataset-specific settings (sequence length, negatives, RoPE granularity) for implementations are summarized in Appendix.

### 4.4 Main Results

#### 4.4.1 Recommendation

On the Book Review dataset with 686K candidate items (Table [2](https://arxiv.org/html/2509.21777v1#S4.T2)), SynerGen achieves the best or highly competitive performance across all metrics. Trained with only collaborative embedding, SynerGen-ID attains the highest Recall, demonstrating the strength of the generative backbone. HLLM-1B-Scratch achieves slightly higher NDCG@10 (4.02), indicating better ranking of the top few items, but at substantially higher computational cost. Incorporating frozen semantic embeddings further improves SynerGen, confirming the benefit of semantic knowledge. Compared to HLLM-1B, which trains a 1B-parameter language model end-to-end, our 100M-parameter SynerGen attains higher Recall and nearly identical NDCG@10 (5.45 vs. 5.65), while being far more efficient. Freezing the semantic encoder reduces training cost without sacrificing retrieval or ranking quality.

#### 4.4.2 Joint Search and Recommendation

Results on the eBook Search Sessions dataset (Table [3](https://arxiv.org/html/2509.21777v1#S4.T3)) show that SynerGen consistently outperforms baselines in both recommendation-only and query-aware search settings. It achieves the highest Recall@1 and NDCG@10, demonstrating its ability to capture user intent across modalities. Against unified models such as UnifiedSSR, UniSAR, and USER, SynerGen delivers notable gains in early precision (Recall@1) and ranking quality (NDCG), particularly in the search scenario. Moreover, it surpasses specialized search systems such as TEM and CoPPS, underscoring the effectiveness of a single generative framework that jointly supports recommendation and search.

### 4.5 Ablations

To assess the contribution of each component in SynerGen, we conduct an ablation study on the Session-US benchmark, covering both query-free recommendation and query-aware search. Results in Table [4](https://arxiv.org/html/2509.21777v1#S4.T4) address the following questions:

Q1: How do retrieval and ranking heads contribute across tasks?
Retrieval head outperforms on recommendation and recall, while ranking head is stronger on fine-grained ranking, showing their complementary roles in the full model.

Q2: Are collaborative embeddings necessary?
Yes. Removing them and unfreezing the semantic encoder to compensate reduces search Recall@300 by over 11%. Collaborative signals from user–item interactions are therefore indispensable, while keeping the semantic encoder frozen preserves pretrained semantic knowledge as a regularizer.

Q3: Does temporal modeling beyond sequential order matter?
Yes. Replacing time-aware RoPE with standard sequential encoding consistently hurts performance, showing that explicit timestamp modeling provides richer temporal context and better captures session dynamics.

Q4: Is training ranking head essential?
Yes. Eliminating it slightly increases recall but sharply degrades ranking quality. The ranking head improves precision, regularizes retrieval representations, and independently achieves the best ranking results, validating our joint training design.

Q5: Do target timestamps provide meaningful supervision?
Yes, albeit modestly. Removing them causes small but consistent drops in ranking metrics, indicating that target-aware temporal cues aid disambiguation, especially in sparse sessions.

These ablations confirm that collaborative embeddings, frozen semantic encoders, temporal modeling, and joint retrieval–ranking optimization are complementary, each contributing to balanced recall and precision across search and recommendation.

## 5 Conclusion

We presented SynerGen, a single generative backbone for unified personalized search and recommendation that jointly optimizes retrieval and ranking within a decoder-only Transformer architecture. Extensive experiments on public and large-scale industrial datasets demonstrate that SynerGen consistently outperforms strong sequential recommenders, unified search–recommendation models, and personalized search baselines, achieving state-of-the-art recall and ranking quality in both query-free and query-aware settings. Ablation studies confirm that each architectural component contributes to this performance, with collaborative embeddings, temporal modeling, and multi-head training proving particularly impactful. Future work includes extending the framework to multimodal inputs and exploring adaptive retrieval–ranking trade-offs for even more efficient deployment.

## References

- Ai et al. (2017)
Qingyao Ai, Yongfeng Zhang, Keping Bi, Xu Chen, and W. Bruce Croft.
Learning a hierarchical embedding model for personalized product search.
In *Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval*, SIGIR ’17, pp. 645–654, New York, NY, USA, 2017. Association for Computing Machinery.
ISBN 9781450350228.
doi: 10.1145/3077136.3080813.
URL [https://doi.org/10.1145/3077136.3080813](https://doi.org/10.1145/3077136.3080813).
- Ai et al. (2019)
Qingyao Ai, Daniel N. Hill, S. V. N. Vishwanathan, and W. Bruce Croft.
A zero attention model for personalized product search.
In *Proceedings of the 28th ACM International Conference on Information and Knowledge Management*, CIKM ’19, pp. 379–388. ACM, November 2019.
doi: 10.1145/3357384.3357980.
URL [http://dx.doi.org/10.1145/3357384.3357980](http://dx.doi.org/10.1145/3357384.3357980).
- BehnamGhader et al. (2024)
Parishad BehnamGhader, Vaibhav Adlakha, Marius Mosbach, Dzmitry Bahdanau, Nicolas Chapados, and Siva Reddy.
LLM2vec: Large language models are secretly powerful text encoders.
In *First Conference on Language Modeling*, 2024.
URL [https://openreview.net/forum?id=IW1PR7vEBf](https://openreview.net/forum?id=IW1PR7vEBf).
- Bi et al. (2020)
Keping Bi, Qingyao Ai, and W. Bruce Croft.
A transformer-based embedding model for personalized product search.
In *Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval*, SIGIR ’20, pp. 1521–1524. ACM, July 2020.
doi: 10.1145/3397271.3401192.
URL [http://dx.doi.org/10.1145/3397271.3401192](http://dx.doi.org/10.1145/3397271.3401192).
- Chen et al. (2024)
Junyi Chen, Lu Chi, Bingyue Peng, and Zehuan Yuan.
Hllm: Enhancing sequential recommendations via hierarchical large language models for item and user modeling, 2024.
URL [https://arxiv.org/abs/2409.12740](https://arxiv.org/abs/2409.12740).
- Covington et al. (2016)
Paul Covington, Jay Adams, and Emre Sargin.
Deep neural networks for youtube recommendations.
In *Proceedings of the 10th ACM conference on recommender systems*, pp. 191–198, 2016.
- Dai et al. (2023)
Shitong Dai, Jiongnan Liu, Zhicheng Dou, Haonan Wang, Lin Liu, Bo Long, and Ji-Rong Wen.
Contrastive learning for user sequence representation in personalized product search.
In *Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining*, KDD ’23, pp. 380–389, New York, NY, USA, 2023. Association for Computing Machinery.
ISBN 9798400701030.
doi: 10.1145/3580305.3599287.
URL [https://doi.org/10.1145/3580305.3599287](https://doi.org/10.1145/3580305.3599287).
- Deng et al. (2025)
Jiaxin Deng, Shiyao Wang, Kuo Cai, Lejian Ren, Qigen Hu, Weifeng Ding, Qiang Luo, and Guorui Zhou.
Onerec: Unifying retrieve and rank with generative recommender and iterative preference alignment.
*arXiv preprint arXiv:2502.18965*, 2025.
- Han et al. (2025)
Ruidong Han, Bin Yin, Shangyu Chen, He Jiang, Fei Jiang, Xiang Li, Chi Ma, Mincong Huang, Xiaoguang Li, Chunzhen Jing, et al.
Mtgr: Industrial-scale generative recommendation framework in meituan.
*arXiv preprint arXiv:2505.18654*, 2025.
- Huang et al. (2025)
Yanhua Huang, Yuqi Chen, Xiong Cao, Rui Yang, Mingliang Qi, Yinghao Zhu, Qingchang Han, Yaowei Liu, Zhaoyu Liu, Xuefeng Yao, et al.
Towards large-scale generative ranking.
*arXiv preprint arXiv:2505.04180*, 2025.
- Kang & McAuley (2018a)
Wang-Cheng Kang and Julian McAuley.
Self-attentive sequential recommendation.
*2018 IEEE International Conference on Data Mining (ICDM)*, pp. 197–206, 2018a.
URL [https://api.semanticscholar.org/CorpusID:52127932](https://api.semanticscholar.org/CorpusID:52127932).
- Kang & McAuley (2018b)
Wang-Cheng Kang and Julian McAuley.
Self-attentive sequential recommendation, 2018b.
URL [https://arxiv.org/abs/1808.09781](https://arxiv.org/abs/1808.09781).
- Li et al. (2024)
Mingming Li, Huimu Wang, Zuxu Chen, Guangtao Nie, Yiming Qiu, Guoyu Tang, Lin Liu, and Jingwei Zhuo.
Generative retrieval with preference optimization for e-commerce search.
*arXiv preprint arXiv:2407.19829*, 2024.
- Linden et al. (2003)
Greg Linden, Brent Smith, and Jeremy York.
Amazon. com recommendations: Item-to-item collaborative filtering.
*IEEE Internet computing*, 7(1):76–80, 2003.
- Liu et al. (2024)
Jinhan Liu, Qiyu Chen, Junjie Xu, Junjie Li, Baoli Li, and Sulong Xu.
A unified search and recommendation framework based on multi-scenario learning for ranking in e-commerce.
*Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval*, 2024.
URL [https://api.semanticscholar.org/CorpusID:269899798](https://api.semanticscholar.org/CorpusID:269899798).
- Makhneva et al. (2023)
Elizaveta Makhneva, Anna Sverkunova, Oleg Lashinin, Marina Ananyeva, and Sergey Kolesnikov.
Make your next item recommendation model time sensitive.
*Adjunct Proceedings of the 31st ACM Conference on User Modeling, Adaptation and Personalization*, 2023.
URL [https://api.semanticscholar.org/CorpusID:259178180](https://api.semanticscholar.org/CorpusID:259178180).
- Oord et al. (2018)
Aaron van den Oord, Yazhe Li, and Oriol Vinyals.
Representation learning with contrastive predictive coding.
*arXiv preprint arXiv:1807.03748*, 2018.
- Robertson et al. (2009)
Stephen Robertson, Hugo Zaragoza, et al.
The probabilistic relevance framework: Bm25 and beyond.
*Foundations and Trends® in Information Retrieval*, 3(4):333–389, 2009.
- Shi et al. (2024)
Teng Shi, Zihua Si, Jun Xu, Xiao Zhang, Xiaoxue Zang, Kai Zheng, Dewei Leng, Yanan Niu, and Yang Song.
Unisar: Modeling user transition behaviors between search and recommendation.
*Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval*, 2024.
URL [https://api.semanticscholar.org/CorpusID:269149624](https://api.semanticscholar.org/CorpusID:269149624).
- Shi et al. (2025)
Teng Shi, Jun Xu, Xiao Zhang, Xiaoxue Zang, Kai Zheng, Yang Song, and Enyun Yu.
Unified generative search and recommendation.
*arXiv preprint arXiv:2504.05730*, 2025.
- Si et al. (2023)
Zihua Si, Zhongxiang Sun, Xiao Zhang, Jun Xu, Xiaoxue Zang, Yang Song, Kun Gai, and Ji-Rong Wen.
When search meets recommendation: Learning disentangled search representation for recommendation.
In *Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval*, SIGIR ’23, pp. 1313–1323. ACM, July 2023.
doi: 10.1145/3539618.3591786.
URL [http://dx.doi.org/10.1145/3539618.3591786](http://dx.doi.org/10.1145/3539618.3591786).
- Su et al. (2021)
Jianlin Su, Yu Lu, Shengfeng Pan, Ahmed Murtadha, Bo Wen, and Yunfeng Liu.
Roformer: enhanced transformer with rotary position embedding. arxiv.
*arXiv preprint arXiv:2104.09864*, 2021.
- Sun et al. (2019)
Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang.
Bert4rec: Sequential recommendation with bidirectional encoder representations from transformer.
*Proceedings of the 28th ACM International Conference on Information and Knowledge Management*, 2019.
URL [https://api.semanticscholar.org/CorpusID:119181611](https://api.semanticscholar.org/CorpusID:119181611).
- Touvron et al. (2023)
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample.
Llama: Open and efficient foundation language models, 2023.
URL [https://arxiv.org/abs/2302.13971](https://arxiv.org/abs/2302.13971).
- Vaswani et al. (2017)
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin.
Attention is all you need.
*Advances in neural information processing systems*, 30, 2017.
- Wang et al. (2023)
Yu Wang, Zhengyang Wang, Hengrui Zhang, Qingyu Yin, Xianfeng Tang, Yinghan Wang, Danqing Zhang, Limeng Cui, Monica Cheng, Bing Yin, Suhang Wang, and Philip S. Yu.
Exploiting intent evolution in e-commercial query recommendation.
In *Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining*, KDD ’23, pp. 5162–5173, New York, NY, USA, 2023. Association for Computing Machinery.
ISBN 9798400701030.
doi: 10.1145/3580305.3599821.
URL [https://doi.org/10.1145/3580305.3599821](https://doi.org/10.1145/3580305.3599821).
- Xie et al. (2023)
Jiayi Xie, Shang Liu, Gao Cong, and Zhenzhong Chen.
Unifiedssr: A unified framework of sequential search and recommendation, 2023.
URL [https://arxiv.org/abs/2310.13921](https://arxiv.org/abs/2310.13921).
- Yao et al. (2021)
Jing Yao, Zhicheng Dou, Ruobing Xie, Yanxiong Lu, Zhiping Wang, and Ji-Rong Wen.
User: A unified information search and recommendation model based on integrated behavior sequence.
In *Proceedings of the 30th ACM International Conference on Information &; Knowledge Management*, CIKM ’21, pp. 2373–2382. ACM, October 2021.
doi: 10.1145/3459637.3482489.
URL [http://dx.doi.org/10.1145/3459637.3482489](http://dx.doi.org/10.1145/3459637.3482489).
- Zhai et al. (2024)
Jiaqi Zhai, Lucy Liao, Xing Liu, Yueming Wang, Rui Li, Xuan Cao, Leon Gao, Zhaojie Gong, Fangda Gu, Michael He, et al.
Actions speak louder than words: Trillion-parameter sequential transducers for generative recommendations.
*arXiv preprint arXiv:2402.17152*, 2024.
- Zhang et al. (2025)
Caojin Zhang, Qiang Zhang, Ke Li, Sai Vidyaranya Nuthalapati, Benyu Zhang, Jason Liu, Serena Li, Lizhu Zhang, and Xiangjun Fan.
Gem: Empowering llm for both embedding generation and language understanding, 2025.
URL [https://arxiv.org/abs/2506.04344](https://arxiv.org/abs/2506.04344).
- Zhao et al. (2025)
Jujia Zhao, Wenjie Wang, Chen Xu, Xiuyi Chen, Zhaochun Ren, and Suzan Verberne.
Unifying search and recommendation: A generative paradigm inspired by information theory.
*ArXiv*, abs/2504.06714, 2025.
URL [https://api.semanticscholar.org/CorpusID:277634440](https://api.semanticscholar.org/CorpusID:277634440).

## Appendix A Implementation Details

### A.1 Dataset Statistics

We present the statistics for each dataset in Table [5](https://arxiv.org/html/2509.21777v1#A1.T5).

**Table 5: Statistics of Benchmark Datasets**
| Dataset | #Users | #Items | #Queries | #Action-Search | #Action-Rank |
| --- | --- | --- | --- | --- | --- |
| Session-US | 155,378,774 | 252,000,000 | - | 87,851,713,040 | - |
| Books | 694,898 | 686,624 | - | - | 10,053,086 |
| Kindle Store | 68,223 | 61,934 | 4,298 | 934,664 | 989,618 |

### A.2 Training Configurations

We provides dataset-specific training configurations for SynerGen in Table  [6](https://arxiv.org/html/2509.21777v1#A1.T6). For all three models, we are using 256-dimensional query embedding $e^{\mathrm{query}}$, 128-dimensional item embeddings for both $e^{\mathrm{item}_{s}}$ and $e^{\mathrm{item}_{c}}$.

**Table 6: Dataset-specific hyperparameters for SynerGen.**
| Dataset | Layers | Hidden dim | Negatives | RoPE bucket |
| --- | --- | --- | --- | --- |
| Book Review SynerGen | 4 | 128 | 26K in-batch | 24h |
| eBook Search Sessions | 4 | 128 | 26K in-batch | 24h |
| Session-US | 8 | 256 | 16K + 6 hard | 1s |

## Appendix B RoPE-based Temporal Attention

##### Definition

Formally, given a token embedding $x_{m}$ at timestamp $m$, query and key vectors are defined as

$$ $q_{m}=R_{\Theta,m}W_{q}x_{m},\quad k_{n}=R_{\Theta,n}W_{k}x_{n},$ $$

where $W_{q}$ and $W_{k}$ are projection matrices, and $R_{\Theta,t}$ is a block-diagonal rotation matrix parameterized by time $t$. The attention score between events at times $m$ and $n$ becomes

$$ $\langle q_{m},k_{n}\rangle=x_{m}^{T}W_{q}^{T}R_{\Theta,m-n}W_{k}x_{n},$ $$

which depends only on the relative time gap $m-n$, ensuring shift invariance.

##### Worked example

Consider the sequence {ASIN1 (01/01/2024), ASIN2 (02/01/2024), ASIN3 (03/01/2024)}. Their Unix timestamps are {1704096000, 1706774400, 1709280000}, and their rotary embeddings are $R_{\Theta,1704096000}$, $R_{\Theta,1706774400}$, and $R_{\Theta,1709280000}$, respectively. In self-attention between ASIN1 and ASIN2, the model applies

$$ $R_{\Theta,1706774400-1704096000}=R_{\Theta,2678400},$ $$

where 2,678,400 seconds is the gap between the two events. Thus, temporal distance is directly encoded as a rotation.

##### Effective context length

If we allow a one-year history at second-level resolution, the maximum relative gap is $\sim$32M. To make training feasible, we shorten history duration (e.g., three months $\sim$780K) or bucket timestamps at coarser granularity (e.g., one minute, reducing one year to $\sim$500K). Combining both yields an effective context length of $\sim$125K. Following Su et al. (2021), the rotation base is then set accordingly (e.g., $\sim$7.8e6 for 125K), ensuring robust coverage of temporal gaps.

## Appendix C Design of Masking Matrix

In Section [3.4](https://arxiv.org/html/2509.21777v1#S3.SS4), we introduced the task-specific masking matrix $M$ that governs attention flow among context, retrieval, and ranking tokens. Here, we provide the exact mathematical specification.

Given queries $Q$, keys $K$, and values $V$, attention is computed as:

$$ $\text{Attn}(Q,K,V)=\text{softmax}\!\left(\frac{QK^{\top}\odot M}{\sqrt{d_{k}}}\right)V,$ $$

where $M\in\{0,1\}^{n\times n}$ is the binary mask.

Formally,

$$ $M_{i,j}=\begin{cases}1&\text{if }\text{ValidAttn}(i,j),\\ 0&\text{otherwise},\end{cases}$ $$

with the validity function $\text{ValidAttn}(i,j)$ defined as:

$$ $\begin{split}\text{ValidAttn}(i,j)={}&(\text{context}_{i}\land\text{context}_{j}\land t_{i}>t_{j})\\ &\lor(\text{retrieval}_{i}\land\text{context}_{j}\land\text{req_group}_{i}>\text{req_group}_{j})\\ &\lor(\text{ranking}_{i}\land\text{context}_{j}\land\text{req_group}_{i}>\text{req_group}_{j})\\ &\lor(\text{ranking}_{i}\land\text{retrieval}_{j}\land\text{event_id}_{i}=\text{event_id}_{j}).\end{split}$ (1) $$

The four terms correspond to the following constraints:
1. Temporal causality: context tokens can only attend to earlier context tokens.
2. Session isolation: retrieval tokens may only attend to context tokens from earlier request groups.
3. Ranking supervision: ranking tokens may only attend to context tokens from earlier request groups.
4. Cross-task alignment: ranking tokens may attend to retrieval tokens that share the same event identifier.

Additionally, during training, a stochastic temporal gap $\theta$ is applied, requiring $t_{i}>t_{j}+\theta$ for certain attention edges. This encourages the model to capture long-term dependencies beyond immediate history.

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 4
presentation: 3
questions: Can the authors provide a direct comparison between their unified model and a retrieval-only (or a ranking-only) baseline? This would help clarify the benefit of joint optimization over task-specific alternatives.
soundness: 3
strengths: S1: Practical relevance - addresses the important problem of unifying search and recommendation pipelines together with retrieval and ranking tasks with a clean, elegant solution.

S2: Comprehensive evaluation on academic datasets. The authors conduct tests on Book Review and eBook Search Sessions (synthetic dataset) with diverse baselines.

S3: Strong Empirical Results on Academic Benchmarks. SynerGen achieves improvements on Book Review and eBook Search Sessions datasets compared to existing baselines.
summary: This paper introduces SynerGen, a decoder-only model that unifies retrieval and ranking tasks for both personalized search and recommendation. The authors address the limitation that existing generative recommender systems typically handle either query-aware search or query-free recommendation, but not both, by introducing task-specific tokens (retrieval, context, ranking). SynerGen employs joint optimization with InfoNCE loss for retrieval and a hybrid pointwise-pairwise loss for ranking, while incorporating time-aware RoPE for temporal dynamics.
weaknesses: W1: Missing clearer justifications for why the proposed approach is superior. It's not clearly justified why the combined ranking or retrieval model outperforms a retrieval-only model only (with eg a pairwise auxiliary loss). For instance, what would be the retrieval NDCG in Table 4 obtained using a retrieval-loss only model?

W2: Missing contextualization wrt recent work, e.g., Zhang et al. Killing Two Birds with One Stone: Unifying Retrieval and Ranking with a Single Generative Recommendation Model. SIGIR’25.

W3. Experiments
a/ Missing baselines. While the paper conducts thorough experiments for academic datasets, there's no baseline for Session-US.
b/ eBook Search Sessions is synthetc.
c/ it's not clear if all models have been calibrated to have similar params/FLOPS budget (eg SynerGen uses 100M params, which AFAIK is a large model for the Amazon Books dataset). This should be clarified directly in the paper but isn't done consistently; eg HLLM-1B has parameter count, but others don't. 

W4. The paper utilizes temporal RoPE and claims to have special mathematical formulation discussed the Appendix. However the submission doesn't have an Appendix.

---

confidence: 5
contribution: 2
initial_rating: 2
presentation: 2
questions: 1. How does SynerGen perform in real-world industrial production environments compared to comprehensive benchmark tests?

2. What is the impact of freezing semantic embeddings on the model's flexibility?

3. How does SynerGen handle cold-start problems for new users or items?
soundness: 3
strengths: 1.  The paper presents a novel generative recommender system (SynerGen) that unifies personalized search and recommendation. It addresses the challenge of integrating these two traditionally separate systems into a single framework, improving both efficiency and performance.

2. The model jointly optimizes retrieval and ranking tasks within the same architecture, making it computationally efficient while maintaining high performance.

3. The model shows impressive results on multiple benchmark datasets.
summary: This paper proposes a generative recommender model, which can provide a single generative backbone for both personalized search and recommendation. Their decoder-only transformer can handle both retrieval and ranking tasks simultaneously. They demonstrated its performance using real-world datasets and compared it with other methods.
weaknesses: 1. While SynerGen is shown to be more efficient than some large-scale models, it still requires significant computational resources, especially for training with larger datasets. This could limit its practical deployment for smaller companies or those with limited computational power.

2. Some sections of the paper, particularly the model architecture and ablation analysis, are highly technical and may be challenging for a broader audience. A simplified explanation of key innovations or a visual overview could make the paper more accessible to non-experts.

3. The paper compares with some baselines, but it is still insufficient; more GRMs should be added for comparison.

4. It seems that instructions for using LLM are missing.

---

confidence: 2
contribution: 3
initial_rating: 4
presentation: 3
questions: 1.Only the last interaction is used as label; intermediate signals in long sequences are ignored.

2.Ranking still relies on random negatives; harder adversarial negatives are not explored.

3.Absolute timestamp shifts may hurt RoPE; cross-domain generalization is not examined.

4.Task-token order is fixed; impact of different arrangements is not studied.

5.No latency–accuracy comparison with lightweight two-tower baselines.

6.Ablations remove components but do not analyze sensitivity of loss weights.

7.Performance under varying user-activity levels is not reported, leaving sparse-scene evidence incomplete.
soundness: 2
strengths: 1. It is the first to achieve dual unification of "query-aware search-query-free recommendation" and "retrieval-ranking" under a decoder-only architecture, solving the problem of single-task limitations or performance trade-offs in existing models. As shown in Table 1, SynerGen is the only model that supports dual modes without performance trade-offs

2. It combines frozen pre-trained semantic embeddings (capturing item/query semantics) with optimizable collaborative embeddings (capturing user-item interactions), balancing generalization ability and long-tail item handling. In experiments, SynerGen (with semantic embeddings) outperformed SynerGen-ID (using only collaborative embeddings)
summary: This paper proposes SynerGen, a decoder-only Transformer-based generative recommendation model aimed at unifying personalized search and query-free recommendation. It jointly optimizes the InfoNCE retrieval loss and hybrid pointwise-pairwise ranking loss, integrates semantic, collaborative, and temporal signals, and introduces a time-aware Rotary Positional Embedding (RoPE) to capture temporal dependencies. Experiments were conducted on the Book Review, eBook Search Sessions, and Session-US datasets. SynerGen outperformed existing baselines in metrics such as Recall@K (R@K) and NDCG, verifying the feasibility of a single generative backbone for industrial-scale information access.
weaknesses: 1. The paper proposes "context/retrieval/ranking tokens" to enable task switching, but the ablation study does not compare the scheme of "no task tokens + unified encoding." It is impossible to rule out the possibility that "task unification can be achieved without dedicated tokens," and the irreplaceability of the token design lacks support 

2. The experimental datasets are concentrated in "book reviews" and "e-commerce search." Although the paper mentions that existing models are applied in fields such as "short videos and food delivery," SynerGen has not verified its performance in these fields. It is impossible to confirm whether the model is suitable for recommendation scenarios dominated by non-text data

---

confidence: 4
contribution: 2
initial_rating: 4
presentation: 3
questions: See Weaknesses.
soundness: 2
strengths: - The proposed SynerGen is versatile and can perform retrieval and ranking in a single model, with support for query-aware and query-free settings.

- The results show strong performance for the proposed method.
summary: This work aims to unify the previously segregated retrieval and ranking processes in recommender systems by proposing SynerGen, which jointly learns to retrieve and rank and supports both query-free and query-aware recommendation. To this end the authors propose a Transformer-based architecture that uses different heads for retrieval and ranking. The former is trained using InfoNCE and the latter using a pairwise ranking loss. Furthermore, the authors use RoPE to incorporate time dependency in self-attention. The shown experiments demonstrate that SynerGen usually attains the best performance compared to several baselines.
weaknesses: **Novelty**

The authors claim that SynerGen is the first instance of a generative model that supports query-aware and query-free recommendation. This is not true, as there is prior work that supports query-aware and query-free recommendation, as mentioned in their related work section, e.g. UniSAR and GenSAR.
Moreover, the second contribution of unifying retrieval and ranking has also been explored in prior work that was not cited [1].
Finally, the application of RoPE as well as task aware embeddings are not novel, see [2][3]
Therefore, I recommend to carve out the contributions more precisely and convey them properly, for example the training paradigm may be a contribution combining ranking and retrieval losses.

[1]  Unifying Generative and Dense Retrieval for Sequential Recommendation, Yang et al., TMLR 2025

[2]  Recommender Systems with Generative Retrieval, Rajput et al., NeurIPS 2023

[3]  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, Devlin et al., NAACL 2019

**Scalability**

In practice, recommender systems must deal with extremely large item and user sets while delivering decent performance efficiently. In line 245 the authors state that retrieval tokens score the entire catalog. This means that the cost of ranking scales linearly with the number of items, which gets expensive for large scale item sets. This is the reason why generative retrieval has been introduced, as it does not rely on scanning the entire item set. As scalability is imperative when it comes to recommendation systems, I strongly recommend to report inference times and memory requirements for SynerGen and competitors for the different datasets.

**Lack of comparison to generative baselines**

The authors claim that SynerGen is a generative model, but the experiments lack comparisons to generative baselines like TIGER [1], Mender [2],  or LiGER [3].

[1]  Recommender Systems with Generative Retrieval, Rajput et al., NeurIPS 2023

[2]  Preference Discerning with LLM-Enhanced Generative Retrieval, Paischer et al., TMLR 2025

[3]  Unifying Generative and Dense Retrieval for Sequential Recommendation, Yang et al., TMLR 2025

 
**Significance of results**

The authors did not include error bars which makes it difficult to judge the significance of the reported results. The gap to baselines is often marginal (Table 2 & Table 4), especially when looking at ablation studies. I highly recommend to also report error bars and ideally test for statistical significance.

**Presentation**

The authors present SynerGen as a generative model, however in line 245/246 the authors say that retrieval tokens store the entire catalog. This is confusing as if it was a truly generative approach, the recommendation set could simply be generated by the model as in [1], instead it is used in a discriminative manner. 
Also there is a claim on SynerGen being a generative foundation model (line 61), but there is no supporting evidence that it actually follows the definition of a foundation model [2].
Finally, overclaiming in lines 442 and 446, TEM is better than SynerGen for R@10 in Table 3.

[1]  Recommender Systems with Generative Retrieval, Rajput et al., NeurIPS 2023

[2] On the opportunities and risks of foundation models, Bommasani et al., arXiv 2021