Fine Tuning Out-of-Vocabulary Item Recommendation with User Sequence Imagination
Authors: Ruochen Liu
Hao Chen
Yuanchen Bei
Qijie Shen
Fangwei Zhong
Senzhang Wang
Jianxin Wang
Venue: NeurIPS 2024 spotlight (2024)
Keywords: Recommendation; Reinforcement Learning;
Primary area: other

Abstract: Recommending out-of-vocabulary (OOV) items is a challenging problem since the in-vocabulary (IV) items have well-trained behavioral embeddings but the OOV items only have content features. Current OOV recommendation models often generate 'makeshift' embeddings for OOV items from content features and then jointly recommend with the `makeshift' OOV item embeddings and the behavioral IV item embeddings. However, merely using the 'makeshift' embedding will result in suboptimal recommendation performance due to the substantial gap between the content feature and the behavioral embeddings. To bridge the gap, we propose a novel **User Sequence IMagination (USIM)** fine-tuning framework, which first imagines the user sequences and then refines the generated OOV embeddings with the user behavioral embeddings. Specifically, we frame the user sequence imagination as a reinforcement learning problem and develop a recommendation-focused reward function to evaluate to what extent a user can help recommend the OOV items. Besides, we propose an embedding-driven transition function to model the embedding transition after imaging a user. USIM has been deployed on a prominent e-commerce platform for months, offering recommendations for millions of OOV items and billions of users. Extensive experiments demonstrate that USIM outperforms traditional generative models in OOV item recommendation performance across traditional collaborative filtering and GNN-based collaborative filtering models.

Fine-Tuning Out-of-Vocabulayy Item Recommendation with User Sequence Imagination
Ruochen Liu', Hao Chen', Yuanchen Bei Qijie Shen', Fangwei Zhong Senzhang Wang" Jianxin Wang* 'Central South University, *City University of Macau, *Zhejiang University, 'Alibaba Group, 'Bejjig Normal University {ruochen, szwang; jxwang]&csu, edu. sundaychenhao@@gaal.. com yuanchenbei@zju edu. en, gjshenxdudgmail. com, fangveizhong@bnu, edu. cn
Abstract
(OOV) items is challenging problem since the in-vocabulary (IV) items have well-trained behavioral embeddings but the OOV items only have content features. Current OOV recommendation models often generate 'makeshift' embeddings for OOV items from content features and then jointly recommend with the 'makeshift' OOV item embeddings and the behavioral IV item embeddings. However, merely using the 'makeshift' embedding will result in suboptimal recommendation performance due the substantial gap between the content feature and the behavioral embeddings. To bridge the gap, we propose a novel User Sequence IMagination (USIM) fine-tuning framework, which first imagines the user sequences and then refines the generated OOV embeddings with the user behavioral embeddings. Specifically, we frame the user sequence imagination as reinforcement learning problem and develop a recommendation- focused reward function evaluate what extent a user can help recommend the OOV items. Besides, we propose embedding driven transition function to model the embedding transition after imaging user. USIM has been deployed prominent e-commerce platform for months, offering recommendations for millions of OOV items and billions of users. Extensive experiments demonstrate that USIM outperforms traditional generative models in OOV item recommendation performance across traditional collaborative filtering and GNN-based collaborative filtering models.
Recommending
Introduction
Recommendation systems, such as collaborative filtering models, learn behavioral embeddings from historical interactions represent the behavioral characteristics of billions of users and items 114 For instance, the embeddings of interacted user-item pairs have higher inner products, whereas those of un-interacted pairs have lower inner products. However, besides the items with user interactions, thousands of -oo-bl-vocabulary (OOV) items-such as short videos, photos, and posts-are generated or uploaded every second. In the age of AGI, the generation speed of Al-made OOV content, including text, images, and videos, will far exceed the speed of human consumption. To avoid being overwhelmed by the OOV content, it is essential to replicate how humans handle these items, recommending them without disrupting in-vocabulary (IV) items.
Traditional OOV recommendation models usually generate 'makeshift' embeddings from the content features and then use them to recommend OOV items. The research can be classified into two categories. (a) Generative models aim generate realistic embeddings. GAR [ uses generative
"Conresonding author
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

(a) 'Makeshift' Embedding
(b) User Sequence Imagination
Interacted
Imagined Users
Users
Generator
Generator
Optimization
Content Generated
Well-trained
Content
Initialized
Imagined Optimized Embedding
Feature Embedding
Embedding
Feature
Embedding
Figure 1: Comparison between (a) traditional 'makeshift' embedding OOV recommendation frame- work, and (b) user sequence imagination OOV recommendation framework.
adversarial structure to ensure the embedding distribution of generated OOV embeddings similar to IV embeddings. ALDI 6 distills knowledge from IV items to OOV items. (b) Dropout models
increase the robustness of recommender systems. Dropout 7 randomly substitutes IV embeddings with "makeshift" ones to enhance system robustness. Heater [8] and CLCRec [9] further utilize mix of experts and contrastive learning techniques to improve OOV recommendation performance.
As shown the left of Figure current recommender systems typically learn the embeddings for any given items by initializing/ gennrratingg the embedding and then optimizing them through user- sequence backpropagation. However, prevalent OOV recommendation models primarily concentrate on generating improved or robust embeddings, overlooking the potential for further optimization from imagining user sequences. This oversight limits these models due to the following issucs. Content-Behawe Gap. 'Makeshift' embeddings are generated from content features, while behavioral embeddings trained using backpropagation. The substantial difference between content features and behavioral embeddings may lead to discrepancies between IV and OOV items, impacting IV, OOV, both. Potential Suboptimality. Focusing only embedding generation overlooks potential improvements from backpropagation, which could finely tune the embeddings to adapt to user preferences and current recommender systems, possibly leading to suboptimal recommendation performance and reduction in revenue.
While imagining the sequential optimization process for OOV item embeddings shows promise, its implementation presents three challenges. 1. Absence of Historical Interactions. The lack of historical user interactions for OOV item embeddings hinders the definition of clear backpropagation and optimization process. 2. Ambiguous Imagination Objectives. Formulating objectives, stopping criteria, and user selection for imagining OOV item interactions remains open challenge. 3. Navigating the Vast User Space. Efficiently identifying suitable user sequences to imagine for OOV items within the massive user space of recommender systems.
To address these challenges, we introduce an User Sequence Imagination (USIM) pipeline that further optimizes the embedding of OOV items. It first imagines potential users who may interact with the OOV item and then refines the item embeddings through backpropagation. Specifically, we pro- pose RL-based USIM solution, which formulates the sequential optimization a Markov Decision Process and introduces recommender-orieneee PPO (RecPPO) to maximize the final recommendation performance of OOV items. In summary, contributions are follows:
We introduce USIM, a novel approach that fundamentally addresses the Out-of- Vocabulary (OOV) problem by imagining user sequences and performing user-sequence backpropagation.
We formally define the Reinforcement Learning (RL) formulation of USIM, including the formula- tion of the Markov Decision Process (MDP) for sequence imagination, and provide the state transition function framework for user-sequence backpropagation.
We implement USIM on major e-commerce platform Allibbb, successfully optimizing millions of OOV items and recommending them to billions of users. The source code is publicly available at /nttps://githul com/ wwt./ooo//eeeeeeeeeeee htt////ooceeeo////nneee
We validate the effectiveness our approach on two benchmark datasets using both traditional collaborative filtering and graph-based collaborative filtering backbones. Extensive experiments demonstrate that USIM outperforms existing state-of-the-art OOV recommendation models in terms of OOV recommendation performance and overall recommendation quality.

---

2 Preliminaries
Notations. Let U and I denote the sets of users and items, respectively. We partition I into in-vocabulary (IV) items Z, (items with interaction history) and out-of-vocabulary (OOV) items Tooo (items without interaction history). We denote the cardinality of these sets as [24], |Tzol and [oool respectively. For clarity the following discussion, we also use U to represent the set of users who have interacted with item i.
The embedding matrices for users, IV items, and OOV items are denoted as E, RR R Eie Rlz./xdd and Eoor Rl xd respectively, where d represents the embedding dimension. For an individual user and item i, their corresponding embeddings are denoted as ea E R and e; E IR Since OOV items lack behavioral embeddings initially, we leverage content features, denoting the content feature vector of item i as
Backpropagated Embedding of IV Items. The embeddings of TV items initialized using stan- dard initialization techniques such as Xavier initialization [1O]. These embeddings are subsequently optimized through backpropagation using historical interaction data [
Σ VL(u, NEL
=
(1)
where L represents the loss function (e.g., BPR loss).
Makeshift Embedding OOV Items. Due to the absence of historical interactions for OOV items, various approaches have been proposed to generate makeshift embeddings that enable joint recommendation with items. These approaches can be broadly categorized into generative models [6] 12 and dropout models [9] 7], which transform content features c; into embeddings through a generator function G.
e; = G(c);
where G is optimized using various objective functions, including similarity- -bssed losses [2] adversarial losses (5), and knowledge distillation losses [6].
MDP Formulation Back Propagation. We formulate the OOV embedding optimization process Markov Decision Process (MDP) narrow the optimization gap between IV and OOV item embeddings. This formulation enables simultancous user imagination and embedding optimization through backpropagation.
MDP at time step t defined by the quintuple (S,A, R, 7), where:
S represents the state space, with each state 8 E S capturing the environment configuration A denotes the action space, where each action a E A represents a possible agent decision
p: x A + S defines the state transition function, with 8,+1 p(s., a.)
R: S A IR specifies the reward function, where r, R(s,,a) a.)
y E [0, represents the discount factor for future rewards
3 Proposed User Sequence Imagination Model
3.1 Framework Overview
To bridge the fundamental gap between IV and OOV item embedding generation caused by disparate interaction histories, we propose USIM, which fine-tunes OOV item embedding by imagining appropriate user sequences. Specifically, in each step, USIM simulates user interaction and optimizes the item embedding accordingly. We formulate this process within reinforcement learning paradigm, illustrated in Figure
3

---

Exploration Set Construction
State Transition Function
Generator
Postive Interaation
Random Action
Teminnatio cction
Optimize with B Backpropagation
Content Feature
Initialized
Action
X
Embeddding)
Sc+1
Agent
Countdown
Reward Function
Interacted User
Emb. Aligmment Reward
Rec. Performance Reward
Imagined User
[tem
Termination
27
-2
The overview framework of USIM. USIM fine-tunes the generated OOV item embeddings through sequential user interaction imagination, guided by exploration set construction, state transi- tion, and a tailored reward mechanism.
Figure 2:
State Space. At time step f, the state encapsulates the essential information required for item embedding optimization, defined s; hh, 14). The state representation h, Rd which refined through optimization, resides in the same embedding space as et, and its final representation will serve as the OOV item embedding. To specifically denote the state of item at time step t. we define 8i, [ l.,)). In contrast, serves as more general denotation. The initial state presentation is obtained by the generator in Eq. 2 as h4,0 G(c)) Details can refer to Appendix B
The temporal component 4 is used as a countdown mechanism, tracking the remaining optimization steps to encourage efficient convergence [13. Given a maximum action limit N, at time step the countdown value is computed as 4 = N t.
Action Space. Given state 8t, the agent selects action a from the action space = 4 U {Gend) where aend denotes the termination action. This selection process entails either imagining a user or terminating the optimization process. The action embedding e, corresponds to the user embedding from E, when a E U, and defaults to O for the termination action.
Policy Network. The agent's decision-making process is governed by policy / (8t). which maps the current state to probability distribution over possible actions. To effectively utilize existing embeddings while accommodating the special termination action, the policy distribution given as,
( (1 oW2s + c)) Eneu exp(e, Wie))'
exp(cu, Wis))
az E U1; aa = Gends
T(ae(s:)
(W23 + +c),
where W, E Idx(d+1), W₂ E gd+1 and parameters, and represents the sigmoid function.
Reward and State Transition. The agent receives an immediate reward r. = R(st, at) after each action, guiding the optimization trajectory. To align with the imagination process, design efficient state transition function 8;+1 = p(st, dz) that facilitates the progressive refinement from content-based to interaction-based embeddings. The experience tuples (st, de, Tt, 8t+1) are collected in replay buffer for subsequent training iterations.
3.2 State Transition Function
State transition involves modifying h. which will ultimately be used the OOV item embedding and each step of the state transition corresponds to optimizing h, using the imagined user(last action a.). Therefore, to design a state transition function that aligns with this optimization process, must

---

first determine the objective of this optimization. Considering that most current recommendation algorithms calculate relevance scores between users and items for recommendations [ adopt the following objective as our optimization goal,
min E:
Euelt Fu.is
(4)
where Vu.s is predicted score between e, and Cu.
Based on the above idea, we can view the user imagination process as the solution to the optimization assume that the users imagined the agent the who have interacted with this item, and by using backpropagation, we can optimize the content-based initialized embeddings. Thus, the transition function Ps. of h4 be written follows,
objective (Eq. (4). Specifically,
(hr + Aaaa if af, U; -he
h++1 Ph (he, a1)
if ag Gend:
where À is hyperparameter and can be understood as the learning rate. If Ua.. is computed using dot product, the final transition function of fy can be written as follows.
h(+1 = -0n(tps at) = [h:
+ A Baa
if at U
(6)
h2
if 02 = Gend-
And can be updated as 441 4e- 1.
3.3 Reward Function
The objective of the USIM is to optimize the initialized embeddings by imagining user sequences. To facilitate this process, design reward function to guide the reinforcement learning approach. Our reward function consists of three components.
Embedding Alignment Reward. We believe that the item embeddings generated by the IV model represent the best solution for our optimization process. Consequently, our objective is to closely align the final state representation with actual item embedding (i.e., the corresponding embedding from the IV model). To achieve this, we calculate the reward based on the concept similarity.
Remo oz) D(hi, e.) D(h,,--1, e.), hi,41 Ph. (hi,,, de),
where D):.) denotes the Euclidean distance between embeddings. This reward represents the change in similarity between the state representation and the actual item embedding, before and after the state transition.
Recommendaion Performance Reward. Although the embedding alignment reward encourages the state representation h be close the actual item embedding, it does not differentiate between states when multiple representations are equally distant from the actual embedding. Additionally, it does not fully utilize the insights from existing user interactions. Therefore, we design a reward function based on recommendation performance as follows,
(hi,t ae) f(h ex) [(hi,tt1, es), hi,41 Ph Phiha, 4t),
((h,,ree)
hha Cu, Cuyl. NyE
(8)
[44
where (hu,t e,) represents the predictive power of state representation h for users in U4 The final performance reward is derived from the change in f before and after the state transition. This change represents the variation in the embedding's 's predictive capability.
Step Regulation. To encourage the agent to achieve the goal in as few steps as possible, we impose a penalty for each action it takes. Therefore, the final reward function is follows,
Tt = R(55,) 44) = Rems 4t) + (Fe,,, ax) -
(9)
where p is a hyperparameter that represents the penalty.
5

---

3.4 Exploration Set Construction
Given the large user base in the recommendation dataset, sampling actions solely by probability during initial reinforcement learning often results in negative rewards, slowing convergence and reducing performance. To explore actions more efficiently, we construct an exploration set according to state 8,, comprising three components.
Positive Action. In the USIM framework (Section 3.2). the agent assumes that the imagined users correspond to those who have interacted with the item, so these users are included in the exploration set. While optimizing f to match e; theoretically requires only the interaction set U4 achieving this often demands combining multiple actions E Ui making exploration complex. To streamline this, we select users most likely bridge hi, and e;, accelerating the process. The Positive Action set Upos is constructed as follows,
Upos
Topk. ((es - (i,,)))))) ea) Ul;
(10)
where Top ((e: hy,s), 6n) represents the set of ky users with the highest cosine similarity to e, h, forming vector that directly points to the actual item embedding.
Random Action. Relying solely on the aforementioned action sets may overly restrict actions, limiting state space coverage and reducing model generalization. Moreover, sampling actions with negative rewards can also benefit training 15). Thus, we augment the action set by randomly selecting l2 actions from the remaining pool, denoted as Uyad-
Termination Action. To enable the agent to learn when to terminate, also incorporate the termination action Gend into the final action set. This inclusion allows the agent to determine the appropriate timing for ending the optimization process.
For simplicity, we set l = 12 = k. And exploration set Asamp can be represented as follows,
Asamp Upas Ultrad U (Aand).
(11)
During the training phase, as the sampling range is narrowed from the full action set A to a specific action set Asamp- we rewrite the Eq. (3) follows,
explen, Wia 1
(1-o (W28 +c)) E. EngAramp exp e, W T(ac(s))= = o (288 tc),
if a Asamp \
{Bend}:
if at aend'
0,
if at Asamp-
(12)
3.5 Training with RecPPO
We incorporate recommendation-speeee supervision signals into PPO [6.. referring to this enhanced approach as Recommender--rieeee PPO (RecPPO), to train our USIM. In the scenario of OOV item recommendation, the optimal action following certain states is clear, allowing the cumulative expected rewards for these states to be calculated directly. When the state representation h; of a specific item equal to its item embedding e; according our designed reward function, the expected value should be 0 because any subsequent actions, except for termination will lead to negative rewards.
So we use these supervision signals to assist in training the value network Va, the specific loss function of the value network is defined as follows,
((w)
Σ
[Tre + (8441) - V=((s))²]
+
(ie random N)))
B
T
(B
SEI
(13)
where B denotes tuples sampled from the buffer pool, and random(O, .N) a random number between O and N. The first term of the loss is the Temporal Difference loss used in value network training, while the second term includes our recommendation-orienne previous actions, when the state representation hú matches e,, the agent should terminate. Here, random(O, (0, N) acts as the timer for each termination state. As for the policy network, we train it using the same method PPO. The whole training process can be found in Appendix C
supervision signals. Regardless of

---

4 Experiments
We conduct comprehensive experiments on two benchmark datasets aiming to address the following three questions: RQI: Can USIM achieve superior OOV item recommendation performance com- pared to state-of-the-ar OOV item recommendation models? RQ2: How key components of USIM affect its performance? RQ3: Is the proposed USIM more effective than representative RL methods? RQ4: What is the tendency of performance during USIM's imagination process? RQ5: How does USIM perform in real-world industrial recommendations? RQ6: How does USIM achieve efficiency compared to other baselines?
4.1 Experimental Setup
Table I: Overall, OOV, and IV item recommendation performance comparison over two representative recommender backbones (MF and GNN). The best and second-best results in each column are
highlighted in bold font and underlined, ** indicates the statistical significance p < 0.01 compared to the best-performed baseline, * indicates the statistical significance p < 0.05 compared to the best-performed baseline.
Overall Recommendation
Recommendation
Recommendation
Method
Cike
MovieLens
Uite
MovicL.nns
CiteULike
MovicLens
Recall
NDCG
Recall
NDCG
Recall
NDCG
Recall
NDCG
Recall
NDCG
Recall
NDCG
DropoutNer
0.0973
0.0768
B.1148
0,1898
0.2442
0.1485
0.1352
0.1417
0,1560
0.0941
0.2577
0.2442
MTPR
0,1090
0.1427
0.1408
0.1808
0,1021
0.3463
0.2233
Heater
0.1149
0.0881
0.1163
0,1901
0.2578
0,1532
0.0601
0.0734
0.1674
0.0909
0.2604
0.2640
CI CReee
0,1242
0,1047
0.1101
0.1556
0.2187
0.1331
0.0744
0.0751
0,2197
0,1411
0.2413
0.2089
DeepMusic
0.1451
0.1287
0.1359
0,2153
0.2194
0.1281
0.0078
0.1003
0.2838
0.1933
0.3623
0.2811
MetaEmb
0.1326
0.1210
0.2303
0.1307
0.1417
0.1445
0.2838
0.1933
0.M02
0.2811
GAR
8.1440
0.1132
D.0422
0.0812
0.2453
0.1479
0.0348
0.0510
0.2272
0.1438
0.1001
0. 1.10
ALDD
0.1584
0,1212
B.1360
0.2154
0.2622
D. 1542
0.1258
0.1262
0,2838
0.1933
0.3023
0.2811
USIM
0
0.1536
0.136
0.2164
0
0.1647
0.1401
0.1466
0,2838
0.1933
0.3023
0,2811
8Improv
19.00%
10.96%
0.66%
0.46%
5.00%
6.81%
1.45%
DropouiNet
0.0914
0.0718
0.1205
0.1962
0.2337
0.136
0.1552
0.1506
0.1596
0.09-19
0.2713
0,2530
MTPR
0,1066
0.0783
0.1217
0.1949
0.2309
0,1355
1412
0.1375
0,1749
0.0963
0.2708
0.2520
0,1116
0.0869
B.1236
0.2467
0.1499
0.0366
0.0319
0,1007
0.2746
0.2647
CLCRee
0.1303
0.1150
0.1127
0.1656
0.2315
0.1295
0.0535
0.0627
0,2260
0.1537
0.2515
0.2226
DeepMusic
0.1244
0.1058
0.1428
0.2353
0.1899
0.1131
0,1539
0.145
0,2685
0.1723
0.3183
0.3048
MetaEibb
0.1349
0.1888
0,1428
0,2353
0,2150
0. 0,1249
1560)
0.1567
0.2685
0.1723
0,3183
0.3048
0.1098
0.0863
0.0179
0.2053
0,1024
0.0288
0.0268
ALDI
0.1371
0.1013
0.1429
0.2351
0.2466
0,1399
0.1358
0.344
0.2685
0.1723
0.3183
0.3048
USIM
0.1629
0,1199
0.1448
0,2377
0.2493
0.1452
0.1564
0.1638
0,2685
0.1723
0.3183
0,3048
Silprro.
5.90%
4.26%
1.32%
0.98%
0.93%
1.65%
0,26/8
4.53%
Datesets. We evaluate the performance of USIM on OOV items using the widely used datasets: CikeULike [8] and MovieLens (17). Specifically. Uike contains 5,551 users, 16,980 articles (items), and 204,986 interactions. MovieLens comprises 6,040 users, 3,883 movies (items), and 1,000,210 interactions. Details about these datasets are shown in Appendix D.I
Baselines. To assess the effectiveness and universality of USIM, conduct a comparative analysis with 8 reading-edge models in OOV item recommendations across two distinct datasets. These models include
main groups. (i) Dropout- methods: DropoutNet [7], MTPR (18), Heater g and CLCRec (9. (ii) Generative-base
methods: DeepMusic [12). MetaEmb [19], GAR (5). and
ALDI ( Details about these models are shown in Appendix [.22 To further verify the generalization ability, we adopted both the widely used collaborative filtering model MF (00 and GNN-based model
NGCF 3 as the recommender, respectively.
Evaluation Metrics. Following the evaluation of cxisting OOV recommendation 6 91. we conduct three different tasks in our experiments: (1) Overall Recommendation, (2) OOV Recommendation. and (3) IV Recommendation. We employ the full-ranking evaluation approach to assess the per- formance of overall, IV, and OOV recommendations. Following previous works 4 B] we utilize Recall@ K and Normalized Discounted Cumulative Gain (NDCG@ K) as metrics.
Implementation Details. We implement the baselines using their officially provided version. See Appendix F for the detailed implementation. The best hyperparameters are found for each dataset. For faimess, we use the same options and follow the designs in their articles for all basclines.
7

---

Table 2: Ablation study results between USIM with its four variants on CiteULike.
MF
GNN
Variant
Overall
OOV
Overall
OOV
Recall
NDCG
Recall
NDCG
Recall
NDCG
Recall
NDCG
w/o
0.1742
0.1439
0.2336
0.1411
0.1590
0.1205
0.2363
0.1198
w/o
0.1754
0.1433
0.2341
0.1412
0.1368
0.1156
0.2411
0.1336
w/o
0.1436
0.1279
0.2232
0.1316
0.1221
0.1000
0.1971
0.1103
w/o pr
0.1866
0.1510
0.2711
0.1596
0.1417
0.1175
0.2414
0.1344
USIM
0.1926
0.1530
0.2753
0.1647
0.1632
0.1245
0.2534
0.1478
4.2
Main Results (RQ1)
The main comparison results of overall, IV, and OOV item recommendation results can be found in T From the results, we can have the following observations.
USIM can generally achieve significant improvements over state-of-the-art methods on both overall and OOV item recommendations while keeping the IV item recommendation. From the tables, we observe that the USIM achieves the highest average Recall and NDCG performance across both MF and GNN recommenders. These comparison results verify the superiority of the imagined embeddings over traditional one-step generated embeddings.
Dropout-based baselines have the performance drop in the IV item recommendation. We find that the USIM and the generative- dropout- based models will lead to performance drop IV items. This suggests that there is difference between OOV items and IV items. Pre-training representations of IV items in advance to generate representations for OOV items may be better for retaining information for IV items.
models can keep the IV item recommendation. However,
4.3
Ablation Study (RQ2)
To validate the effectiveness of the individual components in our model, we compared the full model against four variants: (i) w/o of removes the cosine similarity-based top-k user selection when constructing the positive action set. (ii) w/o ra removes the randomly sampled actions in the exploration set. (iii) w/o es does not construct an exploration set and directly explores the entire action set. (iv) w/o pr removes the performance reward, only using the similarity reward in the RL processing. From the results in Table 2
we make the following observations.
Effectiveness of components in the exploration set construction. The w/o of result indicates that the cosine similarity- based selection of the top-k users can simplify the exploration process by quickly identifying users related to the item embedding. Then, the performance of w/o ra reveals the importance of including negative samples to provide more comprehensive exploration signal, preventing the agent from overestimating action values by only considering positive feedback. Further, the poor performance of the w/o es approach highlights the challenge of effectively exploring an extremely large action space, emphasizing the need for a well-designed exploration strategy to guide the agent towards promising actions.
Effectiveness of components in the reward function, Removing the performance reward component leads to inferior results. This suggests that the reward function should not only consider the distance between the generated embedding and the target embedding but also explicitly incorporate the downstream recommendation performance of the generated embedding.
4.4 Comparison with Representative RL Methods (RQ3)
In Figure we compared proposed model with our initialized MLP and two other traditional reinforcement learning methods: Wolpertinger Policy [21] (WP) and Hierarchical Reinforcement Learning (22 (HRL), WP utilizes the similarity of action representations and value estimation to solve the large discrete action space problem in reinforcement learning, while HRL employs a hierarchical decomposition of action space and sub-task solution improve exploration efficiency.
Our model significantly outperforms the other two methods in both OOV recommendation and overall recommendation. Moreover, it can be observed that the other two methods do not show
8

---

WP
HRL
USIM
Overall Recommendation
OOV Recommendation
0.152
0.119 0.16
0.14
NDCG
0.136
0.107 0.14
0.12
0.120
0.095
0.12
0.10
MF
GNN
MF
GNN
Figure 3: Comparing USIM with other RL methods for overall and OOV recommendation perfor- mance in Cike dataset.
0.183
02V Roserrrnndioo
CW CDV Recommenteeon
0.36
Cvare Nocmmaaddi
11300
GWW Ove Recormrenduton
6.335
0.12
0.000
0.00
20050
0.003
0.0
0.00
0.000
Figure 4: Performance analysis of different generation methods on the Ulike dataset.
apparent improvements compared to their initial states. This supports the effectiveness of our tailored exploration method improving performance in OOV recommendations.
4.5 Case Study (RQ4)
To investigate the performance trend of our proposed method during the optimization process, and to validate the importance of the way of sampling in the embedding optimization process, we compare USIM with the following two user selection strategies: randomly sampling users at each step. and randomly selecting from the top 20 users with the highest relevance scores each step. The results are shown Figure and according to the result, we can draw the following conclusions.
(i) Our method outperforms the random selection of top-20 users and the entire user set in both OOV and overall scenarios, indicating it can effectively identify users beneficial for optimization. However, performance declines as more users are imagined, likely due to reduced exploration by the agent.
(ii) When randomly selecting users from the top 20, we can observe that the performance in OOV recommendation increases after the first step. This suggests that sampling from high-scoring users is more likely to select who beneficial to the optimization process. However, the performance then continuously declines, indicating that this approach is not suitable for all states and has limited help for the optimization process.
(iii) Randomly selecting a user at each step leads to a drastic and non-recoverable decline in perfor- mance in both the OOV and overall scenarios after the first step. This suggests that in recommendation scenarios, it is extremely difficult to sample users from the massive user set who are beneficial to the optimization, and the majority of users are highly detrimental the optimization process.
More experimental hyperparameter analysis can be found Appendix
4.6 Online Evaluation (RQ5)
To evaluate the performance of USIM in an industrial setting, we conducted a two-week online A/B test major e-commerce platform with 5% of users in each group. USIM was compared against three baselines: Random, MetaEmb [9), and ALDI [6]. Details about our test platform and
evaluation metries provided in Appendix G Table 3 presents the results of these online A/B tests.
These remarkable improvements across all metrics underscore the effectiveness of the USIM in addressing the OOV item recommendation problem in real-world recommender systems. The

---

Table 3: Results of online A/B test in the industrial platform.
A/B Test
OOV Item PV
OOV Item PCTR
OOV Item GMV
Random 8.20%
2.80%
20.30%
vs. MetaEmb
6.55%
1.95%
14.95%
Vs. ALDI
4.90%
1.10%
13.60%
consistent and substantial performance gains, particularly in OOV item GMV, highlight the practical impact of our approach on business outcomes in e-commerce settings.
4.7 Efficiency Analysis (RQ6)
To evaluate the time efficiency of USIM, especially in comparison to SOTA baselines, recorded the total training time(T (Traiiing Time), total convergence epochs( Converge Epochs), Time Per Epoch, and Inference Time for USIM and each baseline the Cike and MovieLens datasets. The results are presented in Table Based on these results, we can draw the following conclusions:
Table 4: Results of time efficiency.
Method
Training Time
Converge Epochs
Time Per Epoch
Inference Time
MetaEmb
254s
28
9.07s
0.012s
CiteLiiee
ALDI
225s
51
4,41s
0,013s
Heater
841s
57
14.75s
0.074s
CLCRec
926s
70
13.22s
0,070s
USIM
484s
36
13.44s
0.031s
MetaEmb
415s
11
37.72s
0.005s
MovieLens
ALDI
664s
50
13.28s
0,004s
Heater
1474s
20
73.7s
0,072s
CLCRec
3684s
59
62.44s
0.084s
USIM
330s
45
7.33s
0.045s
(i) USIM Faster than Heater and CLCRee: USIM computes embeddings only for OOV items, whereas Heater and CLCRec must compute embeddings for both OOV and IV items.
(ii) USIM is Comparable with MetaEmb and ALDI: USIM imagines sequences only for OOV items, resulting in inference times comparable to MetaEmb and ALDI.
(iii) USIM is Efficient in Training: By fundamentally addressing OOV recommendation, USIM converges in fewer epochs, making training more efficient.
More experiments about online recommendation efficiency can be found in Appendix H
Conclusion
Recommending out-of-vocabulary (OOV) items is challenging due to the lack of well-trained be- havioral embeddings. Current models use "makeshift" embeddings from content features, leading to suboptimal performance. We introduced the User Sequence Imagination (USIM) framework to refine OOV embeddings by imagining user sequences and incorporating behavioral embeddings. By coommendation-focusee reward
framing this reinforcement learning problem and creating function, USIM effectively enhances OOV recommendations. Extensive experiments demonstrate its superior performance and the ablation study further illustrates the effectiveness USIM.
Acknowledgment
This research was funded by the National Science Foundation of China (No.62172443) and Hunan Provincial Natural Science Foundation China 20.22022133)53). This work was carried out part using computing resources at the Perfforrfrme Computing Center of Central South University.
10

---

References
[1] Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L Hamilton, and Jure Leskovec. Graph convolutional neural networks for web-scale recommender systems. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pages 974-983, 2018.
[2] Hao Chen, Yuanchen Bei, Qijie Shen, Yue Xu, Sheng Zhou, Wenbing Huang, Feiran Huang, Senzhang Wang. and Xiao Huang. Macro graph neural networks for online billion-scale recommender systems. In Proceedings of the ACM on Web Conference 2024, pages 3598-3608 2024.
[3] Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, and Tat-Seng Chua. Neural graph collabo- rative filtering. In Proceedings of the 42nd International ACM SIGIR Conference Research and Development in Information Retrieval, page 165-174, 2019.
[4] Xiangnan He, Kuan Deng, Xiang Wang. Yan Li, YongDong Zhang, and Meng Wang. Lightgen: Simplifying and powering graph convolution network for recommendation. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information 2020.
Retrieval, page
[5] Hao Chen, Zefan Wang, Feiran Huang, Xiao Huang, Yue Xu, Yishi Lin, Peng He, and Zhoujun Li. Generative adversarial framework for cold-start item recommendation. In Proceedings of the 45th International ACM SIGIR Conference Research and Development in Information Retrieval, page 2365-2571, 2022.
[6] Feiran Huang, Zefan Wang, Xiao Huang, Yufeng Qian, Zhetao Li, and Hao Chen. Aligning distillation for cold-start item recommendation. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, page 1147-1157, 2023.
[7] Maksims Volkovs, Guangwei Yu, and Tomi Poutanen. Dropoutnet: addressing cold start in recommender systems. In Proceedings of the 31st International Conference on Neural Information Processing Systems, page 4964-4973, 2017.
[8] Ziwei Zhu, Shahin Sefati, Parsa Saadatpanah, and James Caverlee. Recommendation for new users and new items via randomized training and mixture-of-experts transformation. In Proceedings of the 43rd International ACM SIGIR Conference Research and Development in Information Retrieval, page 1 1111-1130, 2020.
[9] Yinwei Wei, Xiang Wang. Qi Li, Liqiang Nie, Yan Li, Xuanping Li, and Tat-Seng Chua. Con- trastive learning for cold-start recommendation. In Proceedings of the 29th ACM International Conference on Multimedia, page 5382-5390, 2021.
[10] Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedfor- ward neural networks. In Proceedings of the thirleenth international conference artificial intelligence and statistics, pages 249-256, 2010.
[11] Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme Bpr: Bayesian personalized ranking from implicit feedback. In Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence, page 452-461, 2009.
[12] Aäron van den Oord, Sander Dieleman, and Benjamin Schrauwen. Deep content-based music recommendation. In Proceedings of the 26th International Conference on Neural Information Processing Systems- Volume 2. page 2643-2651, 2013.
[13] Fabio Pardo, Arash Tavakoli, Vitaly Levdik, and Petar Kormushev. Time limits in reinforcement learning. In Proceedings of the 35th International Conference on Machine Learning, pages 4045-4054, 2018
[14] Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. Bert4rec: Sequential recommendation with bidirectional encoder representations from transformer. In Pro- ceedings of the 28th ACM international conference on information and knowledge management, pages 1441-1450, 2019.

---

[15] Xiangyu Zhao, Liang Zhang, Zhuoye Ding, Long Xia, Jiliang Tang, and Dawei Yin. Recom- mendations with negative feedback via pairwise deep reinforeement learning, In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, page 1040-1048, 2018.
[16] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.0634 2017.
[17] Maxwell Harper and Joseph Konstan. The movielens datasets: History and context. Acm transactions interactive intelligent systems (tiis), 5(4):1-19, 2015.
[18] Xiaoyu Du, Xiang Wang, Xiangnan He, Zechao Li, Jinhui Tang, and Tat-Seng Chua. How to learn item representation for start multimedia recommendation? In Proceedings of the 28th ACM International Conference on Multimedia, page 3469-3477. 2020.
[19] Yongchun Zhu, Ruobing Xie, Fuzhen Zhuang, Kaikai Ge, Ying Sun, Xu Zhang, Leyu Lin, and Juan Cao. Learning warm up cold item embeddings for cold-start recommendation with meta scaling and shifting networks. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1167-1176, 2021.
[20] Yehuda Koren, Robert Bell, and Chris Volinsky. Matrix factorization techniques for recom- mender systems. Computer, 42(8):30-37, 2009.
[21] Gabriel Dulac-Amold, Richard Evans, Hado van Hasselt, Peter Sunehag, Timothy Lillicrap, Jonathan Hunt, Timothy Mann, Theophane Weber, Thomas Degris, and Ben Coppin. Deep reinforcement learning in large discrete action spaces. arXiv preprint arXiv:1512.07679, 2015.
[22] Pierre-Luc Bacon, Jean Harb, and Doina Precup. The option-critic architecture. In Proceedings of the AAAI conference artificial intelligence, volume 31, 2017.
[23] Yifan Hu, Yehuda Koren, and Chris Volinsky. Collaborative filtering for implicit feedback datasets. In 2008 Eighth IEEE international conference on data mining, pages 263-272, 2008. 2008
[24] Yehuda Koren, Robert Bell, and Chris Volinsky. Matrix factorization techniques for recom- mender systems. Computer, 42(8):30-37, 2009.
[25] Hao Chen. Yue Xu, Feiran Huang, Zengde Deng. Wenbing Huang, Senzhang Wang, Peng He, and Zhoujun Li. Label-aware graph convolutional networks. In Proceedings of the 29th ACM International Conference Information & Knowledge Management, page 1977-1980, 2020.
[26] Hao Chen, Zhong Huang, Yue Xu, Zengde Deng, Feiran Huang. Peng He, and Zhoujun Li. Neighbor enhanced graph convolutional networks for node classification and recommendation. Knowledge-Band Based Systems, 246:1086 10559,, 2022.
[27] Zijin Hong, Zheng Yuan, Hao Chen, Qinggang Zhang, Feiran Huang, and Xiao Huang. Knowledge-to-sql: Enhancing sql generation with data expert Ilm. arXiv preprint arXiv:2402. / 1517, 2024.
[28] Wang-Cheng Kang and Julian McAuley. Self-attentive sequential recommendation. In 2018 IEEE international conference on data mining (ICDM), pages 197-206, 2018.
[29] Dietmar Jannach and Malte Ludewig. When recurrent neural networks meet the neighbor- hood for bessenn-based recommendation. Proceedings of the eleventh ACM conference on recommender systems, pages 306-310, 2017.
[30] Zijin Hong, Zheng Yuan, Qinggang Zhang, Hao Chen, Jumnan Dong, Feiran Huang, and Xiao Huang. Next-generatte database interfaces: A survey of Ilm-based text-to-sql. arXiv preprint arXiv:2406.08426 2024.
[31] Huachi Zhou, Hao Chen, Junnan Dong, Daochen Zha, Chuang Zhou, and Xiao Huang. Adaptive popularity debiasing aggregator for graph collaborative filtering. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 7-17, 2023.
12

---

[32] Peiyan Zhang, Yuchen Yan, Xi Zhang, Liying Kang, Chaozhuo Li, Feiran Huang. Senzhang Wang, and Sunghun Kim. Gpt4rec: Graph prompt tuning for streaming recommendation. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1774-1784, 2024.
[33] Yijie Zhang, Yuanchen Bei, Hao Chen, Qijie Shen. Zheng Yuan, Huan Gong, Senzhang Wang. Feiran Huang, and Xiao Huang. convolutional networks. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 6257-6268, 2024.
collaborative filtering with partial order graph
[34] Feiran Huang, Zhenghang Yang, Junyi Jiang, Yuanchen Bei, Yijie Zhang, and Hao Chen. Large language model interaction simulator for cold-start item recommendation, arXiv preprint 20Xi::2402.09166, 2024.
[35] Xiaoxiao Xu, Chen Yang, Qian Yu, Zhiwei Fang, Jiaxing Wang, Chaosheng Fan, Yang He, Changping Peng, Zhangang Lin, and Jingping Shao. Alleviating cold-start problem ctr prediction with variational embedding learning framework. In Proceedings of the ACM Web Conference 2022, page 27-35, 2022.
[36] Shaoyun Shi, Min Zhang. Xinxing Yu, Yongfeng Zhang, Bin Hao, Yiqun Liu, and Shaoping Ma. Adaptive feature sampling for recommendation with missing content feature values. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, page 1451-145 14600 2019.
[37] Feiyang Pan, Shuokai Li, Xiang Ao, Pingzhong Tang, and Qing He. Warm up cold-start advertisements: Improving ctr predictions via learning to learn id embeddings. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, page 695-704, 2019.
[38] Yuanfei Wang, fangwei zhong, Jing Xu, and Yizhou Wang. Tom2c: Target-oriented multi-agent communication and cooperation with theory of mind. In International Conference on Learning Representations, 2022.
[39] Long Ma. Yuanfei Wang, Fangwei Zhong, Song-Chun Zhu, and Yizhou Wang. Fast peer adaptation with context-aware exploration. In Proceedings of the 41st International Conference on Machine Learning, pages 33963-33982, 2024.
[40] Zhenyu Guan, Xiangyu Kong, Fangwei Zhong, and Yizhou Wang. Richelieu: Self-evolving llm-based agents for ai diplomacy. arXiv preprint arXiv:2407.0683 2024.
[41] Fangwei Zhong, Kui Wu, Hai Ci, Churan Wang, and Hao Chen. Empowering embodied visual tracking with visual foundation models and offline rl. arXiv preprint arXiv:2404.0995 2024.
[42] Yuanchen Bei, Sheng Zhou, Qiaoyu Tan, Hao Xu, Hao Chen, Zhao Li, and Jiajun Bu. Rein- forcement neighborhood selection for unsupervised graph anomaly detection. In 2023 IEEE International Conference on Data Mining (ICDM), pages 11 11-20, 2023.
[43] Xinshi Chen, Shuang Li, Hui Li, Shaohua Jiang, Yuan Qi, and Le Song. Generative adversarial user model for reinforcement learning based recommendation system. In Proceedings of the 36th International Conference on Machine Learning, pages 1052-1061, 2019.
[44] Lixin Zou, Long Xia, Zhuoye Ding, Jiaxing Song, Weidong Liu, and Dawei Yin. Reinforcement learning to optimize long-term user engagement in recommender systems. In Proceedings of the 25th ACM SIGKDD international conference knowledge discovery & data mining, pages 2810-2818, 18, 2019.
[45] Lixin Zou, Long Xia, Pan Du, Zhuo Zhang, Ting Bai, Weidong Liu, Jian- Yun Nie, and Dawei Yin. Pseudo dyna-q: A reinforcement learning framework for interactive recommendation. In Proceedings of the 13th International Conference on Web Search and Data Mining, page 816-824, 2020.
[46] Xiangyu Zhao, Long Xia, Jiliang Tang, and Dawei Yin. "deep reinforcement learning for search, recommendation, and online advertising: a survey" by xiangyu zhao, long xia, jiliang tang, and dawei yin with martin vesely as coordinator. SIGWEB Newsl., 2019(S))ing). 2019.
13

---

[47] Dong Liu and Chenyang Yang. A deep reinforcement learning approach to proactive content pushing and recommendation for mobile users. IEEE Access, 7:83120-831366 2019.
[48] Zefang Liu, Shuran Wen, and Yinzhu Quan. Deep reinforcement learning based group recom- mender system, arXiv preprint arXiv:2106.0668 069000 2021.
14

---

A Related Works
General Recommendation, General recommendation systems typically predict which items users will prefer by leveraging collaborative information. Based on how collaborative information is utilized, general recommendation methods typically include approaches such as Matrix Factorization-bant Graph-based, and Sequential recommendation. Matrix Factorization-basse recommendation methods derive user and item embeddings by decomposing the interaction matrix into two feature matrices using Matrix Factorization techniques [ 23 24). Graph-based recommendations incorporate graph techniques [25] to model high-order relationships between users and items
4 3 26]. Sequential
recommendation methods focus on capturing the temporal patterns user interactions, modeling
the order of users' actions to predict their future preferences [27 28
32 However, these
methods are generally ineffective in addressing the OOV item,
out-of-Vocabulary Item Recommendation. to address the problem of recommending completely new item that has prior user interactions with users 33 34 Its core idea lies in how to map the content information of the cold item into the space defined by the warm item embeddings trained from interactions with users. To achieve this goal, existing methods can be mainly categorized into two approaches. One category is the dropout model, which Iearns a content map function using the dropout approach [18
cott-o--vocablary (OOV) item recommendation aims
The other category is the generative model, which directly learns the relationship between the content of
35
36
the OOV item and the IV behavior embeddings obtain the mapping function 6 22 37
Reinforcement bearning-based Recommendation. Reinforcement learning (RL) a famous type of learning strategy where agent learns make decisions by taking actions in environment to maximize some notion of cumulative reward, which is widely used technique for modeling human behavior 42. Many studies also focus on simulating community. Cascading DQN 4 uses GAN to learn and simulate real users from historical inter- the simulator is trained user historical data
behavior within the recommendation
actions to obtain the reward function. In to simulate user feedback. In Pseudo Dyna-Q [5]. world model (user simulator) is trained by minimizing the error between online and offline rewards. Another common simulation approach is based on collaborative filtering. LIRD [uibusss a memory with (s, a, r) tuples seen in the log dataset and uses a similarity method based on cosine similarity to find the closest stte-action pair to the current state and recommended action. DRR [7] and DRGR [8] use the same intuition but based on different factorizations, respectively.
155
44
B Details of initial model
Loss function Our initial model G is an MLP trained to minimize the Euclidean distance between the output of G and the item embedding. Specifically, the loss function for defined as follows:
((0)
Σ [Tvel SET
1
|le - Go(c.)ll.
(14)
Training details The initial model shares the same training data as USIM. We use the Adam optimizer with learning rate of 0.001 and apply early stopping by monitoring NDCG@K on the validation set. The batch size and regularization weight are set to 1024 and 0.001, respectively.
C Model Details
Our complete training process is shown in Algorithm
D Experimental Details
D.1 Dataset Details
Datasets. We evaluate the USIM performance on cold-start items using the CiteULike and MovieLens datasets.
15

---

Algorithm 1: Training USIM
Input: Policy network Value network u, episode length N, initial state generator G, experience
replay buffer == 0
for each iteration do
for each batch do
Initialize state 5g G(c:)
for in range([V) do
Sample action a, according to Eq. equation
Receive reward ry according to Eq.equation
Transition next state s according to Eq.equation 6
Store transition in buffer: D < D (8t, dt, Yt, 81+1)
end for
for each gradient step do
Sample transitions from for gradient calculation Update the Policy network using the PPO loss Update the Value network w with Eq. equation end for end for end for Output: Trained Policy æ,
Value network w
The dataset contains 5,551 users, 16,980 articles, and 204,986 interactions. On CiteULLike, registered users create scientific article libraries and save articles for future reference. The goal is to leverage these libraries to recommend relevant new articles to each user. The articles are represented by 300-dimensional vectors as item content features.
SiteULike"
MovieLens comprises 6,040 users, 3,883 items, and 1,000,210 interactions. The MovieLens 177
content features of items are represented using 200-dimensional vectors.
this paper, the content features of items are represented using 200-dimensional vectors. each dataset, 20% of items are designated as OOV items, with interactions split into OOV validation set and testing set (1:1 ratio). Records of the remaining 80% of items are divided into training, validation, and testing sets, using an 8:1:1 ratio.
D.2 Baseline Details
Baselines. To assess the effectiveness and universality of USIM, conducted comparative analysis with 8 lcading-edge models in the domain of cold-start recommendations. This comparison was carried out across two distinct datasets. The models we benchmarked against include two main groups: (i) Dropout-based models: DropoutNet 77. MTPR [8], Heater [8], and CLCRec 9 (ii)
Generative models:
DeepMusic [22 MetaEmb
19 GAR 5). and ALDI [6]
DeepMusic utilizes deep neural networks model the mean squared error (MSE) difference between generated and warm embeddings.
MetaEmb trains meta- meta-lcarning-bae based generator for rapid convergence.
GAR generates embeddings through generative adversarial relationship with the warm recom- mendation model.
ALDI employs distillation, using warm items as "teachers" to transfer behavioral information to cold items, referred to as "students".
DropoutNet enhances cold-start robustness by randomly discarding embeddings.
MTPR generates counterfactual cold embeddings considering dropout and Bayesian Personalized Ranking (BPR).
16

---

MF
MF
MF
MF
007
0.30
Figure Hyperparameter analysis of MF backbone on Cike dataset
GNN
GNN
GMN
CNN
3.25
0.32
DON
033
007
032
0.12
0.11
Figure 6: Hyperparameter analysis of GNN backbone on CiteULike dataset
Heater improves DropoutNet by using mix-of-experts network and considering embedding similarity.
CLCRec models cold-start recommendation using contrastive learning from information theoretic perspective.
E
Implementation Details
We implement the baselines using their official implementations. Specifically, for GAR, the updated version provided in the official repository, which evaluated under the same CLCReec settings used in pape The embedding dimension is set 200 for all models. We employ the Adam optimizer with learning rates of 0.001 for the critic and 0.0005 for the actor. Farly stopping is applied by monitoring HDCG@K weight are set to 1024 and 0.001, respectively. The experiment was conducted on an NVIDIA GeForce RTX 3090 with 24GB of memory. Hyperparameters parameters are identified for each dataset. For fairness, we use the same settings and follow the design choices in their respective articles for all baselines.
the validation set. The training batch size and regularization
tuned using grid search, and the optimal
F Additional Experiments
We conducted a parameter analysis on the CiteULike dataset, and the results are shown in Figures and 6. Except for the parameters, all other parameters exhibited the same trends on both the MF and GNN backbones.
For the parameter k, the performance first increased and then decreased k increased, reaching the optimal values at 30 and 20, respectively.
For the parameter N, the OOV recommendation performance first increased and then decreased, reaching the maximum at = 9, while the overall recommendation performance consistently decreased.
For the parameter p. the OOV performance first increased and then decreased, while the IV perfor- mance first decreased and then increased.
For the parameter X, on the MF backbone, the OOV performance consistently increased, while the overall performance first increased and then decreased, reaching the optimal value at À = 0.07. On the GNN backbone, both the OOV and IV performances exhibited trend of first increasing and then decreasing.
httpp::/ggihubb.co//ttttoog
17

---

G Details of online test
Platform. We have implemented USIM the homepage of of the largest e-commerce platforms, which boasts hundreds of millions users and billions of items. The homepage features a feed recommendation system that recommends items to users. Thousands of new items are uploaded every hour.
Framework. Our online implementation consists of two components: 1. Online Recommen- dation; and 2. USIM Imagination. We present our framework in Figure 7 When an OOV item is uploaded, utilize the Large Language Model to embed the content features, including the product name and description. We then employ the USIM structure to predict the most suitable user sequence and optimize the embedding accordingly. Finally, in the online recommendation model.
the USIM-produced as the IV embeddings
Evaluation Metries, We employed three tailored metrics to assess the performance of USIM against existing baselines:
Page Views (OOV item PV): The number of user clicks during the OOV period.
Page Click-Through Rate (OOV item PCTR): The ratio clicks to impressions during the OOV period.
Gross Merchandise Value (OOV item GMV): The total value of user purchases during the OOV period.
Online Recommendation
Recommend
Online Recommendation Model
IV User Embeddingss
IV Item Embeddings
User Device
Browse
USIM
Embexiding Updaling
User Imagination
00V Items
Language Model
Content Embeddings
0 ffline OOV Cold-Start
Figure 7: Overall framework of online implementation.
H Online Recommendation Efficiency
To evaluate the online recommendation efficiency of USIM, recorded the LLM-based content feature extraction time (LLM Content Feature Extraction Time) and the inference time for OOV item embeddings (OOV Time) of MetaEmb, ALDI and USIM. The results shown in Table 5
The results of the online recommendation efficiency indicate that USIM does not serve as a bottleneck for online recommendation, the LLM the main time consumer while USIM's speed remains comparable to that of MetaEmb and ALDI. Additionally, the OOV recommendation an offline, one-time process that has impact online recommendations, shown in Figure 7 And given that the platform supports parallel processing, USIM computations for OOV items can be managed efficiently.
I Limitation
One limitation of our proposed model is the large number of hyperparameters that need to tuned. Our model involves 4 key hyperparameters. Tuning these hyperparameters can be time-consuming
18

---

Table 5: Results of online recommendation efficiency
Method
LLM Content Feature Extraction Time
OOV Time
MetaEmb
3.349s # 3.214s
0.044s + 0.012s
ALDI
3.349s 3.214s
0.047s 0.013s 0. 0.102s 0.010s
USIM
3.349s 3.214s
and computationally expensive process, as it often requires extensive grid search or random search to find the optimal configuration.
Another limitation is the slow generation speed of our model due its autoregressive generation approach. In our model, the output is generated sequentially, with each token being predicted conditioned on the previously generated tokens. This autoregressive generation process can be computationally intensive.
19

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction clearly state the claims made, including the contributions made in the paper and important assumptions and limitation We propose the User Sequence IM agination (USIM) framework along with an RL-based solution to maximize out-of-vocabulay (OOV) item recommendation performance.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We have a limitation in the appendix.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full of assumptions and complete (and correct) proof?
20

---

Answer:[Yee Yes]
Justification:
All the theorems, formulas, and proofs in the paper are numbered and cross-
referenced.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
The paper describe the architecture clearly.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
21

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
Source code is available at http:///aanooooooos..pprpacccccd
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
//nips //ipss.cc/pill wttp:ccppppcccceeeeeeeeee..c.ett httaa//gguudee//eeeeabiisssssss /Cc////oddiiisssseeeeeee ssiiiiiiccyyyyyyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification:
The experimental settings are presented in the core of the paper. And full
details are provided appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in the appendix, or supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification:
We have statistical significance the main experiment.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi-
dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for
example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
22

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative rates).
show in tables or
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We indicate the type of compute workers GPU.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't I make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: We checked and ensured that our paper conforms with the NeurIPS Code of Ethics in every respect.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer[[NA]
there is no societal impact of the work performed
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
23

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
the paper poses such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
We cite the original paper that produced the code package dataset. We state which version of the asset is used
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
24

---

If assets are released, the license, copyright information, and terms of use in the thcode has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
latasets.paperssi
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: the paper does not release new assets
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/codeemoodel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: the paper does not involve crowdsourcing nor research with human subjects
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: the paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
25

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
26


Reviews:
[{"review_id": "uy7ATHGxAl", "reviewer": "Reviewer_wA5P", "summary": "This submission proposes a reinforcement learning framework, termed as USIM, to deal with the issue of out-of-vocabulary items in recommendation system, which is also known as cold start issue in the community. The proposed framework considers a fine-grained user sequence imagining process. Specifically, USIM formulates the imagination process of user sequence as a RL problem, and then designs recommendation-oriented reward functions for user selection. Experiments on two public datasets show that USIM achieves better performance than several baselines.", "questions": "Please refer to the weakness part.", "limitations": "NA", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1. Cold start issue is crucial for recommendation, and a fine-grained optimization process is proposed to deal with it. \n\n2. Code and data are available, which helps understand the implementation.", "weaknesses": "1. It is mentioned in the abstract that 'USIM has been deployed on a prominent e-commerce platform for months, offering recommendations for millions of OOV items and billions of users'. However, there seems no online comparison involved in the experiment. More details about online A/B test should be discussed to support the claims in the abstract.\n\n2. There exist some statements lacking literature or empirical evidence, and some statements lacking clear description. For example:\n    - In Line 47 : 'The substantial distinction between the content features and the behavioral embeddings may result in substantial discrepancies between the IV and OOV items', which is not verified with evidence.\n    - Even though mentioned a lot, there is no clear definition about 'imagine user sequence', which seems not a commonly-used term.\n\n3. The writing of the submission is too poor. The presentation of the submission is hard to follow. For example, in section 3.3.1, while the definition of MDP is described, how the MDP is related to the OOV issue is not discussed. The correspondence relation between (state, environment, action, reward) and OOV should be presented, otherwise it is quite challenging for readers to get through the proposed algorithm.\n\n---After reading the author rebuttal---\nThe author rebuttal has address partial concerns. The results of A/B test have been added, which should be included in the camera ready (if accept). I have rasied my score to 5.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ">W1. It is mentioned in the abstract that 'USIM has been deployed on a prominent e-commerce platform for months, offering recommendations for millions of OOV items and billions of users'. However, there seems no online comparison involved in the experiment. More details about online A/B tests should be discussed to support the claims in the abstract.\n\nThanks for your suggestion, we have conducted a two-week A/B test and we'd like to provide the details.\n\n**Platform** We have implemented our USIM on the homepage of one of the largest e-commerce platforms, which boasts hundreds of millions of users and billions of items. The homepage features a feed recommendation system that recommends items to users. Thousands of new items are uploaded every hour.\n\n**Framework:** Our online implementation consists of two core components: 1. Online Recommendation; and 2. USIM Imagination. We present our framework in Fig 1 in the uploaded PDF. When an OOV item is uploaded, we utilize the Large Language Model to embed the content features, including the product name and description. We then employ the USIM structure to predict the most suitable user sequence and optimize the embedding accordingly. Finally, we use the USIM-produced as the IV embeddings in the online recommendation model. \n\n**Baseline Comparisons:** We conducted an online A/B test on 5% of users for each group over two consecutive weeks. We compared USIM against three different baselines: 1. Random, 2. MetaEmb, and 3. ALDI\nThe results of the online A/B tests on the industrial platform are summarized below.\n\n| A/B Test      | OOV Item PV | OOV Item PCTR | OOV Item GMV |\n|---------------|--------------|----------------|---------------|\n| vs. Random    | +8.20%       | +2.80%         | +20.30%       |\n| vs. MetaEmb   | +6.55%       | +1.95%         | +14.95%       |\n| vs. ALDI      | +4.90%       | +1.10%         | +13.60%       |\n\nFrom the online recommendation results, it presents that our USIM model significantly outperforms the baselines in all three metrics. Further, inspired by these A/B test results, USIM is serving mainstream users and providing OOV recommendations for all newly uploaded items.\n\n---\n\n>W2.There exist some statements lacking literature or empirical evidence, and some statements lacking clear description. For example:\n>- In Line 47 : 'The substantial distinction between the content features and the behavioral embeddings may result in substantial discrepancies between the IV and OOV items', which is not verified with evidence.\n>- Even though mentioned a lot, there is no clear definition about 'imagine user sequence', which seems not a commonly-used term.\n\n\nThank you for your comments. We will add literature and empirical support for the gap, and provide a clearer description of the user sequence imagination. Below are our responses to the specific points raised:\n\n1. **Evidence and Literature Support**\n\n    - **Evidence**: In recommendation systems, identical content items can elicit different user intentions. For example, two items with the same content may show vastly different engagement metrics—one may receive over a million clicks while the other gets none. This indicates that there are gaps between content features and behavioral embeddings.\n\n    - **Literature**: Papers on out-of-vocabulary (OOV) recommendations, such as GAR [1], ALDI [2], and UCC [3], demonstrate that using content features to simulate behavioral embeddings can result in differences in embedding distributions and rating distributions. These studies support the existence of gaps between content features and behavioral embeddings.\n\n2. **Clear Definition of 'Imagine User Sequence'**\n\n    The term \"imagine user sequence\" refers to generating a sequence of hypothetical users using real user behavioral embeddings to refine the embeddings of an OOV item. We use \"imagine\" because these generated users are synthesized solely for embedding optimization. This technique allows for a more accurate representation of potential interactions, thereby improving recommendation performance for OOV items.\n\nWe hope these clarifications address your concerns. We will incorporate these explanations into our revised manuscript to ensure greater clarity.\n\n[1] Chen H, Wang Z, et al. Generative adversarial framework for cold-start item recommendation. (SIGIR, 2022)\n\n[2] Huang F, Wang Z, et al. Aligning distillation for cold-start item recommendation. (SIGIR, 2023)\n\n[3] Liu T, Gao C, et al. Uncertainty-aware Consistency Learning for Cold-Start Item Recommendation. (SIGIR, 2023)\n\n---\n\n>W3.The writing of the submission is too poor. The presentation of the submission is hard to follow. For example, in section 3.3.1, while the definition of MDP is described, how the MDP is related to the OOV issue is not discussed. The correspondence relation between (state, environment, action, reward) and OOV should be presented, otherwise it is quite challenging for readers to get through the proposed algorithm.\n\nThanks for your comments, we will revise our manuscript to improve the overall writing quality and ensure that the explanations are clear and comprehensive. Specifically, we will:\n\n1. Provide a detailed explanation of how MDP is related to the OOV issue.\n2. Clearly present the correspondence between (state, environment, action, reward) and the OOV problem.\n3. Enhance the overall readability and flow of the paper.\n\nWe appreciate your constructive comments and will address these points in our revision to make the paper more accessible and easier to follow.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "V5qSIoWxI0", "reviewer": "Reviewer_axrp", "summary": "The authors propose a novel User Sequence Imagination (USIM) fine-tuning framework. This framework can imagine the user sequences and then refine the generated OOV embeddings with user behavioral embeddings. Specifically, the authors frame the user sequence imagination as a reinforcement learning problem and develop a custom recommendation-focused reward function to evaluate the extent to which a user can help recommend the OOV items. Furthermore, they propose an embedding-driven transition function to model the embedding transition after imagining a user.", "questions": "1. Can the author provide the results of significance tests for the main experimental results?\n\n2. Can the author provide a practical comparison of the experimental time between USIM and some SOTA baselines to better evaluate its efficiency?\n\n3. The author mentions that the efficiency of USIM may not be high, while also claiming that it has been deployed in real-world scenarios. However, the deployment of recommendation systems in practical settings demands high efficiency, which contradicts the characteristics of USIM.", "limitations": "The authors well addressed the limitations. One limitation of USIM is the large number of hyperparameters that need to be tuned, which is time-consuming and computationally expensive. Another limitation is the slow generation speed due to its autoregressive generation paradigm.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The author proposes a RL-based approach to optimize the embedding of out-of-vocabulary (OOV) items, allowing OOV items to benefit not only from content embedding shift but also from user interactions.\n\n2. The author has made specific optimizations and improvements to RL methods for the recommendation scenario of out-of-vocabulary (OOV) items. These enhancements contribute to achieving better performance in recommendation systems.", "weaknesses": "1. The experiments conducted in this study used a limited number of datasets, and their scale was not very large. As a result, it is difficult to clearly observe the performance of USIM on large-scale datasets.\n\n2. It is noticed that USIM does not show significant improvements over SOTA methods in certain metrics on the Movielen dataset. Additionally, the author mentions that USIM heavily relies on the selection of hyperparameters, suggesting that the performance advantage of USIM may not be significant.\n\n3. There is a lack of theoretical analysis or experimental comparisons regarding the time efficiency of USIM. Since it is based on RL methods and processed in the form of MDP, it suggests that USIM may have lower time efficiency. Considering that it does not bring significant improvements on MovieLen, the question arises whether the high time complexity it entails is justified.\n\n4. The author mentions that USIM has been deployed in real-world scenarios and has shown improvements. However, the experimental section of the article lacks the representation of relevant results and does not include offline evaluations on large-scale industrial datasets.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ">W1 & W4 & Q3. Performance on large-scale datasets and online environments.\n\nThanks for your suggestion, we conduct further experiments on other OOV recommendation datasets and on real billion-scale online recommender systems. \n\n### Offline Experiments\nWe have conducted experiments on an additional larger-scale Book-Crossing dataset, containing 92,107 users, 270,170 items, and 1,031,175 interactions. Detailed results are presented as follows:\n| Backbone | Method | Overall Recall | Overall NDCG |OOV Recall | OOV NDCG |IV Recall|IV NDCG |\n|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|\n|MF|MetaEmb|0.0085|0.0094|0.0137|0.0103|**0.0178**|**0.0122**|\n| |ALDI |0.0071|0.0074|0.0117|0.0066|**0.0178**|**0.0122**|\n| |DropoutNet|0.0061|0.0063|0.0102|0.0073|0.0111|0.0067|\n| |CLCRec|0.0035|0.0032|0.0131|0.0104|0.0054|0.0033|\n| |USIM|**0.0088**|**0.0097**|**0.014**|**0.011**|**0.0178**|**0.0122**|\n| |%impov.|3.53%|3.19%|2.19%|5.77%|- |- |\n|GNN|MetaEmb|0.0110|0.0109|0.0172|0.0105|**0.0305**|**0.0172**|\n| |ALDI |0.0103|0.0098|0.0125|0.0062|**0.0305**|**0.0172**|\n| |DropoutNet|0.0079|0.0087|0.0157|0.0082|0.0224|0.0133|\n| |CLCRec|0.0051|0.0046|0.0112|0.0068|0.0073|0.0044|\n| |USIM|**0.0113**|**0.0114**|**0.018**|**0.0125**|**0.0305**|**0.0172**|\n| |%impov.|2.73%|4.59%|4.44%|19.04%|- |- |\n\nExperiments on the larger-scale Book-Crossing dataset also verify the strength of USIM.\n\n### Online Experiments\n**Platform** We have implemented our USIM on the homepage of one of the largest e-commerce platforms, which boasts hundreds of millions of users and billions of items. The homepage features a feed recommendation system that recommends items to users. Thousands of new items are uploaded every hour. The LLM service can parallelly process 20 requests of embedding, while the OOV service can parallelly process 512 items at the same time.\n\n**Framework (PDF Fig. 1):** Our online implementation consists of two core components: 1. Online Recommendation; and 2. **Offline USIM Imagination**. When an OOV item is uploaded, we utilize LLM to embed the content features and then employ the USIM structure to predict the most suitable user sequence and optimize the embedding accordingly. Finally, we use the USIM-produced embeddings as the IV embeddings for online recommendation. \n\n**Baseline Comparisons:** We conducted an online A/B test over two consecutive weeks. We compared USIM against three different baselines: 1. Random, 2. MetaEmb, and 3. ALDI.\nThe results of the online A/B tests on the industrial platform are summarized below.\n\n| A/B Test      | OOV Item PV | OOV Item PCTR | OOV Item GMV |\n|:--:|:--:|:--:|:--:|\n| vs. Random    | +8.20%       | +2.80%         | +20.30%       |\n| vs. MetaEmb   | +6.55%       | +1.95%         | +14.95%       |\n| vs. ALDI      | +4.90%       | +1.10%         | +13.60%       |\n\nThe online recommendation results, it presents that our USIM model significantly outperforms the baselines in all three metrics. Inspired by these A/B test results, USIM is serving mainstream users and providing OOV recommendations for all newly uploaded items.\n\n**Efficiency**\n\n| Method  | LLM Content Feature Extraction Time | OOV Time |\n|:--:|:--:|:--:|\n| MetaEmb | 3.349s $\\pm$ 3.214s  | 0.044s $\\pm$ 0.012   |\n| ALDI    | 3.349s $\\pm$ 3.214s | 0.047s $\\pm$ 0.013   |\n| USIM    | 3.349s $\\pm$ 3.214s | 0.102s $\\pm$ 0.010   |\n\n- **USIM is Not the Bottleneck**: The LLM is the major time consumer, while USIM is comparable to MetaEmb and ALDI.\n- **OOV Recommendation is an Offline, One-Time Process**: As shown in PDF Fig. 1, the OOV recommendation speed does not impact online recommendations, and each item undergoes the OOV process only once.\n- **OOV Recommendation Can Be Parallelized**: The online platform supports parallel processing, allowing USIM to be computed efficiently.\n\n---\n\n>W2. Less significant improvement on MovieLens.\n\nThanks for your comments. USIM provides a new attempt at solving the OOV recommendation problem. We admit that previous models also can achieve good performance. However, USIM can still outperform the best-performing OOV recommendation baselines, especially on CiteULike and Book-Crossing.\n\n---\n>W3 & Q2. Efficiency analysis.\n\nThank you for your insightful comments. We acknowledge the importance of evaluating the time efficiency of USIM, especially in comparison to SOTA baselines. Below, we present the detailed experimental time efficiency for USIM and SOTA methods.\n\n| Dataset  | Method | Total Training Time | Total Converge Epochs | Epoch Time | Inference Time |\n|:--:|:--:|:--:|:--:|:--:|:--:|\n| CiteULike | MetaEmb  | 254s  | 28  | 9.07s | 0.012s  |\n|  | ALDI | 225s  | 51   | 4.41s   | 0.013s     |\n|  | Heater | 841s | 57   | 14.75s    | 0.074s    |\n|  | CLCRec | 926s | 70   | 13.22s    | 0.070s   |\n|  | USIM | 484s | 36   | 13.44s   | 0.031s    |\n| MovieLens | MetaEmb | 415s     | 11      | 37.72s   | 0.005s  |\n|  | ALDI | 664s  | 50     | 13.28s     | 0.004s   |\n|  | Heater | 1474s  | 20    | 73.7s      | 0.072s     |\n|  | CLCRec | 3684s  | 59    | 62.44s     | 0.084s   |\n|  | USIM | 330s | 45   | 7.33s   | 0.045s   |\n\n- **USIM is Faster than Heater and CLCRec**: USIM computes embeddings only for OOV items, whereas Heater and CLCRec must compute embeddings for both OOV and IV items.\n- **USIM is Comparable with MetaEmb and ALDI**: USIM imagines sequences only for OOV items, resulting in inference times comparable to MetaEmb and ALDI.\n- **USIM is Efficient in Training**: By fundamentally addressing OOV recommendation, USIM converges in fewer epochs, making training more efficient.\n\n---\n>Q1. Can the author provide the results of significance tests.\n\nThanks for your suggestion. In our main table, we have already run all experiments 10 times and reported the average performance. Following your advice, we have provided the significance test results in the PDF file.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zVtuCtWNO9", "reviewer": "Reviewer_VoqV", "summary": "The authors propose the USIM (user sequence imagination) framework to tackle the out-of-vocabulary recommendation problem. The authors point out that existing OOV recommendation models such as generative and dropout models will import significant gaps between the IV items and the OOV items, since the OOV items do not have user sequence. Then USIM can imagine user sequence for OOV items to bridge the gap. Generally, I agree USIM can provide better OOV item recommendation performance. \n\nIn terms of technique details:\n- The authors formulate the user sequence imagination process as a MDP, and propose a rec-PPO to optimize the imagination. \n- The authors provide the State Transition Function to describe the embedding optimization process.\n- The authors claim they provide RecPPO to optimize the USIM.\n- The authors already implement USIM on commercial platforms to verify the effectiveness.\n\nIn summary, this paper provide a new prospective of OOV recommendation models, I recommend an accept for this paper. However, there are still some remaining issues as listed in weakness.", "questions": "See weakness", "limitations": "Yes.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The authors effectively summarize the limitations of existing models and accurately identify the core problem of the substantial gap between content features and behavioral embeddings. Their proposed User Sequence IMagination (USIM) framework is a well-reasoned solution, aiming to refine and optimize OOV item embeddings by imagining user sequences.\n- By defining the user sequence imagination process as a Markov Decision Process (MDP), the authors introduce a novel perspective to the problem. The inclusion of a State Transition Function to simulate the embedding optimization of in-vocabulary (IV) items showcases a sophisticated understanding of the recommendation system dynamics and ensures a robust approach to embedding refinement.\n- The authors provide extensive experiments and ablation studies to verify the effectiveness of their proposed solution. The empirical validation on benchmark datasets, along with the demonstration of superior cold-start performance and overall recommendation quality, underscores the practical applicability and robustness of the USIM framework.\n- The implementation of RL-USIM on a major e-commerce platform, where it has been successfully optimizing millions of OOV items and recommending them to billions of users, highlights the real-world impact of this research. This industrial deployment not only demonstrates the scalability and effectiveness of the framework but also confirms its value in a high-stakes, large-scale environment.", "weaknesses": "- The RecPPO, which is a crucial component of the proposed framework, is not clearly defined in Section 3.3.2. Providing more detailed information about RecPPO would help readers better understand its role and implementation within the USIM framework.\n\n- Figure 2, which aims to illustrate the overall USIM process, could be further refined. Enhancing this figure to more clearly and comprehensively present the USIM process would improve the paper’s clarity and help readers grasp the proposed framework's intricacies.\n\n- Although the authors mention that USIM has been deployed on commercial platforms, there is a lack of detailed discussion on its industrial implementation and experiments. Providing insights into the deployment process, challenges faced, and specific industrial experiment results would add valuable context and strengthen the paper’s practical relevance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Weakness and Questions：**\n> W1: The RecPPO, which is a crucial component of the proposed framework, is not clearly defined in Section 3.3.2. Providing more detailed information about RecPPO would help readers better understand its role and implementation within the USIM framework.\n\nThank you for your comments. We incorporate recommendation-specific supervision signals into PPO, referring to this enhanced approach as Recommender-Oriented PPO (RecPPO), to train our USIM. When the state representation $h^i_t$ of specific item equal to its item embedding $e_i$, expected value should be 0. So we use these supervision signals to assist in training the value network $V_{\\omega}$, the specific loss function of the value network is defined as follows:\n\\begin{equation}\n\\begin{aligned}\n    \\mathcal{L}(\\omega)=\\frac{1}{|B|} \\sum_{(s_t, r_t, s_{t+1}) \\in B} \\left[ \\left(r_t + \\gamma V_\\omega(s_{t+1}) - V_\\omega(s_t)\\right)^2 \\right] + \\frac{1}{|\\mathcal{I}|} \\sum_{i \\in \\mathcal{I}} V_\\omega([{e}_i, \\text{random}(0, N)])^2,\n\\end{aligned}\n\\end{equation}\n\nwhere $B$ denotes tuples sampled from the buffer pool, and $\\text{random}(0, N)$ is a random number between 0 and $N$. The first term is the Temporal Difference loss used in value network training, while the second term includes our recommendation-oriented supervision signals. Regardless of previous actions, when the state representation ${h}_t^i$ matches ${e}_i$, the agent should terminate. Therefore, $\\text{random}(0, N)$ serves as the countdown for each termination state. \n    \n\n---\n\n> W2: Figure 2, which aims to illustrate the overall USIM process, could be further refined. Enhancing this figure to more clearly and comprehensively present the USIM process would improve the paper’s clarity and help readers grasp the proposed framework's intricacies.\n\nThank you for your insightful comments, we have revised and improved Fig 2, as illustrated in Fig 2 in the uploaded PDF. The updated figure provides a more detailed depiction of the USIM process, including the composition of the exploration set, the configuration of the reward function, and the state transition function. We hope this enhances your understanding of our work.\n\n---\n\n> W3: Although the authors mention that USIM has been deployed on commercial platforms, there is a lack of detailed discussion on its industrial implementation and experiments. Providing insights into the deployment process, challenges faced, and specific industrial experiment results would add valuable context and strengthen the paper’s practical relevance.\n\nThanks for your comments, we have conducted a two-week A/B test on billion-scale recommender systems and we'd like to provide the details.\n\n**Platform** We have implemented our USIM on the homepage of one of the largest e-commerce platforms, which boasts hundreds of millions of users and billions of items. The homepage features a feed recommendation system that recommends items to users. Thousands of new items are uploaded every hour.\n\n**Framework:** Our online implementation consists of two core components: 1. Online Recommendation; and 2. USIM Imagination. We present our framework in Fig 1 in the uploaded PDF. When an OOV item is uploaded, we utilize the Large Language Model to embed the content features, including the product name and description. We then employ the USIM structure to predict the most suitable user sequence and optimize the embedding accordingly. Finally, we use the USIM-produced as the IV embeddings in the online recommendation model. \n\n**Baseline Comparisons:** We conducted an online A/B test on 5% of users for each group over two consecutive weeks. We compared USIM against three different baselines: 1. Random, 2. MetaEmb, and 3. ALDI.\nThe results of the online A/B tests on the industrial platform are summarized below.\n\n| A/B Test      | OOV Item PV | OOV Item PCTR | OOV Item GMV |\n|---------------|--------------|----------------|---------------|\n| vs. Random    | +8.20%       | +2.80%         | +20.30%       |\n| vs. MetaEmb   | +6.55%       | +1.95%         | +14.95%       |\n| vs. ALDI      | +4.90%       | +1.10%         | +13.60%       |\n\nThe online recommendation results, it presents that our USIM model significantly outperforms the baselines in all three metrics. Inspired by these A/B test results, USIM is serving mainstream users and providing OOV recommendations for all newly uploaded items.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YhIFckn5BI", "reviewer": "Reviewer_bkbE", "summary": "This paper presents a novel Out-of-Vocabulary item recommendation or so-called cold-start recommendation model.\n\nThe OOV recommendation problem especially item recommendation is very important, since in short video recommendation platforms there are thousands of new post videos or new AGI videos being published every second. OOV Item recommendation are so important in dealing with the flourish of the new videos. User Sequence Imagination (USIM) provides a new paradigm for OOV recommendation to imagine behaviors of OOV items. I do like this paper, since this paper provides a totally novel and promising solution in solving OOV recommendations, of which the paradigm is quite different from traditional works. Traditional models such as generative models and dropout models do not solve the OOV recommendation fundamentally. USIM enable the OOV items to be recommended the same as In-Vocabulary items. I think this approach has the potential to fundamentally resolve the OOV issue.", "questions": "Q1. The paper mentioned that USIM has been deployed on prominent e-commerce platforms. Are there any AB-Test results or some results from the industrial side?\n\nQ2. Could you discuss whether this model can be used for short video recommendation platforms?", "limitations": "Yes", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "This paper has the following strengths:\n1. New Paradigm. The User Item Imagination is a branch new solution that can solve the OOV problem. UISM can imagine user sequences for OOV items and thus solve the OOV recommendation fundamentally.\n2. Reasonable Solution. This paper proposes an RL-based USIM solution, which formulates the sequential optimization as a Markov Decision Process and introduces recommender-oriented PPO (RecPPO) to maximize the final recommendation performance of the OOV items.\n3. Nice presentation and solid experiments. The Figure 1 is quite clear and insightful. Experiments are solid, comparing a wide range of baselines across various datasets.", "weaknesses": "1. Figure 2 can be further improved. The presentation of Figure 1 is excellent, why not keep Figure 2 in the same style?\n2. There may be some typos in Equation 4. The optimization objective should be to maximize the score between e_i and e_u.\n3. The fonts in Figure 4 are quite small. Besides, it’s better to modify the Table 1 to be around the main results, rather than the experimental setup.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Weakness：**\n> W1. Figure 2 can be further improved. The presentation of Figure 1 is excellent, why not keep Figure 2 in the same style?\n\nThank you for your advice. We have revised Figure 2 in our updated PDF file. We hope this revised figure provides a clearer understanding of our work.\n\n---\n> W2&W3. Typos and suggestions\n\nThank you for pointing these out, we will polish our paper in the revision.\n\n---\n> Q1: The paper mentioned that USIM has been deployed on prominent e-commerce platforms. Are there any AB-Test results or some results from the industrial side?\nThanks for your comments, we have conducted a two-week A/B test on billion-scale recommender systems and we'd like to provide the details.\n\n**Platform** We have implemented our USIM on the homepage of one of the largest e-commerce platforms, which boasts hundreds of millions of users and billions of items. The homepage features a feed recommendation system that recommends items to users. Thousands of new items are uploaded every hour.\n\n**Framework:** Our online implementation consists of two core components: 1. Online Recommendation; and 2. USIM Imagination. We present our framework in Fig 1 in the uploaded PDF. When an OOV item is uploaded, we utilize the Large Language Model to embed the content features, including the product name and description. We then employ the USIM structure to predict the most suitable user sequence and optimize the embedding accordingly. Finally, we use the USIM-produced as the IV embeddings in the online recommendation model. \n\n**Baseline Comparisons:** We conducted an online A/B test on 5% of users for each group over two consecutive weeks. We compared USIM against three different baselines: 1. Random, 2. MetaEmb, and 3. ALDI.\nThe results of the online A/B tests on the industrial platform are summarized below.\n\n| A/B Test      | OOV Item PV | OOV Item PCTR | OOV Item GMV |\n|---------------|--------------|----------------|---------------|\n| vs. Random    | +8.20%       | +2.80%         | +20.30%       |\n| vs. MetaEmb   | +6.55%       | +1.95%         | +14.95%       |\n| vs. ALDI      | +4.90%       | +1.10%         | +13.60%       |\n\nThe online recommendation results demonstrate that our USIM model significantly outperforms the baselines in all three metrics. Inspired by these A/B test results, USIM is serving mainstream users and providing OOV recommendations for all newly uploaded items.\n\n---\n\n>Q2.Could you discuss whether this model can be used for short video recommendation platforms?\n\nThank you for your interest in applying our model in short video recommendations. Our model can be applied to OOV recommendations in short videos. We obtain the item content information, such as tags, titles, and covers, and use pre-trained models to encode them and obtain the representations. These representations are then input into the USIM model, which generates the corresponding embeddings for each OOV item.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We sincerely thank all reviewers for their insightful reviews. OOV item recommendation is increasingly important in the age of information explosion and AGI. We are honored to share our findings and engage in deeper discussions with all the AC and reviewers.

We appreciate your recognition of our core strengths:

- **New OOV Item Recommendation Paradigm**: We tackle the OOV problem by imagining user sequences and optimizing with real user behavioral embeddings, rather than directly generating behavioral embeddings from content features.
- **Novel and Effective RL-Based OOV Approach**: We formulate the user sequence imagination as an MDP process, achieving promising OOV recommendation performance with an RL strategy.
- **Online Platform Application**: USIM has been implemented on one of the largest e-commerce platforms, providing OOV recommendations for all newly uploaded items on a billion-scale platform.


To better support our paper, we have added the following supplementary materials during this rebuttal process:

- **Online Implementation Framework:** We propose the introduction of our online platform and introduce the online framework that integrates seamlessly with existing systems, efficiently processing OOV items. The implemented framework is illustrated in the attached PDF. 
- **A/B Test Results:**  We conducted a two-week A/B test on a large e-commerce platform to validate our claims, demonstrating significant improvements over baseline methods. 
- **Improved Framework Figure:** We have enhanced the figure to clearly illustrate the components of our framework and their interactions.
- **Additional Large-Scale Dataset:** We included experiments on the Book-Crossing dataset, providing further evidence of the robustness and scalability of our approach.
- **Efficiency Experiment:** We evaluated the time efficiency of our method compared to state-of-the-art baselines, showing that our approach is not the bottleneck in processing.
- **Evidence and Literature Support:** We added references and empirical evidence to address concerns about gaps between content features and behavioral embeddings.

Thank you again for all the reviewers' suggestions, which have greatly improved our paper. For specific details, please kindly check **the corresponding responses and the attached PDF file**.

Meta-review: Solid reviews, all reviewers recommend acceptance (even if some more enthusiastically than others). Reviewers find the problem important, the enhancements made here to be innovative / novel, and the experiments are convincing. Minor comments are made about experiments, but these seem easy enough to deal with.
Decision: Accept (spotlight)