C o VE : Compressed Vocabulary Expansion Makes Better LLM -based Recommender Systems (2025)

Abstract: Recommender systems play a pivotal role in providing relevant content to users. With the rapid development of large language models (LLMs), researchers have begun utilizing LLMs to build more powerful recommender systems. However, existing approaches that focus on aligning LLMs with recommendation tasks do not fully leverage their sequential information processing capabilities, leading to suboptimal performance. In this paper, we propose a novel system called compressed vocabulary expansion (CoVE). In CoVE, each item is assigned a unique ID within the expanded vocabulary. Our framework effectively capitalizes on sequence understanding abilities of LLMs, significantly enhancing their performance on recommendation tasks. Additionally, we compress the embedding layer, making CoVE practical for large-scale industrial applications. The effectiveness and performance of CoVE are demonstrated through comprehensive experiments on multiple recommendation datasets and comparisons with prior works. Our code can be found at https://github.com/HaochenZhang717/CoVE-official-Repo .

## introduction large language models (llms), like gpt series @xcite @xcite , llama @xcite , have demonstrated strong capabilities for reasoning and problem-solving @xcite @xcite . llms are typically pretrained on a large corpus and then finetuned on smaller datasets for downstream tasks @xcite @xcite . the pretrain-and-finetune paradigm achieves the state-of-the-art performance in many tasks @xcite . on the other hand, in current big data era, people increasingly face challenges related to information overload @xcite . this brings sequential recommendation problems, where a recommender system analyzes a user's historical interaction data to predict the next item that is likely to interest them @xcite . with the rise of llms, a large body of research @xcite @xcite @xcite has been dedicated to developing proper frameworks to effectively leverage llms for sequential recommender systems. among existing works, there are two main ways to utilize llms for recommender systems. one approach is to use embeddings provided by llms to initialize non-llm recommender systems. for example, @xcite ; @xcite show that llm embeddings improve the performance of several non-llm deep learning sequential recommender models @xcite @xcite . despite the positive results from this pipeline, using llms solely as embedding providers takes advantage of their embedding ability but does not effectively leverage their content comprehension capability. this can potentially result in suboptimal performance. the other approach is finetuning llms to directly provide recommendations @xcite which use the pipeline consisting of 1) finetuning llms on a recommendation dataset, 2) using finetuned llms to generate the next item's title, and 3) conducting embedding retrieval to avoid hallucinated item titles. @xcite shows that this pipeline produces more accurate recommendations than prior works: it leverages llms' token prediction ability, which may address the limitations of the first line of work. however, there are issues in the design of this framework. 1) under this framework, llm needs to accurately predict the next item's title, which consists of multiple tokens, and this is a difficult the user has bought the following items before: 
: chair, 
: wallet, 
: kitchen paper. recommend one item to them. as an alternative to the two broad lines of llmbased recommendation, we propose a novel system herein called compressed vocabulary expansion (cove) which is significantly different from the existing pipeline. cove first expands the tokenizer's vocabulary and embedding table by assigning each item in the item space a unique token in the tokenizer and a corresponding unique embedding in the embedding table. during finetuning, the item embedding table is updated together with the lora adapter @xcite to align the pretrained llm with the recommendation task, as shown in figure 1 . during inference, cove can provide a recommendation by generating one token representing the unique id of the next item, instead of generating the full title. the advantage of cove over existing frameworks is three-fold: the inspiration for cove comes from vocabulary expansion for domain-specific llm alignment @xcite @xcite , where it has been shown that expanding the vocabulary in a downstream domain improves an llm's ability in content comprehension and generation. however, in cove the memory cost of an additional embedding table is prohibitive in largescale industrial scenarios, where the item space may contain millions of products. for instance, the newest version of the amazon review dataset contains 48.19 million items in total foot_0 . when finetuning on such a large dataset, the item embedding table itself takes up approximately 96gb in gpu memory, which makes training infeasible for even the most powerful gpus. therefore, a key research question that we need to address in this paper is: how can we make cove memory-efficient while preserving its superior performance compared to the state-of-the-art results? the solution to this problem is using embedding layer compression methods which have been explored by prior works @xcite @xcite to alleviate the memory problem brought by large embedding tables. inspired by them, we propose to use a hash function to compress the item embedding table to make cove more memory efficient. in summary, the contribution of this study is as follows: ## proposed framework finetuning task formulation our goal is to leverage an llm to construct a recommender system. to achieve this, we finetune the llm on a recommendation dataset. the key difference between our finetuning task and common finetuning tasks is that we also tune the embedding table and lm_head. cove as shown in figure 1 , cove involves tokenizer vocabulary expansion, item embedding table expansion, and a low-rank weight adapter. the core idea of cove is to assign each item a unique embedding. to achieve this, we expand the tokenizer's vocabulary by adding item ids, such @xmath0, 
, @xmath1. these item ids are then encoded as distinct embeddings. cove finetunes item embeddings, weights in transformer blocks, and weights in lm_head to align pretrained llms with sequential recommendation tasks. hashing-based embedding compression due to the large item space, item embeddings must be compressed to enable efficient gpu training. we propose a hashing-based embedding compression technique, inspired by @xcite . let i denote the item space and s the latent shared item space, where |s| ≪ |i|. we define h as a universal hash family, where each hash function h : @xmath2. we sample k such hash functions, denoted as h 1 , . . . , h k . each shared item @xmath3, and an item's embedding is obtained by averaging the embeddings of its hashed counterparts in the shared space. specifically, the embedding for an item i is given by we select a universal hash function based on simple arithmetic operations-addition, multiplication, and modulo-which can be efficiently computed on gpus. specifically, given an item's index i, the hash code for its shared embedding is computed as ## a case study in this section, we analyze what an llm can learn after finetuning under the cove framework. to show this, we give an example of the input and the generated output of finetuned llm by cove: the generated content strictly follows the format of the user's interaction history, i.e., each item's id is followed by its title. we find that the finetuned llm consistently outputs the correct item title corresponding to its id. this indicates that under the cove framework, the finetuned llm learns the mapping between an item's id and its title. we argue that this is crucial for high-quality recommendations. one key advantage of cove is its ability to convert the item title prediction task into an item id prediction task. while the latter is easier than the former-since the number of tokens to predict is significantly smaller-item id prediction is meaningful only when the llm correctly understands the relationship between ids and titles. llms in sequential recommendation currently, llms are primarily utilized in recommendation systems in two ways: providing embeddings for items @xcite @xcite and directly generating recommendations @xcite @xcite @xcite . our work is more related to generative recommendation systems. for instance, @xcite proposes a finetuning framework to align llms with recommendations, where the task is defined as a yes-no question-answering problem. however, under this framework, generating the next item for users is challenging. gpt4rec @xcite integrates llms with a search engine, where the llm generates multiple queries representing the user's interests, and the search engine outputs the final recommendation. the existing work most similar to our approach is the finetune-retrieval framwork adopted in @xcite . during training, bigrec finetunes an llm on a recommendation dataset. during inference, bigrec computes the distances between the embedding of the sentence generated by the finetuned llm and the embeddings of all items in the item space. based on these distances, bigrec determines the next recommended item for the user. our approach is similar to bigrec in the training stage; however, in the inference stage, we do not need to generate content using llms. instead, we provide recommendations directly based on logits of items. this approach reduces inference time and eliminates hallucinations. the trade-off is a larger embedding layer, which we mitigate through embedding layer compression. @xcite studies the long-tail problem in sequential recommendation by enhancing item embeddings for rare items. it uses a pre-trained llm (llama-7b) and adapts it to the recommendation domain via contrastive fine-tuning (scft) and recommendation adaptation, and focuses on preserving semantic richness while integrating collaborative signals. ## conclusion we propose a novel framework, cove, to better align llms with sequential recommendation problems. the core idea of cove is to assign each item a unique id that can be encoded as a token and given a unique embedding. the item embedding table is finetuned along with the weights in transformer blocks during training. cove outperforms baselines by up to 62% in ng@5, ng@10, hr@5, and hr@10 on amazon review datasets. additionally, cove achieves approximately 100× speed-up in inference time compared to the existing finetuneand-retrieval framework. through comprehensive ablation experiments, we find that including an item's textual information enhances cove's performance. from case studies, we observe that cove effectively memorizes id-title pairs for different items, which is crucial for providing high-quality recommendations. in this paper, we used ai tools solely for language assistance.