Pre-train, Prompt, and Recommendation: A Comprehensive Survey of Language Modeling Paradigm Adaptations in Recommender Systems (2023)

Abstract: The emergence of Pre-trained Language Models (PLMs) has achieved tremendous success in the field of Natural Language Processing (NLP) by learning universal representations on large corpora in a self-supervised manner. The pre-trained models and the learned representations can be beneficial to a series of downstream NLP tasks. This training paradigm has recently been adapted to the recommendation domain and is considered a promising approach by both academia and industry. In this paper, we systematically investigate how to extract and transfer knowledge from pre-trained models learned by different PLM-related training paradigms to improve recommendation performance from various perspectives, such as generality, sparsity, efficiency and effectiveness. Specifically, we propose a comprehensive taxonomy to divide existing PLM-based recommender systems w.r.t. their training strategies and objectives. Then, we analyze and summarize the connection between PLM-based training paradigms and different input data types for recommender systems. Finally, we elaborate on open issues and future research directions in this vibrant field.

## introduction as an important part of the online environment, recommender systems (rss) play a key role in discovering users' interests and alleviating information overload in their decision-making process. recent years have witnessed tremendous success in recommender systems empowered by deep neural architectures and increasingly improved computing infrastructures. however, deep recommendation models are inherently data-hungry with an enormous amount of parameters to learn, which are likely to overfit and fail to generalize well in * equal contribution. practice when their training data (i.e., user-item interactions) are insufficient. such scenarios widely exist in practical rss when a large number of new users join in but have fewer interactions. consequently, the data sparsity issue becomes a major performance bottleneck of the current deep recommendation models. with the thriving of pre-training in nlp @xcite , many language models have been pre-trained on large-scale unsupervised corpora and then fine-tuned in various downstream supervised tasks to achieve state-of-the-art results, such as gpt @xcite , and bert @xcite . one of the advantages of this pre-training and fine-tuning paradigm is that it can extract informative and transferrable knowledge from abundant unlabelled data through selfsupervision tasks such as masked lm @xcite , which will benefit downstream tasks when the labelled data for these tasks is insufficient and avoid training a new model from scratch. a recently proposed paradigm, prompt learning @xcite , further unifies the use of pre-trained language models (plms) on different tasks in a simple yet flexible manner. in general, prompt learning relies on a suite of appropriate prompts, either hard text templates @xcite , or soft continuous embeddings @xcite , to reformulate the downstream tasks as the pre-training task. the advantage of this paradigm lies in two aspects: (1) it bridges the gap between pre-training and downstream objectives, allowing better utilization of the rich knowledge in pre-trained models. this advantage will be multiplied when very little downstream data is available. (2) only a small set of parameters are needed to tune for prompt engineering, which is more efficient. motivated by the remarkable effectiveness of the aforementioned paradigms in solving data sparsity and efficiency issues, adapting language modeling paradigms for recommendation is seen as a promising direction in both academia and industry, which has greatly advanced the state-ofthe-art in rss. although there have been several surveys on pre-training paradigms in the fields of cv @xcite , nlp @xcite , and graph learning @xcite , only a handful of literature reviews are relevant to rss. @xcite summarize some research on the pre-training of recommendation models and discusses knowledge transfer methods between different domains. but this paper only covers a small number of bert-like works and does not go deep into the training details of pre-trained recommendation models. @xcite give a brief overview of the advances of self-supervised learning in rss. however, its focus is on a purely self-supervised recommendation setting, which means the supervision signals used to train the model are semi-automatically generated from the raw data itself. our work does not strictly focus on the self-supervised training strategies but also incorporates the adaptation and exploration of supervised signals and data augmentation techniques in the pre-training, fine-tuning, and prompting process for various recommendation purposes. furthermore, none of them systematically analyzed the relationship between different data types and training paradigm choices in rss. to the best of our knowledge, our survey is the first work that presents an up-to-date and comprehensive review of language modeling paradigm adaptations for recommender systems (lmrs). 1 the main contributions of this paper are summerized as follows: ## data types encoding input data as embeddings is usually the first step in recommendations. however, the input for recommender systems is more diverse than most nlp tasks, and therefore encoding techniques and processes may need to be adjusted to align with different input types. textual data as a powerful medium of spreading and transmitting knowledge are commonly used as input for modeling user preferences. examples of textual data include reviews, comments, summaries, news, conversations, and codes. note that we also consider item metadata and user profiles as a kind of textual data for simplicity. sequential data, such as user-item interactions strictly arranged chronologically or in a specific order, are used as sequential input for sequential and session-based recommender systems. graphs, usually containing different semantic information from other types of data inputs such as the user-user social graph or heterogeneous knowledge graph, are also commonly used to extract structural knowledge to improve recommendation performance. the diversity of online environments promotes the generation of massive multimedia content, which has been shown to improve recommendation performance in numerous research works. therefore, multi-modal data such as images, videos and audios can also be importance sources for lmrs. multi-modal data plays a crucial role in recommendation systems. however, the utilization of multi-modal data in lmrs papers is scarce, possibly due to the absence of accessible datasets. a few scholars have gathered their individual datasets to facilitate text-video-audio tri-modal music recommendations @xcite or to establish benchmarks for shopping scenarios @xcite . ## training strategies of lmrs given the significant impact that plms have had on nlp tasks in the pre-train and fine-tune paradigm, there has been a surge recently in adapting such paradigms to multiple recommendation tasks. as illustrated in figure ## formulating training with data types to associate training strategy, learning objectives with different input data types, we summarize representative works in this domain in the listed training strategies and objectives are carefully selected and are typical in existing work. for the page limit, we only selected part of recent research on lmrs. for more research progress and related resources, please refer to https:// github.com/smartmediaai/lmrs . considering that datasets are another important factor for empirical analysis of lmrs approaches, in training strategy paper learning objective recommendation task data type source code pre-training & fine-tuning pre-training w/o fine-tuning (sun et al., 2019) pre-train: mlm sequential rs sequential data link (geng et al., 2022b) pre-train: am explainable rs graph n/a (de souza pereira moreira et al., 2021) pre-train: am + mlm + perlm + rtd session-based rs textual + sequential data link fine-tuning holistic model (kang et al., 2021) pre-train: cross-entropy cross-library api rs textual data (code) link fine-tune: cross-entropy @xcite pre-train: am conversational rs textual data + graph link fine-tune: am + cross-entropy @xcite pre-train: am + mlm news rs textual + sequential data link fine-tune: negative sampling loss @xcite pre-train: mlm + nt-xent social rs textual data link fine-tune: negative sampling loss @xcite pre-train: mnp + mep + cross-entropy + top-n rs graph n/a contrastive loss; fine-tune: cross-entropy fine-tuning partial model @xcite pre-train: contrastive loss cross-domain rs textual + sequential data link fine-tune: cross-entropy sequential rs @xcite pre-train: mlm + am news rs textual + sequential data link fine-tune: cross-entropy + mse + infonce @xcite pre-train: mmm + map news rs sequential + multi-modal data link fine-tune: cross-entropy fine-tuning external part @xcite pre-train: mim sequential rs textual + sequential data link fine-tune: pairwise ranking loss @xcite pre-train: mtp + cross-entropy news rs textual + sequential data link fine-tune: cross-entropy @xcite pre-train: binary cross-entropy medication rs graph link fine-tune: cross-entropy @xcite pre-train: binary cross-entropy top-n rs textual data + graph link fine-tune: bpr + binary cross-entropy prompting fixed-ptm prompt tuning @xcite pre-train: am + mlm + cross-entropy conversational rs textual data link prompt-tuning: am + cross-entropy @xcite ) pre-train: pairwise ranking loss cross-domain rs textual + sequential data n/a prompt-tuning: pairwise ranking loss + sequential rs contrastive loss fixed-prompt ptm tuning (yang et al., 2022a) pre-train: am + mlm conversational rs textual data link ptm fine-tune: am + cross-entropy (deng et al., 2023) pre-train: am; ptm fine-tune: am conversational rs textual data link tuning-free prompting (sileo et al., 2022) pre-train: am zero-shot rs textual data link (geng et al., 2022c) pre-train: am zero-shot rs textual + sequential data link cross-domain rs prompt+ptm tuning (li et al., 2023b) pre-train: am; prompt-tuning: nll explainable rs textual data link prompt+ptm tuning: nll + mse (xin et al., 2022) prompt+ptm tuning: cross-entropy next item rs sequential data n/a note. nt-xent: normalized temperature-scaled cross entropy loss; mmm: masked multi-modal modeling; map: multi-modal alignment prediction; mim: mutual information maximization loss; mtp: masked news/user token prediction; nll: negative log-likelihood loss. ## discussion and future directions despite the effectiveness of lm training paradigms has been verified in various recommendation tasks, there are still several challenges that could be the future research directions.