Stealthy Attack on Large Language Model based Recommendation (2024)

Abstract: Recently, the powerful large language models (LLMs) have been instrumental in propelling the progress of recommender systems (RS). However, while these systems have flourished, their susceptibility to security threats has been largely overlooked. In this work, we reveal that the introduction of LLMs into recommendation models presents new security vulnerabilities due to their emphasis on the textual content of items. We demonstrate that attackers can significantly boost an item’s exposure by merely altering its textual content during the testing phase, without requiring direct interference with the model’s training process. Additionally, the attack is notably stealthy, as it does not affect the overall recommendation performance and the modifications to the text are subtle, making it difficult for users and platforms to detect. Our comprehensive experiments across four mainstream LLM-based recommendation models demonstrate the superior efficacy and stealthiness of our approach. Our work unveils a significant security gap in LLM-based recommendation systems and paves the way for future research on protecting these systems. 1

text: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5839–5857 August 11-16, 2024 ©2024 Association for Computational Linguistics
1 Introduction
Over the past few decades, recommender systems (RS) have gained considerable significance across various domains. Recently, the powerful large language models (LLMs) have been instrumental in propelling the progress of recommender systems. There has been a notable upswing of interest dedicated to developing LLMs tailored for recommendation task.
Contrary to traditional recommendation models, which rely heavily on abstract and less interpretable ID-based information, LLM-based recommendation models exploit the semantic understanding
*To whom correspondence should be addressed. 1The source code will be released at https://github.
com/CRIPAC-DIG/RecTextAttack.
and strong transferability of LLMs. This approach places a heightened focus on the textual content of items, such as titles and descriptions (Lin et al., 2023a; Chen et al., 2023). For instance, many researchers (Hou et al., 2022, 2023a; Yuan et al., 2023; Li et al., 2023a; Yang et al., 2023; Geng et al., 2022; Cui et al., 2022; Bao et al., 2023a; Zhang et al., 2023b; Li et al., 2023b; Zhang et al., 2023c) have explored modeling user preferences and item characteristics through a linguistic lens. This methodology promises a revolutionary shift in the conventional paradigm of recommendations by providing generalization capabilities to novel items and datasets.
Despite these advancements, the security of RS remains a largely unaddressed issue. Malicious attacks on these systems can lead to undesirable outcomes, such as the unwarranted promotion of low-quality products in e-commerce platforms or the spread of misinformation in news dissemination contexts. Traditional shilling attack strategies on RS (Wang et al., 2023a, 2024c) involve the generation of fake users who are programmed to give high ratings to specific target items. By introducing such cheating data, it aims at influencing the training of the recommender models and subsequently
5839
increasing the exposure of the target items. However, the introduction of LLMs into recommendation models presents new security vulnerabilities. In this paper, to the best of our knowledge, we are the first to demonstrate that LLM-based recommendation systems are more vulnerable due to their emphasis on the textual content of items. We demonstrate that attackers can significantly boost an item’s exposure by merely altering its textual content during the testing phase, utilizing simple heuristic re-writing or black-box text attack strategies (Morris et al., 2020). Compared with traditional shilling attacks, this attack paradigm is notably stealthy, as it does not require influencing the training of the model, and the overall recommendation performance is almost unchanged. Moreover, the modifications to the title are subtle, making it difficult for users and platforms to detect.
We construct comprehensive experiments on four mainstream LLM-based recommendation models (Geng et al., 2022; Bao et al., 2023a; Li et al., 2023a; Zhang et al., 2023c) as victim models to validate the outstanding efficacy and stealthiness of the textual attack paradigm compared with traditional shilling attacks (Burke et al., 2005b; Kaur and Goel, 2016; Lin et al., 2020). We further delve into the effects of model fine-tuning and item popularity on the attack. Additionally, we investigate the transferability of the attack across various victim models and recommendation tasks to demonstrate its practical applicability and utility in real-world scenarios. Finally, we evaluate a simple re-writing defense strategy, which also can mitigate the issue to some extent.
To summarize our contributions:
1. We highlight that LLM-based recommendation models, due to their emphasis on textual content information, could raise previously overlooked security issues.
2. To the best of our knowledge, we are the first to attack LLM-based recommendation models and propose the use of textual attacks to promote the exposure of target items.
3. We perform extensive experiments to demonstrate the efficacy and stealthiness of the textual attack paradigm. Further experiments have revealed the impact of item popularity and model fine-tuning on attacks, as well as explored the transferability of attacks.
4. Finally, we proposed a simple rewriting defense strategy. While it cannot fully defend against text-based attacks, it can provide some level of defense and contribute to future research.
2 Method
In this section, we first introduce the LLM-based recommendation model and formulate the objectives of the attacks. Then, we present two simple model-agnostic text rewriting approaches. Finally, we provide a detailed introduction of black-box text attacks.
2.1 Problem Definition
We use the notation I = {i1, · · · , iN} and U = {u1, · · · , uM} to represent the sets of N items and M users, respectively. Each item i ∈ I is associated with textual content ti. Each user u ∈ U has interacted with a number of items Iu, indicating that the preference score yui = 1 for i ∈ Iu.
LLM-based RS models user preference and item feature by transforming user historical behavior sequences Iu and target item i into textual prompt Pu,i = [tu, ti, xu, xi], where tu =[ ti1 , · · · , ti|Iu| ] . xu and xi denotes the ID of i and u which are optional in LLM-based RS. We have listed example prompts of four victim models in Table 1. Please refer to Section 3.1.1 for the details of them. The recommendation process can be formulated as: ŷu,i = fθ(Pu,i) where fθ denotes the LLM-based model.
The goal of the attack task is to promote target items I ′ (increasing the exposure or user interaction probability) through imperceptibly modifying their textual content (we use title in this work).
2.2 Victim Model-Agnostic Attack
In this subsection, we first introduce two simple, victim model-agnostic strategies employed for altering item textual content to make them more linguistically attractive to users. Our approaches include trivial attack with word insertion and rewriting leveraging Generative Pre-trained Transformers (GPTs).
2.2.1 Trivial Attack with Word Insertion
The core premise of this strategy is founded on the assumption that positive or exclamatory words can attract users. By infusing item titles with a select number of positive words, we aim to increase the items’ attractiveness and, consequently, their likelihood of being recommended by the system. Specifically, we randomly select k words form a pre-defined word corpus which is common-used in item titles. These selected words are then inserted to the end of the original text content to retain the overall coherence.
Positive word corpus: [‘good’, ‘great’, ‘best’, ‘nice’, ‘excellent’, ‘amazing’, ‘awesome’, ‘fantastic’, ‘wonderful’, ‘perfect’, ‘ultimate’, ‘love’, ‘like’, ‘beautiful’, ‘well’, ‘better’, ‘easy’, ‘happy’, ‘recommend’, ‘works’, ‘fine’, ‘fast’, ‘fun’, ‘price’, ‘quality’, ‘product’, ‘value’, ‘bought’, ‘purchase’, ‘top’, ‘popular’, ‘choice’, ‘!!!’ ]
2.2.2 Re-writing with GPTs
While the insertion of positive words offers a straightforward means of enhancing content appeal, it can sometimes result in awkward or forced phrasings that diminish the content’s natural flow and potentially arouse user suspicion. To address these shortcomings, we propose to use GPTs to rewrite the content of items in a more attractive way by leveraging its rich common sense knowledge and powerful generation capabilities. Specifically, we instruct GPT-3.5-turbo with the following prompts to generate attractive and fluency titles.
Prompt 1: You are a marketing expert that helps to promote the product selling. Rewrite the product title in <MaxLen> words to keep its body the same but more attractive to customers: <ItemTitle>. Prompt 2: Here is a basic title of a product. Use your creativity to transform it into a catchy and unique title in <MaxLen> words that could attract more attention: <ItemTitle>. Prompt 3: Rewrite this product’s title by integrating positive and appealing words, making it more attractive to potential users without altering its original meaning (in <MaxLen> words): <ItemTitle>.
2.3 Exploring Vulnerabilities in LLM-Based Recommendation Models through Black-Box Text Attacks
In this subsection, we present an examination of traditional black-box text attack methods to explore the vulnerabilities within LLM-based recommendation models. Black-box text attack methods typically involve manipulating or perturbing text inputs to deceive or mislead a natural language processing (NLP) model while having no access to the model’s internal parameters or gradients. The goal of such attacks is mathematically formulated as:
argmax t′i
Eu∈U ′ fθ(P ′u,i), (1)
where P ′u,i = [tu, t′i, xu, xi] denotes the prompts consisting of the user text tu and the manipulated title of the target item t′i. Following the framework proposed by Morris et al. (2020), text attacks are comprised of four principal components:
• Goal Function: This function evaluates the effectiveness of the perturbed input x′ in achieving a specified objective, serving as a heuristic for the search method to identify the optimal solution. In this study, the aim is to promote the target items as in Equation 1.
• Constraints: These are conditions that ensure the perturbations remain valid alterations of the original input, emphasizing aspects such as semantic retention and maintaining consistency in part-of-speech tags.
• Transformation: A process that applies to an input to generate possible perturbations, which could involve strategies like swapping
words with similar ones based on word embeddings, using synonyms from a thesaurus, or substituting characters with homoglyphs.
• Search Method: This method involves iteratively querying the model to select promising perturbations generated through transformations, employing techniques such as a greedy approach with word importance ranking, beam search, or a genetic algorithm.
While the specific components of text attack methodologies may vary, the overarching framework remains consistent, as depicted in Algorithm 1. In this work, we have implemented four widelyused attacks: DeepwordBug (Gao et al., 2018), TextFooler (Jin et al., 2020), BertAttack (Li et al., 2020), and PuncAttack (Formento et al., 2023). DeepwordBug and PuncAttack are character-level which manipulate texts by introducing typos and inserting punctuation. TextFooler and BertAttack are word-level that aim to replace words with synonyms or contextually similar words. Please refer to the appendix B for the details of text attack paradigm and these four methods.
Algorithm 1 Text Attack Framework Require: Original text x, Target model M , Goal
function G, Constraints C, Transformations T , Search Method S Ensure: Adversarial text x′, Adversarial score G.score(x′)
1: Initialize x′ as a copy of x. 2: while not S.StoppingCriteria() do 3: Select a transformation t from allowable transformations T . 4: Generate x′ by applying t to x. 5: if C.Satisfied(x′) then 6: if S.AchieveGoal(x′) then 7: return x′, G.score(x′). 8: end if 9: end if
10: end while
3 Experiments
3.1 Experimental Settings 3.1.1 Victim Models We choose four mainstream LLM-based recommendation models as our victim models: Recformer (Li et al., 2023a), P5 (Geng et al., 2022), TALLRec (Bao et al., 2023a) and CoLLM (Zhang
et al., 2023c). Please refer to Appendix A.1 for more details.
3.1.2 Compared Shilling Attacks Shilling attacks aim to generate fake users that assign high ratings for a target item, while also rating other items to act like normal users for evading. We compare our text attack paradigm with white-box shilling attacks Random attack (Kaur and Goel, 2016), Bandwagon attack (Burke et al., 2005b), and gray-box Aush (Lin et al., 2020) and Leg-UP (Lin et al., 2024). Please refer to Appendix A.2 for more details.
3.1.3 Datasets We conduct experiments on three categories of widely-used (Li et al., 2023a; Geng et al., 2022; Bao et al., 2023a; Zhang et al., 2023c) Amazon review dataset introduced by McAuley et al. (2015): ‘Beauty’, ‘Toys and Games’, ’Sports and Outdoors’, which are named as Beauty, Toys and Sports in brief. We use the 5-core version of Amazon datasets where each user and item have 5 interactions at least. The statistics of these datasets are summarized in Appendix A.3.
3.1.4 Implementation Details All victim models and compared shilling attacks are implemented in PyTorch. We random select 10% items as target items. For more implementation details, please refer to Appendix A.3.
3.1.5 Evaluation metrics We evaluate the attack from two aspects: effectiveness and stealthiness.
Effectiveness. This metric gauges the extent to which our methodology can promote the specified target items.
• Exposure. For the victim model RecFormer, which allows for full ranking, we employed a direct metric, exposure rate. We define the exposure rate as expi = N irec Nu
, where Nu represents the total number of users, and N irec denotes the count of users for whom target item i appears in their top-K (K = 50 by default) recommendation list.
• Purchasing propensity. For other three victim models which could not conduct full ranking, we define the purchasing propensity of item i as pi = Eu∈U ŷui, where ŷui denotes the predicted probability that user u tends to interact with item i.
• # queries. In a black-box scenario, the adversary need query the victim model in order to detect any alterations in the output logits. The lower the value, the more effective the attack will be.
Stealthiness. The stealthy attack aims to promote target items while maintaining imperceptibility, thereby avoiding detection by users and platforms. Therefore, the coherence and authenticity of our generated content are crucial to uphold.
• Overall recommendation performance. An ideal stealthy attack should keep the overall recommendation performance unchanged. The recommendation performance includes Recall@K, NDCG@K and AUC.
• Text quality. The generated adversarial content should be of high quality that are acceptable to users. Firstly, it should be consistent with the corresponding item. We measure the cosine semantic similarity and ROUGE scores between the original content and adversarial content for this purpose. Secondly, the adversarial content itself should be readable. We assess the fluency of the adversarial title, measured by the perplexity of GPT-Neo (Black et al., 2021).
• # perturbed words. The number of words
changed on an average to generate an adversarial content. The lower the value, the more imperceptible the attack will be.
3.2 Performance Comparison Table 2 shows the performance of all attacking methods on Recformer, P5, TALLRec and CoLLM, respectively. The scatter plot and shilling attack comparison for RecFormer is in Figure 2 and Table 3, while those for other victim models are in the appendix C. From them we can observe that:
• Our text attack paradigm can greatly promote the target items, demonstrating the vulnerability of LLM-based RS. Even the simplest word insertion and rewriting using GPT can increase the exposure of the target item to a certain extent. Furthermore, blackbox text attack methods could lead to manifold increases in the exposure rate of the targeted items.
• Our text attack strategy is also remarkably stealthy, making it difficult for users and platforms to detect. Primarily, the overall performance of RS remains largely unchanged (even we choose 10% items as target items), signifying that the attack does not disrupt the normal operation of RS. Additionally, the generated adversarial titles exhibit high semantic
integrity that they are acceptable (or imperceptible) to human comprehension.
• Traditional shilling attacks are not effective in LLM-based recommendation models. Even with access to a portion of the training data, they fail to significantly enhance the exposure of the targeted items. This is attributed to the fact that LLM-based recommendation models prioritize content information primarily in textual form. Additionally, since fake user-generated training data is introduced during the model training phase, they significantly impact the overall performance of the victim model, which is easily detectable.
• Word-level attacks are effective in boosting the exposure of target items, albeit demanding more queries and higher costs. On the other hand, character-level text attacks demonstrate superior results compared to shilling attacks, even with a reduced number of queries directed towards victim models.
3.3 The influence of fine-tuning In this subsection, we examined the impact of model fine-tuning on attacks. A significant advantage of LLM-based recommendation models is
their zero-shot transferability across datasets; however, we have also uncovered the vulnerability of such zero-shot models.
Figure 3 shows a direct comparison between zero-shot RecFormer and fine-tuned Recformer. The detailed performance of fine-tuned Recformer is shown in Appendix C. From these, we observe that fine-tuned models are more resilient to attacks compared to zero-shot models. This is manifested in three aspects: attacking fine-tuned models requires more queries, yields lesser promoting of target items, and maintains poorer semantic integrity.
3.4 The influence of item popularity In this section, we examined the impact of the initial popularity of target items on attacks. Popularity bias in recommendation systems favors popular items over personalized ones, limiting diversity, fairness and potentially dissatisfying users’ preferences (Zhang et al., 2021b). We selected the top 150 and bottom 150 items in popularity from the dataset as target items and presented the attack results in Table 4. We can observe that items with high popularity experience greater promotion, thereby exacerbating popularity bias. High popularity target items can achieve greater exposure
3.5 Transferability
3.5.1 Transferability across tasks. A significant feature of LLM-based recommendation models, like P5, is their capability to unify various recommendation tasks in a shared instructionbased framework. We evaluate the transferability of adversarial content across direct recommendation task and rating prediction task of P5. The results are presented in Table 5. We can observe there exists strong transferability between different tasks in such unified model. Attacks targeting a single task can boost the exposure of target items across multiple tasks.
3.5.2 Transferability across victim models. Firstly, we evaluate the transferability of the generated adversarial content across RecFormer, TALLRec and CoLLM. We select one model as the
source model and apply the adversarial content generated from attacking it to two other models to verify if it can similarly boost target items. Experimental results of TextFooler are presented in Table 6 and similar trends are observed with other attack methods as well. We observe that transferability only exists among recommendation models utilizing the same backbone LLMs: there’s strong mutual transferability between CoLLM and TALLRec because both models are based on LLaMA (Touvron et al., 2023) as the backbone; whereas, there is no transferability between Recformer (using Longformer (Beltagy et al., 2020) as the backbone) and either of the former two.
3.6 Re-writing Defense In this subsection, we explore potential strategies for addressing the identified vulnerability of LLMbased recommendation models. The most direct strategy is to detect and revise potential adversarial elements in the content, such as spelling errors and potential word substitutions. We utilize GPT-3.5turbo to accomplish the rewriting of adversarial content to achieve defense.
Re-writing Prompt: Correct possible grammar, spelling and word substitution errors in the product title (dirctly output the revised title only): <AdversarialTitle>
The exposure rates and recommendation performance (NDCG@10) before and after defense on RecFormer are shown in Table 7. The results of other victim models are shown in Appendix C. We can observe the defense works well against character-level attacks like DeepwordBug and PuncAttack, but struggles with more complex word substitution attacks such as TextFooler and BertAttack, since character-level spelling errors and punctuation insertions are relatively easy to detect. Moreover, it doesn’t impact overall recommendation performance.
4 Related Work
4.1 LLM-based Recommendation
The techniques used by LLMs in the recommendation domain (Li et al., 2024; Bao et al., 2023b; Lin et al., 2023b) involve translating recommendation tasks into natural language tasks and adapting LLMs to generate recommendation results directly. These generative LLM-based approaches can be further divided into two paradigms based on whether parameters are tuned: non-tuning and tuning paradigms. The non-tuning paradigm assumes LLMs already have the recommendation
abilities and attempt to trigger the strong zero/fewshot abilities by introducing specific prompts (Liu et al., 2023; Dai et al., 2023; Mysore et al., 2023; Wang et al., 2023b; Hou et al., 2023b; Wang et al., 2024a). The tuning paradigm uses fine-tuning, prompt learning, or instruction tuning (Kang et al., 2023; Bao et al., 2023a; Wang et al., 2022; Geng et al., 2022; Cui et al., 2022) to enhance LLM’s recommendation abilities by using LLMs as encoders to extract user and item representations and then fine-tuning their parameters on specific loss functions.
4.2 Shilling Attack
Shilling attacks aim to interfere with the recommendation strategy of a victim recommender system by injecting fake users into the training matrix (Deldjoo et al., 2019; Toyer et al., 2023; Wang et al., 2024b). This can be implemented through (1) heuristic attacks (Burke et al., 2005a; Linden et al., 2003; Kaur and Goel, 2016), where fake profiles are created based on subjective inference and existing knowledge; (2) gradient attacks (Fang et al., 2020; Li et al., 2016; Zhang et al., 2020; Fang et al., 2018; Huang et al., 2021), which optimize the objective function through a continuous space; and neural attacks (Wang and Zhang, 2023; Lin et al., 2020, 2024; Song et al., 2020; Zhang et al., 2022b), which use deep learning to generate realistic profiles.
5 Conclusion
In conclusion, our investigation exposes a critical security issue within LLM-based recommendation systems, brought on by their reliance on textual content. By showcasing the ability of attackers to boost item exposure through subtle text modifications, we stress the urgent need for heightened security measures. Our findings not only highlight the vulnerability of these systems but also serve as a call to action for the development of more robust, attack-resistant models.
Limitations
The main constraints can be summarized in the following two aspects: Firstly, although the blackbox text attack model does not require access to the victim model’s parameters and gradients, it necessitates multiple queries to the model. And it is challenging to query the model in real-world large-scale recommendation systems (Ying et al., 2018). Secondly, this study solely focuses on the content features of text modality. In reality, recommendation systems encompass other modalities such as images and videos (Wei et al., 2019; Zhang et al., 2021a, 2022a, 2023a). The issue of attacking models based on these modalities also represents a worthy direction for research.
Ethics Statement
The experimental datasets are publicly available from some previous works, downloaded via official APIs. The information regarding users in the Amazon dataset has been anonymized, ensuring there are no privacy concerns related to the users. We do not disclose any non-open-source data, and we ensure that our actions comply with ethical standards. We use publicly available pre-trained models, i.e., RecFormer, P5, LLaMA, GPT-Neo. All the checkpoints and datasets are carefully processed by their authors to ensure that there are no ethical problems.
However, it is worth noting that our research has uncovered vulnerabilities in LLM-based RS. Despite proposing potential defense methods in Section 3.6, there still exists a risk of misuse of our attack paradigm. Future research based on this attack should proceed with caution and consider the potential consequences of any proposed methods.
Acknowledgements
This work is supported by National Natural Science Foundation of China (62141608, 62206291,
62236010).
References Keqin Bao, Jizhi Zhang, Yang Zhang, Wenjie Wang,
Fuli Feng, and Xiangnan He. 2023a. Tallrec: An effective and efficient tuning framework to align large language model with recommendation. Proceedings of the 17th ACM Conference on Recommender Systems.
Keqin Bao, Jizhi Zhang, Yang Zhang, Wang Wenjie, Fuli Feng, and Xiangnan He. 2023b. Large language models for recommendation: Progresses and future directions. In Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region, pages 306–309.
Iz Beltagy, Matthew E. Peters, and Arman Cohan. 2020. Longformer: The long-document transformer. arXiv:2004.05150.
Sid Black, Leo Gao, Phil Wang, Connor Leahy, and Stella Biderman. 2021. GPT-Neo: Large Scale Autoregressive Language Modeling with MeshTensorflow. If you use this software, please cite it using these metadata.
Robin Burke, Bamshad Mobasher, and Runa Bhaumik. 2005a. Limited knowledge shilling attacks in collaborative filtering systems. In Proceedings of 3rd international workshop on intelligent techniques for web personalization (ITWP 2005), 19th international joint conference on artificial intelligence (IJCAI 2005), pages 17–24.
Robin Burke, Bamshad Mobasher, Runa Bhaumik, and Chad Williams. 2005b. Segment-based injection attacks against collaborative filtering recommender systems. In ICDM 2005.
Jin Chen, Zheng Liu, Xu Huang, Chenwang Wu, Qi Liu, Gangwei Jiang, Yuanhao Pu, Yuxuan Lei, Xiaolong Chen, Xingmei Wang, Defu Lian, and Enhong Chen. 2023. When large language models meet personalization: Perspectives of challenges and opportunities.
Zeyu Cui, Jianxin Ma, Chang Zhou, Jingren Zhou, and Hongxia Yang. 2022. M6-rec: Generative pretrained language models are open-ended recommender systems.
Sunhao Dai, Ninglu Shao, Haiyuan Zhao, Weijie Yu, Zihua Si, Chen Xu, ZhongXiang Sun, Xiao Zhang, and Jun Xu. 2023. Uncovering chatgpt’s capabilities in recommender systems. Proceedings of the 17th ACM Conference on Recommender Systems.
Yashar Deldjoo, T. D. Noia, and Felice Antonio Merra. 2019. Assessing the impact of a user-item collaborative attack on class of users. ArXiv, abs/1908.07968.
Minghong Fang, Neil Zhenqiang Gong, and Jia Liu. 2020. Influence function based data poisoning attacks to top-n recommender systems. Proceedings of The Web Conference 2020.
Minghong Fang, Guolei Yang, Neil Zhenqiang Gong, and Jia Liu. 2018. Poisoning attacks to graph-based recommender systems. Proceedings of the 34th Annual Computer Security Applications Conference.
Brian Formento, Chuan Sheng Foo, Luu Anh Tuan, and See Kiong Ng. 2023. Using punctuation as an adversarial attack on deep learning-based nlp systems: An empirical study. In Findings of the Association for Computational Linguistics: EACL 2023, pages 1–34.
Ji Gao, Jack Lanchantin, Mary Lou Soffa, and Yanjun Qi. 2018. Black-box generation of adversarial text sequences to evade deep learning classifiers. In 2018 IEEE Security and Privacy Workshops (SPW), pages 50–56. IEEE.
Shijie Geng, Shuchang Liu, Zuohui Fu, Yingqiang Ge, and Yongfeng Zhang. 2022. Recommendation as language processing (rlp): A unified pretrain, personalized prompt & predict paradigm (p5). In RecSys 2022.
Yupeng Hou, Zhankui He, Julian McAuley, and Wayne Xin Zhao. 2023a. Learning vector-quantized item representation for transferable sequential recommenders. In WWW 2023.
Yupeng Hou, Shanlei Mu, Wayne Xin Zhao, Yaliang Li, Bolin Ding, and Ji-Rong Wen. 2022. Towards universal sequence representation learning for recommender systems. In KDD 2022.
Yupeng Hou, Junjie Zhang, Zihan Lin, Hongyu Lu, Ruobing Xie, Julian McAuley, and Wayne Xin Zhao. 2023b. Large language models are zero-shot rankers for recommender systems. ArXiv, abs/2305.08845.
Hai Huang, Jiaming Mu, Neil Zhenqiang Gong, Qi Li, Bin Liu, and Mingwei Xu. 2021. Data poisoning attacks to deep learning based recommender systems. ArXiv, abs/2101.02644.
Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits. 2020. Is bert really robust? a strong baseline for natural language attack on text classification and entailment. In AAAI 2020, pages 8018–8025.
Wang-Cheng Kang, Jianmo Ni, Nikhil Mehta, Maheswaran Sathiamoorthy, Lichan Hong, Ed H. Chi, and Derek Zhiyuan Cheng. 2023. Do llms understand user preferences? evaluating llms on user rating prediction. ArXiv, abs/2305.06474.
Parneet Kaur and Shivani Goel. 2016. Shilling attack models in recommender system. In ICICT 2016, volume 2, pages 1–5.
Bo Li, Yining Wang, Aarti Singh, and Yevgeniy Vorobeychik. 2016. Data poisoning attacks on factorization-based collaborative filtering. ArXiv, abs/1608.08182.
Jiacheng Li, Ming Wang, Jin Li, Jinmiao Fu, Xin Shen, Jingbo Shang, and Julian McAuley. 2023a. Text is all you need: Learning language representations for sequential recommendation. In KDD 2023.
Linyang Li, Ruotian Ma, Qipeng Guo, Xiangyang Xue, and Xipeng Qiu. 2020. Bert-attack: Adversarial attack against bert using bert. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6193–6202.
Xinhang Li, Chong Chen, Xiangyu Zhao, Yong Zhang, and Chunxiao Xing. 2023b. E4srec: An elegant effective efficient extensible solution of large language models for sequential recommendation.
Yongqi Li, Xinyu Lin, Wenjie Wang, Fuli Feng, Liang Pang, Wenjie Li, Liqiang Nie, Xiangnan He, and TatSeng Chua. 2024. A survey of generative search and recommendation in the era of large language models. arXiv preprint arXiv:2404.16924.
Chen Lin, Si Chen, Hui Li, Yanghua Xiao, Lianyun Li, and Qian Yang. 2020. Attacking recommender systems with augmented user profiles. In CIKM 2020, pages 855–864.
Chen Lin, Si Chen, Meifang Zeng, Sheng Zhang, Min Gao, and Hui Li. 2024. Shilling Black-box Recommender Systems by Learning to Generate Fake User Profiles. IEEE Transactions on Neural Networks and Learning Systems, 35(1):1305–1319.
Jianghao Lin, Xinyi Dai, Yunjia Xi, Weiwen Liu, Bo Chen, Xiangyang Li, Chenxu Zhu, Huifeng Guo, Yong Yu, Ruiming Tang, and Weinan Zhang. 2023a. How can recommender systems benefit from large language models: A survey.
Jianghao Lin, Xinyi Dai, Yunjia Xi, Weiwen Liu, Bo Chen, Xiangyang Li, Chenxu Zhu, Huifeng Guo, Yong Yu, Ruiming Tang, et al. 2023b. How can recommender systems benefit from large language models: A survey. arXiv preprint arXiv:2306.05817.
Greg Linden, Brent Smith, and Jeremy York. 2003. Amazon.com recommendations: Item-to-item collaborative filtering. IEEE Internet Comput., 7:76–80.
Junling Liu, Chaoyong Liu, Renjie Lv, Kangdi Zhou, and Yan Bin Zhang. 2023. Is chatgpt a good recommender? a preliminary study. ArXiv, abs/2304.10149.
Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton van den Hengel. 2015. Image-based recommendations on styles and substitutes. In SIGIR 2015, pages 43–52.
John Morris, Eli Lifland, Jin Yong Yoo, Jake Grigsby, Di Jin, and Yanjun Qi. 2020. TextAttack: A framework for adversarial attacks, data augmentation, and adversarial training in NLP. In EMNLP 2020, pages 119–126.
Sheshera Mysore, Andrew McCallum, and Hamed Zamani. 2023. Large language model augmented narrative driven recommendations. Proceedings of the 17th ACM Conference on Recommender Systems.
Junshuai Song, Zhao Li, Zehong Hu, Yucheng Wu, Zhenpeng Li, Jian Li, and Jun Gao. 2020. Poisonrec: An adaptive data poisoning framework for attacking black-box recommender systems. 2020 IEEE 36th International Conference on Data Engineering (ICDE), pages 157–168.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971.
Sam Toyer, Olivia Watkins, Ethan Mendes, Justin Svegliato, Luke Bailey, Tiffany Wang, Isaac Ong, Karim Elmaaroufi, Pieter Abbeel, Trevor Darrell, Alan Ritter, and Stuart Russell. 2023. Tensor trust: Interpretable prompt injection attacks from an online game. ArXiv, abs/2311.01011.
Changsheng Wang, Jianbai Ye, Wenjie Wang, Chongming Gao, Fuli Feng, and Xiangnan He. 2023a. Recad: Towards a unified library for recommender attack and defense. In RecSys 2023.
Wenjie Wang, Honghui Bao, Xinyu Lin, Jizhi Zhang, Yongqi Li, Fuli Feng, See-Kiong Ng, and TatSeng Chua. 2024a. Learnable tokenizer for llmbased generative recommendation. arXiv preprint arXiv:2405.07314.
Wenjie Wang, Changsheng Wang, Fuli Feng, Wentao Shi, Daizong Ding, and Tat-Seng Chua. 2024b. Uplift modeling for target user attacks on recommender systems. In WWW 2024, pages 3343–3354.
Xiaolei Wang, Xinyu Tang, Wayne Xin Zhao, Jingyuan Wang, and Ji rong Wen. 2023b. Rethinking the evaluation for conversational recommendation in the era of large language models. ArXiv, abs/2305.13112.
Xiaolei Wang, Kun Zhou, Ji rong Wen, and Wayne Xin Zhao. 2022. Towards unified conversational recommender systems via knowledge-enhanced prompt learning. KDD.
Yawen Wang and Shihua Zhang. 2023. Prediction of tumor lymph node metastasis using wasserstein distance-based generative adversarial networks combing with neural architecture search for predicting. Mathematics.
Zongwei Wang, Min Gao, Junliang Yu, Hao Ma, Hongzhi Yin, and Shazia Sadiq. 2024c. Poisoning attacks against recommender systems: A survey.
Yinwei Wei, Xiang Wang, Liqiang Nie, Xiangnan He, Richang Hong, and Tat-Seng Chua. 2019. Mmgcn: Multi-modal graph convolution network for personalized recommendation of micro-video. In ACM Multimedia 2019, pages 1437–1445.
Fangzhao Wu, Ying Qiao, Jiun-Hung Chen, Chuhan Wu, Tao Qi, Jianxun Lian, Danyang Liu, Xing Xie, Jianfeng Gao, Winnie Wu, and Ming Zhou. 2020. MIND: A large-scale dataset for news recommendation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3597–3606, Online. Association for Computational Linguistics.
Shenghao Yang, Chenyang Wang, Yankai Liu, Kangping Xu, Weizhi Ma, Yiqun Liu, Min Zhang, Haitao Zeng, Junlan Feng, and Chao Deng. 2023. Collaborative word-based pre-trained item representation for transferable recommendation. In ICDM 2023.
Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L Hamilton, and Jure Leskovec. 2018. Graph convolutional neural networks for web-scale recommender systems. In KDD 2018, pages 974– 983.
Zheng Yuan, Fajie Yuan, Yu Song, Youhua Li, Junchen Fu, Fei Yang, Yunzhu Pan, and Yongxin Ni. 2023. Where to go next for recommender systems? id- vs. modality-based recommender models revisited. In SIGIR 2023.
Hengtong Zhang, Yaliang Li, Bolin Ding, and Jing Gao. 2020. Practical data poisoning attack against nextitem recommendation. Proceedings of The Web Conference 2020.
Jinghao Zhang, Qiang Liu, Shu Wu, and Liang Wang. 2023a. Mining stable preferences: Adaptive modality decorrelation for multimedia recommendation. In SIGIR 2023, pages 443–452.
Jinghao Zhang, Yanqiao Zhu, Qiang Liu, Shu Wu, Shuhui Wang, and Liang Wang. 2021a. Mining latent structures for multimedia recommendation. In ACM Multimedia 2021, pages 3872–3880.
Jinghao Zhang, Yanqiao Zhu, Qiang Liu, Mengqi Zhang, Shu Wu, and Liang Wang. 2022a. Latent structure mining with contrastive modality fusion for multimedia recommendation. IEEE Transactions on Knowledge and Data Engineering.
Junjie Zhang, Ruobing Xie, Yupeng Hou, Wayne Xin Zhao, Leyu Lin, and Ji-Rong Wen. 2023b. Recommendation as instruction following: A large language model empowered recommendation approach.
Xudong Zhang, Zan Wang, Jingke Zhao, and Lanjun Wang. 2022b. Targeted data poisoning attack on news recommendation system by content perturbation. ArXiv, abs/2203.03560.
Yang Zhang, Fuli Feng, Xiangnan He, Tianxin Wei, Chonggang Song, Guohui Ling, and Yongdong Zhang. 2021b. Causal intervention for leveraging popularity bias in recommendation. In SIGIR 2021, pages 11–20.
Yang Zhang, Fuli Feng, Jizhi Zhang, Keqin Bao, Qifan Wang, and Xiangnan He. 2023c. Collm: Integrating collaborative embeddings into large language models for recommendation.
Kaijie Zhu, Jindong Wang, Jiaheng Zhou, Zichen Wang, Hao Chen, Yidong Wang, Linyi Yang, Wei Ye, Neil Zhenqiang Gong, Yue Zhang, and Xing Xie. 2023. PromptBench: Towards Evaluating the Robustness of Large Language Models on Adversarial Prompts.
A Experimental Settings
A.1 Victim Models We choose four mainstream LLM-based recommendation models as our victim models.
• Recformer (Li et al., 2023a). Recformer proposes to formulate an item as a "sentence" (word sequence) and can effectively recommend the next item based on language representations.
• P5 (Geng et al., 2022). P5 presents a flexible and unified text-to-text paradigm called "Pretrain, Personalized Prompt, and Predict Paradigm" (P5) for recommendation, which unifies various recommendation tasks in a shared framework.
• TALLRec (Bao et al., 2023a). TALLRec proposes to align LLMs with recommendation by tunning LLMs with recommendation data.
• CoLLM (Zhang et al., 2023c). CoLLM seamlessly incorporates collaborative information into LLMs for recommendation by mapping ID embedding to the input token embedding space of LLM.
A.2 Compared Shilling Attacks Shilling attacks aim to generate fake users that assign high ratings for a target item, while also rating other items to act like normal users for evading.
• Heuristic attacks. Heuristic attacks involve selecting items to create fake profiles based
on heuristic rules. Random attack (Kaur and Goel, 2016) selects filler items randomly while Bandwagon attack (Burke et al., 2005b) selects the popular items as users fake preferences, which is white-box as it requires knowledge of the popularity of items, i.e., the training data.
• Neural attacks. Neural attacks utilize neural networks to generate fake users that maximize the objective function. Aush (Lin et al., 2020). Aush utilizes Generative Adversarial Network (GAN) to generate fake users based on known knowledge. Leg-UP (Lin et al., 2024). Leg-UP learns user behavior patterns from real users in the sampled “templates” and constructs fake user profiles. Both of them are gray-box attack models, requiring a portion of training data.
A.3 Implementation Details
The statistics of these datasets are summarized in Table 8. All victim models and compared shilling attacks are implemented in PyTorch. We random select 10% items as target items for each dataset. For RecFormer, we use both pre-trained checkpoint1 and also fine-tune it with the three datasets. For P5, we directly use the fine-tuned checkpoints2. For TALLRec3 and CoLLM4, we fine-tune them from scratch. We implement shilling attack methods using RecAD5. Both Aush and Leg-UP are gray-box and we set them to access 20% of the training data.
B Text Attack
B.1 Text Attack Components
A textual attack consists of four main components: Goal Function, Constraint, Transformation, and Search Method. Here is a breakdown example of each component.
B.1.1 Goal Function The Goal Function defines the objective of the attack. It also scores how ”good” the given manipulated text is for achieving the desired goal. The core part could be simplified as:
1https://github.com/AaronHeee/RecFormer 2https://github.com/jeykigung/P5 3https://github.com/SAI990323/TALLRec 4https://github.com/zyang1580/CoLLM 5https://github.com/gusye1234/recad
1 def goal_function(target_item_id , original_text , perturbed_text , threshold =0.5): 2 """ 3 Return the attacked score and determines if the attack is successful. 4 5 :param target_item_id: The target item's id. 6 :param original_text: The original text of target item. 7 :param perturbed_text: The perturbed text of target item. 8 :param threshold: The threshold of success attack. 9 :return: attacked_score , is_successful
10 """ 11 init_score = call_model(original_text , target_item_id) 12 attacked_score = call_model(perturbed_text , target_item_id) 13 is_successful = attacked_score - init_score > threshold 14 return attacked_score , is_successful
B.1.2 Constraint Constraints are conditions that must be met for the perturbed text to be considered valid. These often ensure the perturbed text remains natural and similar to the original text in some aspects (e.g., semantic similarity). Examples are as follows:
1 def maintain_semantic(original_text , perturbed_text , threshold =0.8): 2 """ 3 Checks if the perturbed text maintains semantic similarity. 4 5 :param original_text: Original text. 6 :param perturbed_text: Perturbed version of the text. 7 :param threshold: Threshold for semantic similarity. 8 :return: True if similarity is above the threshold , False otherwise. 9 """
10 similarity = compute_semantic_similarity( input_text , perturbed_text) 11 return similarity > threshold
B.1.3 Transformation The Transformation component refers to the methods applied to modify the original text to achieve the adversarial goal. This could involve synonym replacement, insertion, or deletion of words.
1 def synonym_replacement(original_text): 2 """ 3 Manipulates the original text by replacing synonyms. 4 5 :param original_text: The original text to be manipulated. 6 :return: A list of manipulated texts. 7 """ 8 words = original_text.words 9 transformed_texts = []
10 for i in range(len(words)): 11 replacement_word = get_synonyms(words[i]) 12 modified_text = original_text. replace_word_at_index(i, replacement_word) 13 transformed_texts.append(modified_text) 14 return transformed_texts
B.1.4 Search Method The Search Method dictates the strategy used to explore the space of possible perturbations. For
example, a greedy search method might iteratively apply transformations that maximally increase the attack’s success likelihood.
1 def greedy_search(target_item_id , original_text): 2 """ 3 Applies greedy search to find successful perturbation. 4 5 :param target_item_id: The target item's id. 6 :param original_text: Original text to be perturbed. 7 :return: Best perturbed text. 8 """ 9 best_score = 0
10 best_perturbed_text = original_text 11 perturbed_texts = get_transformations( original_text) 12 for perturbed_text in perturbed_texts: 13 attacked_score , is_successful =
goal_function(target_item_id , original_text , perturbed_text)
14 if satisfy_constraints(original_text , perturbed_text): 15 if attacked_score > best_score: 16 best_score = attacked_score 17 best_perturbed_text = perturbed_text 18 if is_successful: 19 return best_perturbed_text 20 return None
B.2 Implementations
The majority of our text attacks have been developed by revising strategies from TextAttack6 (Morris et al., 2020) and PromptBench7 (Zhu et al., 2023).
All four attack methods, DeepwordBug, PuncAttack, TextFooler and BertAttack share the same Goal Function, where we set the success threshold to 0.05 increasing exposure rate for RecFormer. Since the other three victim models cannot perform full ranking and calculate exposure rates, we set an increase in interaction probability as the objective. Specifically, we set success threshold to 0.3, 0.15, 0.15 increasing interaction probability for P5, TALLRec and CoLLM, respectively. During the attack process, we randomly select 10% of users to calculate the average exposure rate or interaction probability instead of using all users. This approach is lower in cost and more aligned with the constraints of practical attacks.
The recipes of Constraint, Transformation, and Search Method of implemented text attacks are as follows:
1 """ 2 Recipes for DeepwordBug 3 """ 4 5 transformation = CompositeTransformation( 6 [ 7 WordSwapNeighboringCharacterSwap (), 8 WordSwapRandomCharacterSubstitution (), 9 WordSwapRandomCharacterDeletion (),
10 WordSwapRandomCharacterInsertion (),
6https://github.com/QData/TextAttack 7https://github.com/microsoft/promptbench
11 ] 12 ) 13 14 constraints = [ 15 RepeatModification (), 16 StopwordModification (), 17 LevenshteinEditDistance (30) 18 ] 19 search_method = GreedyWordSwapWIR ()
1 """ 2 Recipes for PuncAttack 3 """ 4 punctuations = '\'-' 5 transformation =
WordSwapTokenSpecificPunctuationInsertion( letters_to_insert=punctuations)
6 constraints = [ 7 RepeatModification (), 8 StopwordModification (), 9 WordEmbeddingDistance(min_cos_sim =0.6) ,
10 PartOfSpeech(allow_verb_noun_swap=True), 11 UniversalSentenceEncoder(threshold =0.8) 12 ] 13 search_method = GreedyWordSwapWIR ()
1 """ 2 Recipes for TextFooler 3 """ 4 transformation = WordSwapEmbedding(max_candidates =50) 5 constraints = [ 6 RepeatModification (), 7 StopwordModification (), 8 WordEmbeddingDistance(min_cos_sim =0.6) , 9 PartOfSpeech(allow_verb_noun_swap=True),
10 UniversalSentenceEncoder(threshold =0.84 , metric=" angular") 11 ] 12 search_method = GreedyWordSwapWIR ()
1 """ 2 Recipes for BertAttack 3 """ 4 transformation = WordSwapMaskedLM(max_candidates =48) 5 constraints = [ 6 RepeatModification (), 7 StopwordModification (), 8 MaxWordsPerturbed(max_percent =1), 9 UniversalSentenceEncoder(threshold =0.8)
10 ] 11 search_method = GreedyWordSwapWIR ()
C Detailed Experimental Results
In this section, we present detailed experimental results that could not be shown in the main text due to space limitation.
• The overall attack performances of P5, TALLRec and CoLLM are shown in Table 11, Table 12 and Table 13, respectively. The scatter plots of them are shown in Figure 7.
• The performances of attacking fine-tuned Recformer is shown in Table 14.
• The performance comparison with traditional shilling attacks of TALLRec and CoLLM are shown in 15 and 16.
• We supplemented the experimental results on the MIND-small news recommendation dataset (Wu et al., 2020) with RecFormer as
victim model. The performance is shown in Table 9. It can be observed that in the news scenario, LLM-based recommendation systems are also vulnerable to text attacks. This could potentially boost the dissemination of low-quality, fake news, or news with evident bias.
• We conducted experiments on the impact of different numbers of fake users on the effectiveness of shilling attacks in LLM-based recommendation systems. The results are shown in Table 10. We can observe that as the number of fake users increases, the exposure of the target items also improves. However, compared to the effects of text attacks, this improvement is limited. Additionally, shilling attack shows less stealthiness. As the number of fake users increases, there is a noticeable decline in the overall performance of the recommendation model, which could make it easier for platforms to detect such attacks.
D Case Studies
In this section, we present examples of attacks and defenses against RecFormer as a victim model, as shown in Table 17 - 22.

heading: Future Work
text: While it cannot fully defend against text-based attacks, it can provide some level of defense and contribute to future research. 2 Method
In this section, we first introduce the LLM-based recommendation model and formulate the objectives of the attacks. Then, we present two simple model-agnostic text rewriting approaches. Finally, we provide a detailed introduction of black-box text attacks. 2. 1 Problem Definition
We use the notation I = {i1, · · · , iN} and U = {u1, · · · , uM} to represent the sets of N items and M users, respectively. Each item i ∈ I is associated with textual content ti. Each user u ∈ U has interacted with a number of items Iu, indicating that the preference score yui = 1 for i ∈ Iu. LLM-based RS models user preference and item feature by transforming user historical behavior sequences Iu and target item i into textual prompt Pu,i = [tu, ti, xu, xi], where tu =[ ti1 , · · · , ti|Iu| ]. xu and xi denotes the ID of i and u which are optional in LLM-based RS. We have listed example prompts of four victim models in Table 1. Please refer to Section 3. 1. 1 for the details of them. The recommendation process can be formulated as: ŷu,i = fθ(Pu,i) where fθ denotes the LLM-based model. The goal of the attack task is to promote target items I ′ (increasing the exposure or user interaction probability) through imperceptibly modifying their textual content (we use title in this work). 2. 2 Victim Model-Agnostic Attack
In this subsection, we first introduce two simple, victim model-agnostic strategies employed for altering item textual content to make them more linguistically attractive to users. Our approaches include trivial attack with word insertion and rewriting leveraging Generative Pre-trained Transformers (GPTs). 2. 2. 1 Trivial Attack with Word Insertion
The core premise of this strategy is founded on the assumption that positive or exclamatory words can attract users. By infusing item titles with a select number of positive words, we aim to increase the items’ attractiveness and, consequently, their likelihood of being recommended by the system. Specifically, we randomly select k words form a pre-defined word corpus which is common-used in item titles. These selected words are then inserted to the end of the original text content to retain the overall coherence. Positive word corpus: [‘good’, ‘great’, ‘best’, ‘nice’, ‘excellent’, ‘amazing’, ‘awesome’, ‘fantastic’, ‘wonderful’, ‘perfect’, ‘ultimate’, ‘love’, ‘like’, ‘beautiful’, ‘well’, ‘better’, ‘easy’, ‘happy’, ‘recommend’, ‘works’, ‘fine’, ‘fast’, ‘fun’, ‘price’, ‘quality’, ‘product’, ‘value’, ‘bought’, ‘purchase’, ‘top’, ‘popular’, ‘choice’, ‘!!!’ ]
2. 2. 2 Re-writing with GPTs
While the insertion of positive words offers a straightforward means of enhancing content appeal, it can sometimes result in awkward or forced phrasings that diminish the content’s natural flow and potentially arouse user suspicion. To address these shortcomings, we propose to use GPTs to rewrite the content of items in a more attractive way by leveraging its rich common sense knowledge and powerful generation capabilities. Specifically, we instruct GPT-3. 5-turbo with the following prompts to generate attractive and fluency titles. Prompt 1: You are a marketing expert that helps to promote the product selling. Rewrite the product title in <MaxLen> words to keep its body the same but more attractive to customers: <ItemTitle>. Prompt 2: Here is a basic title of a product. Use your creativity to transform it into a catchy and unique title in <MaxLen> words that could attract more attention: <ItemTitle>. Prompt 3: Rewrite this product’s title by integrating positive and appealing words, making it more attractive to potential users without altering its original meaning (in <MaxLen> words): <ItemTitle>. 2. 3 Exploring Vulnerabilities in LLM-Based Recommendation Models through Black-Box Text Attacks
In this subsection, we present an examination of traditional black-box text attack methods to explore the vulnerabilities within LLM-based recommendation models. Black-box text attack methods typically involve manipulating or perturbing text inputs to deceive or mislead a natural language processing (NLP) model while having no access to the model’s internal parameters or gradients. The goal of such attacks is mathematically formulated as:
argmax t′i
Eu∈U ′ fθ(P ′u,i), (1)
where P ′u,i = [tu, t′i, xu, xi] denotes the prompts consisting of the user text tu and the manipulated title of the target item t′i. Following the framework proposed by Morris et al. (2020), text attacks are comprised of four principal components:
• Goal Function: This function evaluates the effectiveness of the perturbed input x′ in achieving a specified objective, serving as a heuristic for the search method to identify the optimal solution. In this study, the aim is to promote the target items as in Equation 1. • Constraints: These are conditions that ensure the perturbations remain valid alterations of the original input, emphasizing aspects such as semantic retention and maintaining consistency in part-of-speech tags. • Transformation: A process that applies to an input to generate possible perturbations, which could involve strategies like swapping
words with similar ones based on word embeddings, using synonyms from a thesaurus, or substituting characters with homoglyphs. • Search Method: This method involves iteratively querying the model to select promising perturbations generated through transformations, employing techniques such as a greedy approach with word importance ranking, beam search, or a genetic algorithm. While the specific components of text attack methodologies may vary, the overarching framework remains consistent, as depicted in Algorithm 1. In this work, we have implemented four widelyused attacks: DeepwordBug (Gao et al. , 2018), TextFooler (Jin et al. , 2020), BertAttack (Li et al. , 2020), and PuncAttack (Formento et al. , 2023). DeepwordBug and PuncAttack are character-level which manipulate texts by introducing typos and inserting punctuation. TextFooler and BertAttack are word-level that aim to replace words with synonyms or contextually similar words. Please refer to the appendix B for the details of text attack paradigm and these four methods. Algorithm 1 Text Attack Framework Require: Original text x, Target model M , Goal
function G, Constraints C, Transformations T , Search Method S Ensure: Adversarial text x′, Adversarial score G. score(x′)
1: Initialize x′ as a copy of x. 2: while not S. StoppingCriteria() do 3: Select a transformation t from allowable transformations T. 4: Generate x′ by applying t to x. 5: if C. Satisfied(x′) then 6: if S. AchieveGoal(x′) then 7: return x′, G. score(x′). 8: end if 9: end if
10: end while
3 Experiments
3. 1 Experimental Settings 3. 1. 1 Victim Models We choose four mainstream LLM-based recommendation models as our victim models: Recformer (Li et al. , 2023a), P5 (Geng et al. , 2022), TALLRec (Bao et al. , 2023a) and CoLLM (Zhang
et al. , 2023c). Please refer to Appendix A. 1 for more details. 3. 1. 2 Compared Shilling Attacks Shilling attacks aim to generate fake users that assign high ratings for a target item, while also rating other items to act like normal users for evading. We compare our text attack paradigm with white-box shilling attacks Random attack (Kaur and Goel, 2016), Bandwagon attack (Burke et al. , 2005b), and gray-box Aush (Lin et al. , 2020) and Leg-UP (Lin et al. , 2024). Please refer to Appendix A. 2 for more details. 3. 1. 3 Datasets We conduct experiments on three categories of widely-used (Li et al. , 2023a; Geng et al. , 2022; Bao et al. , 2023a; Zhang et al. , 2023c) Amazon review dataset introduced by McAuley et al. (2015): ‘Beauty’, ‘Toys and Games’, ’Sports and Outdoors’, which are named as Beauty, Toys and Sports in brief. We use the 5-core version of Amazon datasets where each user and item have 5 interactions at least. The statistics of these datasets are summarized in Appendix A. 3. 3. 1. 4 Implementation Details All victim models and compared shilling attacks are implemented in PyTorch. We random select 10% items as target items. For more implementation details, please refer to Appendix A. 3. 3. 1. 5 Evaluation metrics We evaluate the attack from two aspects: effectiveness and stealthiness. Effectiveness. This metric gauges the extent to which our methodology can promote the specified target items. • Exposure. For the victim model RecFormer, which allows for full ranking, we employed a direct metric, exposure rate. We define the exposure rate as expi = N irec Nu
, where Nu represents the total number of users, and N irec denotes the count of users for whom target item i appears in their top-K (K = 50 by default) recommendation list. • Purchasing propensity. For other three victim models which could not conduct full ranking, we define the purchasing propensity of item i as pi = Eu∈U ŷui, where ŷui denotes the predicted probability that user u tends to interact with item i. • # queries. In a black-box scenario, the adversary need query the victim model in order to detect any alterations in the output logits. The lower the value, the more effective the attack will be. Stealthiness. The stealthy attack aims to promote target items while maintaining imperceptibility, thereby avoiding detection by users and platforms. Therefore, the coherence and authenticity of our generated content are crucial to uphold. • Overall recommendation performance. An ideal stealthy attack should keep the overall recommendation performance unchanged. The recommendation performance includes Recall@K, NDCG@K and AUC. • Text quality. The generated adversarial content should be of high quality that are acceptable to users. Firstly, it should be consistent with the corresponding item. We measure the cosine semantic similarity and ROUGE scores between the original content and adversarial content for this purpose. Secondly, the adversarial content itself should be readable. We assess the fluency of the adversarial title, measured by the perplexity of GPT-Neo (Black et al. , 2021). • # perturbed words. The number of words
changed on an average to generate an adversarial content. The lower the value, the more imperceptible the attack will be. 3. 2 Performance Comparison Table 2 shows the performance of all attacking methods on Recformer, P5, TALLRec and CoLLM, respectively. The scatter plot and shilling attack comparison for RecFormer is in Figure 2 and Table 3, while those for other victim models are in the appendix C. From them we can observe that:
• Our text attack paradigm can greatly promote the target items, demonstrating the vulnerability of LLM-based RS. Even the simplest word insertion and rewriting using GPT can increase the exposure of the target item to a certain extent. Furthermore, blackbox text attack methods could lead to manifold increases in the exposure rate of the targeted items. • Our text attack strategy is also remarkably stealthy, making it difficult for users and platforms to detect. Primarily, the overall performance of RS remains largely unchanged (even we choose 10% items as target items), signifying that the attack does not disrupt the normal operation of RS. Additionally, the generated adversarial titles exhibit high semantic
integrity that they are acceptable (or imperceptible) to human comprehension. • Traditional shilling attacks are not effective in LLM-based recommendation models. Even with access to a portion of the training data, they fail to significantly enhance the exposure of the targeted items. This is attributed to the fact that LLM-based recommendation models prioritize content information primarily in textual form. Additionally, since fake user-generated training data is introduced during the model training phase, they significantly impact the overall performance of the victim model, which is easily detectable. • Word-level attacks are effective in boosting the exposure of target items, albeit demanding more queries and higher costs. On the other hand, character-level text attacks demonstrate superior results compared to shilling attacks, even with a reduced number of queries directed towards victim models. 3. 3 The influence of fine-tuning In this subsection, we examined the impact of model fine-tuning on attacks. A significant advantage of LLM-based recommendation models is
their zero-shot transferability across datasets; however, we have also uncovered the vulnerability of such zero-shot models. Figure 3 shows a direct comparison between zero-shot RecFormer and fine-tuned Recformer. The detailed performance of fine-tuned Recformer is shown in Appendix C. From these, we observe that fine-tuned models are more resilient to attacks compared to zero-shot models. This is manifested in three aspects: attacking fine-tuned models requires more queries, yields lesser promoting of target items, and maintains poorer semantic integrity. 3. 4 The influence of item popularity In this section, we examined the impact of the initial popularity of target items on attacks. Popularity bias in recommendation systems favors popular items over personalized ones, limiting diversity, fairness and potentially dissatisfying users’ preferences (Zhang et al. , 2021b). We selected the top 150 and bottom 150 items in popularity from the dataset as target items and presented the attack results in Table 4. We can observe that items with high popularity experience greater promotion, thereby exacerbating popularity bias. High popularity target items can achieve greater exposure
3. 5 Transferability
3. 5. 1 Transferability across tasks. A significant feature of LLM-based recommendation models, like P5, is their capability to unify various recommendation tasks in a shared instructionbased framework. We evaluate the transferability of adversarial content across direct recommendation task and rating prediction task of P5. The results are presented in Table 5. We can observe there exists strong transferability between different tasks in such unified model. Attacks targeting a single task can boost the exposure of target items across multiple tasks. 3. 5. 2 Transferability across victim models. Firstly, we evaluate the transferability of the generated adversarial content across RecFormer, TALLRec and CoLLM. We select one model as the
source model and apply the adversarial content generated from attacking it to two other models to verify if it can similarly boost target items. Experimental results of TextFooler are presented in Table 6 and similar trends are observed with other attack methods as well. We observe that transferability only exists among recommendation models utilizing the same backbone LLMs: there’s strong mutual transferability between CoLLM and TALLRec because both models are based on LLaMA (Touvron et al. , 2023) as the backbone; whereas, there is no transferability between Recformer (using Longformer (Beltagy et al. , 2020) as the backbone) and either of the former two. 3. 6 Re-writing Defense In this subsection, we explore potential strategies for addressing the identified vulnerability of LLMbased recommendation models. The most direct strategy is to detect and revise potential adversarial elements in the content, such as spelling errors and potential word substitutions. We utilize GPT-3. 5turbo to accomplish the rewriting of adversarial content to achieve defense. Re-writing Prompt: Correct possible grammar, spelling and word substitution errors in the product title (dirctly output the revised title only): <AdversarialTitle>
The exposure rates and recommendation performance (NDCG@10) before and after defense on RecFormer are shown in Table 7. The results of other victim models are shown in Appendix C. We can observe the defense works well against character-level attacks like DeepwordBug and PuncAttack, but struggles with more complex word substitution attacks such as TextFooler and BertAttack, since character-level spelling errors and punctuation insertions are relatively easy to detect. Moreover, it doesn’t impact overall recommendation performance. 4 Related Work
4. 1 LLM-based Recommendation
The techniques used by LLMs in the recommendation domain (Li et al. , 2024; Bao et al. , 2023b; Lin et al. , 2023b) involve translating recommendation tasks into natural language tasks and adapting LLMs to generate recommendation results directly. These generative LLM-based approaches can be further divided into two paradigms based on whether parameters are tuned: non-tuning and tuning paradigms. The non-tuning paradigm assumes LLMs already have the recommendation
abilities and attempt to trigger the strong zero/fewshot abilities by introducing specific prompts (Liu et al. , 2023; Dai et al. , 2023; Mysore et al. , 2023; Wang et al. , 2023b; Hou et al. , 2023b; Wang et al. , 2024a). The tuning paradigm uses fine-tuning, prompt learning, or instruction tuning (Kang et al. , 2023; Bao et al. , 2023a; Wang et al. , 2022; Geng et al. , 2022; Cui et al. , 2022) to enhance LLM’s recommendation abilities by using LLMs as encoders to extract user and item representations and then fine-tuning their parameters on specific loss functions. 4. 2 Shilling Attack
Shilling attacks aim to interfere with the recommendation strategy of a victim recommender system by injecting fake users into the training matrix (Deldjoo et al. , 2019; Toyer et al. , 2023; Wang et al. , 2024b). This can be implemented through (1) heuristic attacks (Burke et al. , 2005a; Linden et al. , 2003; Kaur and Goel, 2016), where fake profiles are created based on subjective inference and existing knowledge; (2) gradient attacks (Fang et al. , 2020; Li et al. , 2016; Zhang et al. , 2020; Fang et al. , 2018; Huang et al. , 2021), which optimize the objective function through a continuous space; and neural attacks (Wang and Zhang, 2023; Lin et al. , 2020, 2024; Song et al. , 2020; Zhang et al. , 2022b), which use deep learning to generate realistic profiles