Stealthy Attack on Large Language Model based Recommendation (2024)

Abstract: Recently, the powerful large language models (LLMs) have been instrumental in propelling the progress of recommender systems (RS). However, while these systems have flourished, their susceptibility to security threats has been largely overlooked. In this work, we reveal that the introduction of LLMs into recommendation models presents new security vulnerabilities due to their emphasis on the textual content of items. We demonstrate that attackers can significantly boost an item’s exposure by merely altering its textual content during the testing phase, without requiring direct interference with the model’s training process. Additionally, the attack is notably stealthy, as it does not affect the overall recommendation performance and the modifications to the text are subtle, making it difficult for users and platforms to detect. Our comprehensive experiments across four mainstream LLM-based recommendation models demonstrate the superior efficacy and stealthiness of our approach. Our work unveils a significant security gap in LLM-based recommendation systems and paves the way for future research on protecting these systems.

## introduction over the past few decades, recommender systems (rs) have gained considerable significance across various domains. recently, the powerful large language models (llms) have been instrumental in propelling the progress of recommender systems. there has been a notable upswing of interest dedicated to developing llms tailored for recommendation task. contrary to traditional recommendation models, which rely heavily on abstract and less interpretable id-based information, llm-based recommendation models exploit the semantic understanding and strong transferability of llms. this approach places a heightened focus on the textual content of items, such as titles and descriptions @xcite . for instance, many researchers @xcite @xcite @xcite @xcite @xcite @xcite ) have explored modeling user preferences and item characteristics through a linguistic lens. this methodology promises a revolutionary shift in the conventional paradigm of recommendations by providing generalization capabilities to novel items and datasets. despite these advancements, the security of rs remains a largely unaddressed issue. malicious attacks on these systems can lead to undesirable outcomes, such as the unwarranted promotion of low-quality products in e-commerce platforms or the spread of misinformation in news dissemination contexts. traditional shilling attack strategies on rs @xcite involve the generation of fake users who are programmed to give high ratings to specific target items. by introducing such cheating data, it aims at influencing the training of the recommender models and subsequently increasing the exposure of the target items. however, the introduction of llms into recommendation models presents new security vulnerabilities. in this paper, to the best of our knowledge, we are the first to demonstrate that llm-based recommendation systems are more vulnerable due to their emphasis on the textual content of items. we demonstrate that attackers can significantly boost an item's exposure by merely altering its textual content during the testing phase, utilizing simple heuristic re-writing or black-box text attack strategies @xcite . compared with traditional shilling attacks, this attack paradigm is notably stealthy, as it does not require influencing the training of the model, and the overall recommendation performance is almost unchanged. moreover, the modifications to the title are subtle, making it difficult for users and platforms to detect. we construct comprehensive experiments on four mainstream llm-based recommendation models @xcite @xcite as victim models to validate the outstanding efficacy and stealthiness of the textual attack paradigm compared with traditional shilling attacks @xcite @xcite . we further delve into the effects of model fine-tuning and item popularity on the attack. additionally, we investigate the transferability of the attack across various victim models and recommendation tasks to demonstrate its practical applicability and utility in real-world scenarios. finally, we evaluate a simple re-writing defense strategy, which also can mitigate the issue to some extent. to summarize our contributions: ## method in this section, we first introduce the llm-based recommendation model and formulate the objectives of the attacks. then, we present two simple model-agnostic text rewriting approaches. finally, we provide a detailed introduction of black-box text attacks. ## conclusion in conclusion, our investigation exposes a critical security issue within llm-based recommendation systems, brought on by their reliance on textual content. by showcasing the ability of attackers to boost item exposure through subtle text modifications, we stress the urgent need for heightened security measures. our findings not only highlight the vulnerability of these systems but also serve as a call to action for the development of more robust, attack-resistant models.