Profiling LLM ’s Copyright Infringement Risks under Adversarial Persuasive Prompting (2025)

Abstract: Large Language Models (LLMs) have demonstrated impressive capabilities in text generation but raise concerns regarding potential copyright infringement. While prior research has explored mitigation strategies like content filtering and alignment, the impact of adversarial persuasion techniques in eliciting copyrighted content remains underexplored. This paper investigates how structured persuasion strategies, including logical appeals, emotional framing, and compliance techniques, can be used to manipulate LLM outputs and potentially increase copyright risks. We introduce a structured persuasion workflow, incorporating query mutation, intention-preserving filtering, and few-shot prompting, to systematically analyze the influence of persuasive prompts on LLM responses. Through experiments on state-of-the-art LLMs, including GPT-4o-mini and Claude-3-haiku, we quantify the effectiveness of different persuasion techniques and assess their implications for AI safety. Our results highlight the vulnerabilities of LLMs to adversarial persuasion and provide empirical evidence of the increased risk of generating copyrighted content under such influence. We conclude with recommendations for strengthening model safeguards and future directions for enhancing LLM robustness against manipulation. Code is available at https://github.com/Rongite/Persuasion.

## introduction large language models (llms) have demonstrated remarkable capabilities in generating human-like text @xcite , raising concerns about their potential to inadvertently produce copyrighted content @xcite @xcite . while extensive research has focused on mitigating direct copyright violations through content filtering and * corresponding authors: {zxu79,dzhang42}@stevens.edu figure 1 : an example of a copyright concerned query, a persuasive query (pq) adapted using a predefined persuasion template and foot-in-the-door technique while preserving the original intention, and a resulting inference that successfully avoids copyright infringement. persuasion techniques, such as logical appeals, emotional framing, and compliance-based strategies, have been widely studied in human communication. recent research suggests that llms, trained on vast textual corpora, exhibit sensitivity to such strategies, making them susceptible to structured adversarial prompts. by systematically applying persuasive mutations to queries, we explore whether llms can be influenced to generate text that closely resembles copyrighted material. to this end, we propose a structured persuasion workflow to evaluate the role of persuasion in prompting llms to generate copyrighted content. our approach consists of three key components: through extensive experiments on state-of-theart llms, including gpt-4o and claude-3, we systematically evaluate the effectiveness of different persuasive strategies in eliciting unauthorized text generation. our results provide empirical insights into the vulnerabilities of llms to adversarial persuasion and highlight potential risks associated with persuasive query mutations. furthermore, we discuss the implications of our findings for ai safety, responsible llm deployment, and the development of more robust safeguards against adversarial attacks. overall, this work contributes to the growing discourse on llm security and copyright risk assessment by demonstrating how structured persuasion techniques can influence model behavior. we conclude with recommendations for mitigating the risks associated with persuasive adversarial prompts and outline future research to enhance the robustness of llms against manipulation. ## related works copyright concerns of llms. large language models (llms) raise significant copyright concerns @xcite , particularly regarding the use of copyrighted materials during training and the potential for verbatim memorization in their outputs @xcite @xcite . legal cases, such as the new york times v. openai (2023), highlight the urgency of these issues, with research showing that while refusal training can mitigate memorization risks, larger models remain more likely to retain and reproduce copyrighted content @xcite . further investigation by @xcite raises concerns about llms' ability to respect embedded copyright information in user inputs. while llms typically refuse explicit requests for copyrighted content, they often overlook embedded copyright notices, heightening the risk of violations. this highlights the urgent need for better alignment and safeguards to address copyright concerns in ai systems. for a broader and up-to-date overview of copyright risks and mitigation approaches in llms, we refer readers to a recent tutorial that systematizes this area @xcite . persuasive communication with llms. we also situate our work within the broader landscape of llm safety research @xcite . recent studies have examined llms in persuasive communication @xcite . @xcite surveys persuasive techniques in llms, such as content customization (tailoring messages to personality traits) @xcite , emotional framing to influence perception @xcite , and logical reasoning for persuasive effectiveness @xcite , while addressing ethical concerns like privacy violations arising from personalization techniques that leverage sensitive user data @xcite . @xcite introduce a taxonomy of 40 persuasion techniques, including negotiation and threats, grouped into 15 strategies, and demonstrate how persuasionbased adversarial prompts can bypass llm safety mechanisms. @xcite apply the foot-inthe-door (fitd) technique @xcite to exploit cognitive consistency in llms, starting with small requests to increase compliance for jailbreaking. ## persuasion workflow to profile llm's copyright infringement risk this section explores the potential for copyright infringement in content generated by llms when influenced by persuasive queries. we first define the problem and its implications. then, we introduce a structured workflow illustrating how persuasive prompt mutations, intention preservation through an llm-based judge, and inference scaling affect the likelihood of copyright infringement in llm-generated content. ## experiment in this section, we would like to profile the copyright risk of llms raised by the persuasion technique by evaluating the implementation of the persuasion workflow. in particular, we would like to answer the following research questions: preparing copyrighted novels. we compile a copyrighted material dataset comprising three novels: harry potter and the sorcerer's stone, the hobbit, and a game of thrones. the content of the dataset includes the first 100 words of the main text of these three books. see ethical considerations for legal rationale, data-handling safeguards, and release policy. ## conclusion this study systematically examined the potential for llms to generate copyrighted content under adversarial persuasion techniques. by introducing a structured persuasion workflow, we evaluated how persuasive query mutations, intentionpreserving mechanisms, and inference scaling affect the likelihood of copyright infringement. our experimental results demonstrate that certain persuasion strategies-such as the foot-in-the-door technique-significantly increase the risk of unauthorized content generation. moreover, incorporating few-shot prompting further amplifies these risks, highlighting the need for enhanced safeguards in llm deployment. the findings underscore the necessity for developing more robust ai alignment strategies to mitigate adversarial manipulation. future research should explore adaptive defenses, such as real-time intent verification, dynamic refusal mechanisms, and reinforcement learning-based adversarial training. additionally, interdisciplinary collaboration between ai researchers, policymakers, and legal experts is crucial for addressing emerging ethical and legal challenges in llm deployment. as llms continue to evolve, ensuring their responsible use while minimizing risks associated with adversarial persuasion remains a pressing concern.