P sy S cam: A Benchmark for Psychological Techniques in Real-World Scams (2025)

Abstract: Over the years, online scams have grown dramatically,with nearly 50% of global consumersencountering scam attempts each week.These scams cause not only significant financiallosses to individuals and businesses, butalso lasting psychological trauma, largely dueto scammers’ strategic employment of psychologicaltechniques (PTs) to manipulate victims.Meanwhile, scammers continually evolve theirtactics by leveraging advances in Large LanguageModels (LLMs) to generate diverse scamvariants that easily bypass existing defenses.To address this pressing problem, we introducePsyScam, a benchmark designed to systematicallycapture the PTs employed in real-worldscam reports, and investigate how LLMs canbe utilized to generate variants of scams basedon the PTs and the contexts provided by thesescams. Specifically, we collect a wide range ofscam reports and ground its annotations of employedPTs in well-established cognitive andpsychological theories. We further demonstrateLLMs’ capabilities in generating through twodownstream tasks: scam completion, and scamaugmentation. Experimental results show thatPsyScam presents significant challenges toexisting models in both detecting and generatingscam content based on the PTs used byreal-world scammers. Our code and dataset areavailable.

## introduction online scams have become a global epidemic, causing severe financial and psychological harm to individuals and organizations. according to the u.s. federal trade commission (ftc), consumers reported over 5.7 billion in losses to fraud in 2024 alone, marking a 125% increase from the previous year @xcite . similarly, singapore's government recorded 929.6 million in scam-related losses in 2024, reflecting a 41% year-over-year rise (spf, therefore, our company is offering you a remote online job, which is a great remote part-time/full-time online job. we will provide you free training and necessary equipment. work for 60 to 90 minutes (depending on your schedule) and earn @xmath0500 per day ## task design to demonstrate the utility of our benchmark, we design three representative downstream tasks, which are defined as follows: ## conclusion in this work, we introduce psyscam, the first benchmark designed to systematically capture and evaluate pts in real-world scam incidents. by collecting diverse scam reports from six public reporting platforms and grounding our annotations in established cognitive and persuasion theories, psyscam bridges the gap between psychology and practical cyber security analysis. our human-llm collaborative annotation framework enables scalable, high-quality pt labeling and our evaluation on three downstream tasks shows that psyscam poses challenges to existing models. we believe psyscam lays the foundation for future research on scam detection and generation, persuasive language understanding, and the development of trustworthy ai systems for combating online scams and fraud.