Tackling the Myriads of Collusion Scams on Y ou T ube Comments of Cryptocurrency Videos (2023)

Abstract: Despite repeated measures, YouTube’s comment section has been a fertile ground for scammers. With the growth of the cryptocurrency market and obscurity around it, a new form of scam, namely “Collusion Scam” has emerged as a dominant force within YouTube’s comment space. Unlike typical scams and spams, collusion scams employ a cunning persuasion strategy, using the facade of genuine social interactions within comment threads to create an aura of trust and success to entrap innocent users. In this research, we collect 1,174 such collusion scam threads and perform a detailed analysis, which is tailored towards the successful detection of these scams. We find that utilization of the collusion dynamics can provide an accuracy of 96.67% and an F1-score of 93.04%. Furthermore, we demonstrate the robust predictive power of metadata associated with these threads and user channels, which act as compelling indicators of collusion scams. Finally, we show that modern LLM, like chatGPT, can effectively detect collusion scams without the need for any training.

## introduction the most popular online video-sharing platform youtube has seen a surge of scams and spam comments since its creation in 2005. although measures have been taken, financial frauds, especially related to cryptocurrency investment have not been slowed down @xcite . scammers have adapted their tactics to circumvent the scam-detection algorithm by adopting the disguise of genuine users, engaging in seemingly ordinary conversations, and perpetrating a previously undocumented form of deceit known as the "collusion scam". due to their facades, such scams frequently go unnoticed by automated detection systems, posing a significant threat to users who may unwittingly fall victim to such schemes. consequently, it has become imperative to employ rigorous linguistic, psycholinguistic, and metadata analyses to effectively detect and combat these collusion scams. the "collusion scam" can be defined as a fake conversation where the participants pretend to be beneficiaries of a person or an entity to entrap users for their monetary gain. typically, a scammer or a group of scammers will share their success and gratitude in working with a person or entity. often another group joins the conversation by pretending to be curious or newbies, and on later turns they also express to be a beneficiary. in this method, the scammers share the entity's handles or contact information to get around youtube's rules. figure 1 shows an example of the collusion scam where some scammers engage in a conversation by pretending to be a beneficiary of a cryptocurrency investment through a claim expert. the rise of cryptocurrencies has not only attracted genuine enthusiasts and investors but has also unfortunately attracted a surge in fraudulent activities. the absence of comprehensive regulations, limited awareness among users, and the inherent obscurity of cryptocurrency transactions have created fertile ground for scammers to exploit unsuspecting individuals. one prominent avenue for scams in the cryptocurrency space is youtube, where misleading and collusive comments on cryptocurrency videos can deceive and manipulate unsuspecting viewers. youtube's own machine learning algorithms deleted over 950 million comments in q4, 2021 (9to, accessed: 2023-1-21), however, the measures were not adequate because of the evolving nature of these scams. due to youtube's policy on spam comments, it often deletes comments that strictly violate the policy. for example, the comments that trick others into leaving the site for another one, offer monetary incentives, repetitive, links to coun-terfeits, etc (you, accessed: 2023-1-21). however, collusion scam is a fairly new approach of scamming where multiple scamming strategies are used to deceive the user, and current spam filters are not able to detect these contents. hence, there is an urgent need to address the pervasive issue of collusion scams to establish trust, combat the distortion in information exchange, and ensure a safer online environment for the cryptocurrency community and beyond. in this research, we collect 7,335 conversation threads (comment-replies) from 112 cryptocurrency-related youtube videos. we manually label them for the presence or absence of collusive scams. next, we delve into a comprehensive analysis of the linguistic patterns, as well as an exploration of the persuasive strategies employed within these conversations. we also analyze the collusion dynamics within a conversation by using a bert-lstm architecture. furthermore, we explore how the collusion scam detection performance improves with the progression of the thread, and find that we can obtain 96.67% accuracy and 93.04% f1-score when utilizing the initial comment, and all subsequent replies in a conversation. additionally, we explore how different metadata, like the timespan between the comments and replies, the number of like counts, age of the users' channels can provide strong cues for collusion scams. finally, we examine the performance of chatgpt in the realm of collusion scam detection. the main contributions of our research can be summarized as follows: the data is publicly available at https://github.com/sadat1971/youtube collusion scam . ## related works researchers explored several aspects of youtube comments, such as, analyzing the user interactions, sentiment analysis, hate speech, and bias and misinformation @xcite @xcite . a number of studies worked on spam detection in youtube videos. there are some research initiatives around scams on youtube. @xcite performed a comparative analysis of machine learning algorithms to detect monetary scam videos. bouma-sims and reaves (2021) explored the metadata aspect of scam videos on youtube. they found that scammers' accounts have lesser activity and scam videos have less longevity than non-scam videos. however, these works do not address cryptocurrency-related scam comments or collusion scams. notably, researchers have explored bitcoin-related scam comments and relevant keywords on platforms like bitcointalk (atondo @xcite . other studies have also investigated cryptocurrency scams, albeit with a primary focus on ponzi schemes and pump-and-dump schemes, which differ from the intricacies of collusion scams. @xcite @xcite . ponzi schemes involve promising high returns to investors using funds from new participants, while pump-and-dump schemes manipulate asset prices through coordinated buying and selling. in contrast, collusion scams employ social and psychological strategies, such as mimicking regular conversations and leveraging social proof, to deceive users. hence, the existing research lacks in effectively detecting and addressing the nuances of collusion scams. ## methodology our work involves a meticulous data collection process, labeling, and employing machine learning techniques to detect collusion scam. ## chatgpt and collusion scam among the family of large language models (llm), chatgpt has shown enormous promise, due to its language generation and comprehension abilities (chatgpt). first, we use the chatgpt prompt to provide the following instruction: the collusion scam can be defined as a fake conversation where the participants pretend to be beneficiaries of a person or an entity to entrap the users for their monetary gain. i will provide some examples, can you tell me if they are creating a collusion scam or not? subsequently, we provide it with a set of threads involving both collusion scams and non-scams. these prompts are presented within a single chat session. we manually extract the output from the response. we find that chatgpt as collusion scam detector yields an accuracy of 89.40%, with an f1-score of 88.54%. in 8.53% of the cases, it does not provide any direct answer, and we use the prompt to ask further questions to have a clear response. we also find that after an average of 8.33 responses, chat-gpt seems to forget the task, and we provide the task description again. since chatgpt provides a linguistic response, it first summarizes the conversation, and then the verdict with its reasoning. although its performance falls short of our bert-lstm model, its explanations accompanying the responses can enhance collusion scam detection reliability for users. however, given the large number of collusion scams on youtube and the lack of a fine-tunable architecture, further research is necessary to incorporate chatgpt into collusion scam detection. nevertheless, it is crucial to acknowledge that while chatgpt demonstrates responsible behavior by refraining from offering harmful or improper responses, it remains susceptible to manipulation by scammers @xcite . for example, when prompted with instructions for writing a comment in a youtube video about being financial beneficiaries of a person, chatgpt answers with a legitimate-sounded response with a specific amount of "profit" and "investment". thus, collusion scam detection in the post-ai era may require more careful work and sophistication with a responsible ai research. ## conclusion and future work in this research, we address the issue of collusion scams within youtube's comment section, particularly in the cryptocurrency market. we have demonstrated scammers' deceptive tactics, luring unsuspecting users through social interactions. we also explore different collusion scam detection strategies, where the comments may have an important initial signal, and the thread dynamics can further bolster the detection performance. additionally, our study of youtube metadata shows promising discriminators between collusion scams and genuine discussions, including likes, reply patterns, and user channel age.