What Does the Bot Say? Opportunities and Risks of Large Language Models in Social Media Bot Detection (2024)

Abstract: Social media bot detection has always been an arms race between advancements in machine learning bot detectors and adversarial bot strategies to evade detection. In this work, we bring the arms race to the next level by investigating the opportunities and risks of state-of-the-art large language models (LLMs) in social bot detection. To investigate the opportunities, we design novel LLM-based bot detectors by proposing a mixture-of-heterogeneous-experts framework to divide and conquer diverse user information modalities. To illuminate the risks, we explore the possibility of LLM-guided manipulation of user textual and structured information to evade detection. Extensive experiments with three LLMs on two datasets demonstrate that instruction tuning on merely 1,000 annotated examples produces specialized LLMs that outperform state-of-the-art baselines by up to 9.1% on both datasets, while LLM-guided manipulation strategies could significantly bring down the performance of existing bot detectors by up to 29.6% and harm the calibration and reliability of bot detection systems.

## introduction social media bot accounts are behind many online perils such as misinformation @xcite , election interference @xcite @xcite , extremist campaigns @xcite , and conspiracy theories @xcite @xcite . research on detecting social media bots has always been an arms race @xcite : early methods focus on analyzing user metadata with machine learning classifiers @xcite , while bot operators manipulate user features to evade detection @xcite ; later approaches employed word embeddings and encoder-based language models to characterize user texts @xcite , while bot operators re-post genuine content to dilute malicious content and appear innocuous @xcite ; recent models tap into the network information of user interactions with graph neural networks @xcite @xcite , while advanced bots strategically follow and unfollow users to appear out-ofdistribution @xcite . recent advances brought us large language models (llms) that excel in academic tasks and benchmarks @xcite , capable of following instructions @xcite , but they also come with risks and biases that could cause realworld harms @xcite @xcite . in this work, we ask: what are the opportunities and risks of large language models in social bot detection? as the arms race escalates, we focus on how state-of-the-art large language models could aid robust bot detection systems and how llms might be maliciously employed to design more evasive bots. for opportunities, we propose a mixtureof-heterogeneous-experts framework, employing llms to divide and conquer various user information modalities such as metadata, text, and user interaction networks. for user metadata, we verbalize categorical and numerical user features in natural language sequences and employ in-context learning for bot detection. for user-generated texts, we retrieve similar posts from an annotated training set as in-context learning examples. for the network information, guided by previous works about llms' graph reasoning capabilities @xcite , we include the user's following information, in either random or similarity-based order, as part of the prompt context ## experiment settings models and settings we employ three llms to study their opportunities and risks in social media bot detection: mistral-7b @xcite , llama2-70b @xcite , and chatgpt. for in-context learning, we employ 16 in-context examples by default. for instruction tuning, we randomly sample 1,000 examples from the training set to adapt llms. we set temperature τ = 0.1 for language generation by default. specific prompt templates are listed in appendix c. ## analysis model calibration robust social bot detectors should provide not only a binary prediction but also a well-calibrated confidence score to facilitate content moderation. we evaluate how well are llmbased bot detectors calibrated, with the vanilla twibot-20 dataset as well as manipulated with the both combine strategy, in figure 2 . specifically, we use the probability of the prediction token ("human" or "bot") from the instruction-tuned chatgpt models as the bot likelihood, bin it into 10 buckets, and calculate the estimated calibration error (ece) @xcite . it is demonstrated that llm-based bot detectors are moderately calibrated with an ece of around 0.2, while llm-guided manipulation strategies harm calibration and increase ece by 28.4% on average. as a result, the risks of llms in social bot detection not only lie in decreased performance but also in less calibrated and thus less trustworthy predictions. ## conclusion we propose to investigate the opportunities and risks of llms in social media bot detection. as promising opportunities, we propose a mixture-ofheterogeneous-experts framework to adapt llms for bot detection through in-context learning or instruction tuning. as tangible risks, we propose text-and graph-based strategies to manipulate the information of bot accounts with the help of llms aiming to evade detection. extensive experiments demonstrate that llm-based bot detectors achieve state-of-the-art performance on two widely adopted bot detection datasets, but it is easier than ever to deploy an adversarial llm-based bot that successfully evades detection, especially for existing non-llm social bot detection models.