What Does the Bot Say? Opportunities and Risks of Large Language Models in Social Media Bot Detection (2024)

Abstract: Social media bot detection has always been an arms race between advancements in machine learning bot detectors and adversarial bot strategies to evade detection. In this work, we bring the arms race to the next level by investigating the opportunities and risks of state-of-the-art large language models (LLMs) in social bot detection. To investigate the opportunities, we design novel LLM-based bot detectors by proposing a mixture-of-heterogeneous-experts framework to divide and conquer diverse user information modalities. To illuminate the risks, we explore the possibility of LLM-guided manipulation of user textual and structured information to evade detection. Extensive experiments with three LLMs on two datasets demonstrate that instruction tuning on merely 1,000 annotated examples produces specialized LLMs that outperform state-of-the-art bot detection baselines by up to 9.1% on both datasets. On the other hand, LLM-guided manipulation strategies could significantly bring down the performance of existing bot detectors by up to 29.6% and harm the calibration and reliability of bot detection systems. Ultimately, this works identifies LLMs as the new frontier of social bot detection research.1

heading: 1 Introduction
text: Social media bot accounts are behind many online perils such as misinformation (Lu and Li, 2020; Huang et al., 2022), election interference (Howard et al., 2016; Rossi et al., 2020; Ng et al., 2022), extremist campaigns (Ferrara et al., 2016; Marcellino et al., 2020), and conspiracy theories (Ferrara, 2020; Ahmed et al., 2020; Ginossar et al., 2022). Research on detecting social media bots has always been an arms race (Cresci et al., 2017): early methods focus on analyzing user metadata with machine learning classifiers (Yang et al., 2020;
1Code and data will be publicly available at https://github.com/BunsenFeng/botsay.
Echeverrï£¡ a et al., 2018), while bot operators manipulate user features to evade detection (Cresci, 2020); later approaches employed word embeddings and encoder-based language models to characterize user texts (Wei and Nguyen, 2019; Dukić et al., 2020), while bot operators re-post genuine content to dilute malicious content and appear innocuous (Cresci, 2020); recent models tap into the network information of user interactions with graph neural networks (Feng et al., 2021c; Huang et al., 2022; Lei et al., 2023), while advanced bots strategically follow and unfollow users to appear out-ofdistribution (Ye et al., 2023; Li et al., 2023b).
Recent advances brought us large language models (LLMs) that excel in academic tasks and benchmarks (Liang et al., 2023), capable of following instructions (Ouyang et al., 2022), but they also come with risks and biases that could cause realworld harms (Weidinger et al., 2022; Kumar et al., 2023b; Feng et al., 2023). In this work, we ask: What are the opportunities and risks of large language models in social bot detection? As the arms race escalates, we focus on how state-of-the-art large language models could aid robust bot detection systems and how LLMs might be maliciously employed to design more evasive bots.
For opportunities, we propose a mixtureof-heterogeneous-experts framework, employing LLMs to divide and conquer various user information modalities such as metadata, text, and user interaction networks. For user metadata, we verbalize categorical and numerical user features in natural language sequences and employ in-context learning for bot detection. For user-generated texts, we retrieve similar posts from an annotated training set as in-context learning examples. For the network information, guided by previous works about LLMs’ graph reasoning capabilities (Wang et al., 2024; Huang et al., 2023b), we include the user’s following information, in either random or similarity-based order, as part of the prompt context
3580
Opportunities
In-Context Learning  Instruction Tuning ⚙
Risks
Ensemble 
Metadata Description The following task focuses on evaluating whether a Twitter user is a bot or human with the help of several labeled examples.
<in-context examples>

heading: Username: <username>
text: Follower count: 309 Following count: 1412 Tweet count: 1745 Verified: False Active years: 12 years Label: bot
The following task focuses on evaluating whether a Twitter user is a bot or human with the help of the user's self-written description.
<in-context examples>
Description: A marketer in and out. Writes on marketing & sometimes straight from the heart. Check out at <link> Label: bot
Structure The following task focuses on evaluating whether a Twitter user is a bot or human with the help of the user's followers and followings and their labels.
These users follow the target user: <user information> The target user follows these users: <user information> Target user: <target user information> Label: human
Textual Manipulation Zero-shot Few-shot
Classifier Guide
Text attribute
Below is a description of a Twitter user and its variants, paired with their score predicted by a bot classifier. Please rewrite the description to make the user appear more human. Description: <description at step 0> Score: 0.68 … Description: <description at step i> Score: 0.26 Description: <description at step i+1>
Step 1: Bot Descriptions: <top-k similar bot descriptions> Human Descriptions: <top-k similar human descriptions> Compare and give the key distinct feature of human's descriptions: <generated attribute> Step 2: <generated attribute> Based on the description, paraphrase this to human description: Bot: <description> Human:
Text Attribute
Structural Manipulation Add Neighbor
Below is a target Twitter bot and five potential new users to follow. Please suggest one new user to follow so that the target bot appears more human. Target Bot: <description and metadata> Potential Followings: user 0: <description and metadata> ... Please select one user to follow (1-k): Remove Neighbor Below is a target Twitter bot and five potential users to unfollow. Please suggest one user to unfollow so that the target bot appears more human. Target Bot: <description and metadata> Potential users to unfollow: user 0: <description and metadata> … Please select one user to unfollow (1-k):
Combine
Both (add & remove)
add
add
re mo
ve
re mo
ve
description
score
(a) opportunities (b) risks
Figure 1: Overview of the opportunities of LLM-based bot detectors and risks of LLM-based evasive bots.
to aid detection. These modality-specific LLMs are then used through in-context learning prompting or instruction tuning, and modality-specific results are ensembled through majority voting.
For risks, we investigate the possibility of LLMguided bot design to evade detection by tampering with the textual and structural information of bot accounts. For text al information, we explore rewriting user posts with LLMs to appear genuine with four mechanisms: 1) zero-shot prompting; 2) few-shot rewriting to imitate the posts of genuine users; 3) interactive rewriting between LLMs and an external bot classifier; 4) synthesizing the attributes of related posts from bots and humans for style transfer. For structural information, we employ LLMs to suggest new users to follow or existing users to unfollow, editing the neighborhood of bot accounts. LLM-guided manipulation of textual and structural features is then merged to produce LLM-guided social media bots.
We conduct extensive experiments with three LLMs on two standard bot detection datasets to evaluate the proposed detectors and manipulation strategies. We find that on the opportunities side, LLMs are liable to become state-of-the-art detectors: while in-context learning struggles to capture the nuances of bot accounts, instruction tuning outperforms baselines by up to 9.1% on both datasets. With respect to threat and risk modeling, LLMguided manipulations on both textual and structural information reduce the performance of existing detectors by up to 29.6%, and LLM-based detectors are more robust towards bots designed by LLMs. Our work opens up new research avenues in the ever-lasting arms race between researchers and bot operators, focusing on LLMs as the new frontier of social bot detection research.

heading: 2 Methodology

heading: 2.1 Opportunities: Large Language Models as Better Bot Detectors
text: Social media bot detection focuses on evaluating and classifying social media accounts into bot or human based on diverse user information: user metadata M = {m1, . . . ,mk} where each mi is either a numerical or categorical feature; user posts T = {t1, . . . , tℓ} where each ti is a natural language sequence; user network information N = {N1,N2} where N1 denotes the user’s followers’ set and N2 denotes the following set. We aim to develop bot detectors f(M, T ,N ) → {human, bot}.
We develop LLM-based bot detectors by proposing a mixture-of-heterogeneous-experts framework to tackle the diverse user information. Specifically, different user information modalities are separately analyzed with LLMs while majority voting is conducted to ensemble uni-modality predictions. Each modality-specific predictor either uses the LLM off-the-shelf with in-context learning (Brown et al., 2020) or employs instruction tuning (Ouyang et al., 2022) to adapt LLM for analyzing a particular set of user information. We present an overview of the proposed framework in Figure 1.
Metadata-Based We sequentially concatenate an account’s metadata M to linearize it as a natural language sequence. We then randomly select a balanced set of n in-context examples, and provide their metadata as well as the labels in the prompt.
Text-Based For each textual sequence t ∈ T , we first retrieve the top-n similar user posts in the training set with a retrieval system (Robertson et al., 2009). We then similarly employ in-context learning with the LLMs to make predictions for all posts in T and conduct a majority vote.
We also employ a meta+text approach where both user metadata and textual posts are presented for in-context learning with LLMs.
Structure-Based In addition to analyzing each user individually, interactions among users and the graph structure they form are also crucial in identifying advanced bot clusters (Liu et al., 2023). Grounded in previous research demonstrating that LLMs do have preliminary abilities to reason over graphs and structured data (Wang et al., 2024), we employ LLMs to analyze a user’s neighborhood N of follow relations.
Concretely, we employ the following prompt to linearize the neighborhood structure of a given user: “These users follow the target user: PERM(N1). The target user follows these users: PERM(N2)”, where PERM(·) denotes a permutation function regarding how to order and arrange the follower/following set. We employ two modes for PERM: 1) random, where users along with their information are linearized in random order; 2) attention: inspired by the success of graph attention networks (Veličković et al., 2018; Huang et al., 2023b) and the variation in edge importance in a network, we arrange users based on their similarity to the target account. Formally, given the target user’s post t, a neighboring user’s similarity score could be defined as sim(enc(t), enc(t′)), where sim(·, ·) denotes cosine similarity, enc(·) denotes an encoder-based LM, and t′ denotes the post of the neighboring account. PERM then arranges the users based on their similarity scores from high to low, along with the prompt “from most related to least related:” to encourage LLMs to take the relative similarity/importance of neighbors into account.
After developing five LLM predictors analyzing different user information modalities (metadata, text, metadata+text, structure-random, and structure-attention), they are employed through either in-context learning or instruction tuning.
In-Context Learning We directly prompt the LLM off-the-shelf, without any tuning or adaptation, with the n in-context examples and labels as well as the target user’s information.
Instruction Tuning We employ meta-learning with in-context learning (Min et al., 2022a) to adapt the LLM for better analyzing a specific user information source through instruction tuning. Instruction tuning aims to improve LLMs’ ability to follow instructions by fine-tuning LLMs on triples of
{instruction, input, output} (Ouyang et al., 2022). We write a short instruction based on each modality, use the information of in-context examples and target user as input, and the gold label as output2.
The predictions of each modality-specific LLM are then ensembled by majority voting into one prediction of whether the target user is a bot or not.

heading: 2.2 Risks: Large Language Models as Evasive Bot Designers
text: On the risks side, we explore how LLMs might be employed to design advanced bots to evade detection. While user metadata M is often hard to manipulate with the help of LLMs (e.g. # of followers and account creation time), textual information T and structural information N could be easily altered with LLM-generated post paraphrases and LLM-suggested users to follow and unfollow. We first explore possibilities of manipulating textual information T , focusing on rewriting the posts of bot accounts with LLMs to evade detection.
Zero-Shot Rewriting We directly prompt the LLM with “Please rewrite the description of this bot account to sound like a genuine user.”
Few-Shot Rewriting We employ a retrieval system to employ the top-n most similar posts to the target post that are written by genuine users. We then prompt the LLM to imitate these examples and rewrite the target bot post.
Classifier Guidance We propose to empower LLMs to iteratively refine a bot-generated post with feedback from an external classifier. Specifically, we first train an encoder-based LM to classify user posts into bot or human and produce a confidence score f(t) → [0, 1]. At each step, the LLM learns from the rewritten posts in the previous steps along with the confidence scores given to those posts, aiming to reduce the bot likelihood in the eye of the external classifier. Formally, ti+1 = LLM(ti, f(ti), . . . , t0, f(t0)) where t0 is the original bot post. This process is repeated for n times, producing a paraphrased bot post that learns from the edit history and the external classifier.
Text Attributes Previous works have demonstrated that LLMs could summarize the differences between machine-generated and human-written text and employ the summary for better detection (Lu et al., 2023). To this end, we first retrieve the
2Prompt details in Appendix C.
top-n similar posts from human accounts and topn from bots, then prompt the LLM to summarize the differences in text attributes between the two groups of posts. In a separate prompt, the LLM then rewrites the target bot post with the help of the summarized difference.
Aside from editing user textual information, we also tap into LLMs’ capabilities of preliminary graph reasoning (Wang et al., 2024) and employ them to edit the structural information, specifically by adding and removing users to follow for a target bot. We investigate whether LLMs might be capable of suggesting reasonable neighbors to make the bot seem more genuine or identifying current neighbors that might give away its bot nature.
Add Neighbor We randomly select n users that the target bot is not currently following. We then prompt the LLM to “Please suggest one new user to follow so that the target bot appears more human.” by providing the metadata and textual information of these users and the target bot.
Remove Neighbor We prompt the LLM to “Please suggest one user to unfollow so that the target bot appears more human.” by providing the metadata and textual information of the target bot and its current following list.
Combine Neighbor We combine the results of add neighbor and remove neighbor.
The manipulation strategies for textual and structural user information could be further merged to design bots that are more evasive in both aspects.
Selective Combine State-of-the-art bot detection approaches often jointly analyze multiple sources of user information (Tan et al., 2023), but not all modalities are malicious and give away the bot nature (Liu et al., 2023). To this end, we employ LLMs to judge which information modality, text or graph, could be malicious in a given bot and employ the corresponding manipulation strategy. Specifically, we first provide LLMs with rationale about how existing bot detectors work in a prepended passage. We then provide all M, T , and N for a given bot, prompting the LLM to evaluate whether the textual, structural, or both user information seems malicious. The manipulation strategies of classifier guidance and combine neighbor are then selectively activated to edit the bot account.
Both Combine We simply merge the edits of classifier guidance and combine neighbor for a
given bot account’s textual and structural features.

heading: 3 Experiment Settings
text: Models and Settings We employ three LLMs to study their opportunities and risks in social media bot detection: Mistral-7B (Jiang et al., 2023a), LLaMA2-70b (Touvron et al., 2023), and ChatGPT. For in-context learning, we employ 16 in-context examples by default. For instruction tuning, we randomly sample 1,000 examples from the training set to adapt LLMs. We set temperature τ = 0.1 for language generation by default. Specific prompt templates are listed in Appendix C.
Datasets We experiment with two comprehensive benchmarks of social bot detection: TwiBot20 (Feng et al., 2021b) and TwiBot-22 (Feng et al., 2022b), two graph-based datasets providing diverse user and bot interactions on social media. These datasets mainly feature English social media posts but other languages are occasionally included.
Baselines On the opportunities side, we compare our proposed LLM-based bot detectors with 9 baselines leveraging varying aspects of user information: SGBot (Yang et al., 2020), LOBO (Echeverrï£¡ a et al., 2018), RoBERTa (Liu et al., 2019), RGT (Feng et al., 2022a), Botometer (Yang et al., 2022), BotBuster (Ng and Carley, 2023), BotPercent (Tan et al., 2023), BIC (Lei et al., 2023), and LMBot (Cai et al., 2024). We provide more baseline details in Appendix A.3.

heading: 4 Results

heading: 4.1 Opportunities
text: We present the performance of baselines and our LLM-based detectors in Table 1.
LLM-based detectors achieve state-of-the-art performance. On both datasets, ChatGPTensemble with instruction tuning outperforms the strongest baseline by 2.6% and 9.1% on F1-score. In addition, ChatGPT with instruction tuning outperforms in-context learning by 34.7% in accuracy: we hypothesize that while in-context learning abilities are attributed to pretraining data (Min et al., 2022b) and LLMs have seen social media texts (Dodge et al., 2021), the nuances of bot accounts are beyond simple data artifacts and would need model adaptation and reasoning. We also find that larger LMs are better at social bot detection. On average, Mistral-7B, LLaMA2-70B, and ChatGPT
achieve 0.5651, 0.6347, and 0.6478 accuracy on the two datasets. This ranking is in line with their general utility on standard NLP benchmarks.
A combination of modality-specific LLMs yields promising results. For ChatGPT with instruction tuning, while the text-only detector trails in performance and LLMs are better in leveraging the structural information of accounts, an ensemble of modality-specific predictions through majority voting improves performance. This echoes the finding that not all modalities of a bot account are malicious (Liu et al., 2023) and our proposed mixtureof-heterogeneous-experts framework jointly considers multiple user information modalities.
LLMs are worth the tradeoff between compute and data annotations. While existing su-
pervised approaches are lightweight and inexpensive to run, they are trained on large quantities of annotated accounts (around 8k and 700k for the two datasets). On the contrary, while LLM-based approaches require significant computational resources, they are only instruction-tuned on 1k annotated users and achieve superior results. We argue that LLM-based bot detectors are thus promising approaches, given that data annotations in bot detection are hard, noisy, and scarce (Feng et al., 2021a), while the compute overhead will be continuously reduced due to innovations in efficient training and inference (Dao, 2023; Dettmers et al., 2024).

heading: 4.2 Risks
text: We evaluate existing detectors and LLM-based approaches on the LLM-manipulated bot accounts in
Twibot-20 and present performance in Table 2.
LLM-based detectors are less sensitive to manipulation strategies. While BotPercent and BotRGCN suffer from a 10.9% and 7.7% drop in accuracy on average due to manipulation strategies, LLM-ensemble only shows a 2.3% drop. In addition, ChatGPT-based detectors are less robust to edits by another LLM (LLaMA2-70B) than itself, suggesting that LLMs might be able to identify artifacts generated by themselves (Pu et al., 2023).
Classifier guidance is the most successful among textual manipulations. On average, classifier guidance achieved a 6.0% and 3.2% drop in accuracy and F1-score. This indicates that LLMs could iteratively refine generations based on feedback from an external classifier; we further investigate the LLM-classifier interaction in Section 4.
Removing neighbors is better than adding. The two strategies achieve 5.0% and 2.5% drops in accuracy on average, respectively: we hypothesize that while suggesting a new account to follow from five accounts is a noisy task, removing one of the existing followings that makes the bot seem malicious is more straightforward and effective. Combining the removals and additions only led to performance drops in 5 of the 16 scenarios, suggest-
ing that strategically following accounts is harder for existing LLMs.

heading: 5 Analysis
text: Model Calibration Robust social bot detectors should provide not only a binary prediction but also a well-calibrated confidence score to facilitate content moderation. We evaluate how well are LLMbased bot detectors calibrated, with the vanilla Twibot-20 dataset as well as manipulated with the BOTH COMBINE strategy, in Figure 2. Specifically, we use the probability of the prediction token (“human” or “bot”) from the instruction-tuned ChatGPT models as the bot likelihood, bin it into 10 buckets, and calculate the estimated calibration error (ECE) (Guo et al., 2017). It is demonstrated that LLM-based bot detectors are moderately calibrated with an ECE of around 0.2, while LLM-guided manipulation strategies harm calibration and increase ECE by 28.4% on average. As a result, the risks of LLMs in social bot detection not only lie in decreased performance but also in less calibrated and thus less trustworthy predictions.
Text Rewrite Similarity To evade detection, it would be most effective if LLM removed all malicious content/intent in the bot-generated posts: however, that would defeat the purpose of LLM-
guided bot design. Following previous works (Li et al., 2023a; Kim et al., 2023a), we employ GPT-4 to evaluate whether the LLM-paraphrased bot posts still “preserve” the potentially malicious content. Specifically, we prompt GPT-4 with “For the following two posts of social media users, how similar are they in content?” and solicit a response on a 4-point Likert scale from “1: very different” to “4: very similar”. Figure 3 demonstrates that LLMs are generally preserving the content of bot posts, while the text attribute strategy is most faithful.
Classifier Guidance Convergence Section 4.2 demonstrates that classifier guidance is the most effective approach among text-based manipulations, showcasing the potential of LLMs iteratively refining generations based on feedback from external classifiers, but with increased inference latency. We further investigate the trend of bot scores given by the external classifier along with the five iterations in Figure 4: It is demonstrated that the bot scores do steadily decrease through iterations, while ChatGPT is more effective than LLaMA2-70B.
Statistics of Added/Removed Neighbors LLMguided additions/removals of bot neighbors are also successful in compromising existing bot detectors: we investigate the statistics of the removed/added accounts in Figure 5. It is demonstrated that LLMs do not simply follow established heuristics, such as “follow accounts with a lot of followers to seem genuine”, but rather examine in a case-by-case manner and suggest diverse edits of bot neighborhood.
# of In-Context Examples We investigate the impact of in-context examples in LLM-based bot detectors by increasing the amount from 0 to 16 and present model performance in Figure 6: Performance steadily increases with the amount of in-context examples. However, the context length limit of LLMs sets an upper bound of the amount of in-context examples: future work might explore whether long/infinite-context LLMs (Chen et al., 2023b; Bertsch et al., 2024) might benefit from a growing amount of in-context examples.

heading: 6 Related Work
text: Social Media Bot Detection Existing social media bot detection methods fall into three categories: feature-, text-, and graph-based (Feng et al., 2022b). Feature-based methods extract features from users’ metadata (Yang et al., 2020; Kudugunta and Ferrara, 2018), tweets (Miller et al., 2014), description (Hayawi et al., 2022), temporal patterns (Mazza et al., 2019), and follow relationships (Feng et al., 2021a) for feature engineering. Text-based models mine user-generated content such as tweets and descriptions using NLP techniques, including word embeddings (Wei and Nguyen, 2019), RNN (Kudugunta and Ferrara, 2018), attention mechanism (Feng et al., 2021a), and pretrained language models (Dukić et al., 2020). Graph-based methods focus on modeling user interactions in social networks and achieve state-of-the-art bot detection performance, approaches including node centrality (Dehghan et al., 2023), node representation learning (Pham et al., 2022), graph neural networks (Feng et al., 2021c, 2022a), and mixture-of-expert (Liu et al., 2023; Tan et al., 2023). As LLMs are revolutionizing text and graph mining on social networks (Tan and Jiang, 2023; Jin et al., 2023), we are the first to explore the opportunities and risks of LLMs in social bot detection.
LLMs for Content Moderation Aside from advancing on standard NLP tasks and benchmarks, LLMs have also shown great potential for various scenarios of content moderation (Kumar et al., 2023a; Ziems et al., 2023; Ma et al., 2023). LLMs have been widely employed to detect and counter hate speech (Jiang et al., 2023b; Vishwamitra et al.,
2024; Pendzel et al., 2023; Van and Wu, 2023; Nasir et al., 2023; Agarwal et al., 2023; Roy et al., 2023; Mendelsohn et al., 2023), with existing works focusing on improving their reasoning and robustness (Yang et al., 2023; Roy et al., 2023), mitigating LLMs’ social biases (Zhang et al., 2023; Mun et al., 2023), enhancing LLMs for machinegenerated hate speech in adversarial settings (Kim et al., 2023b; Sen et al., 2023; Ocampo et al., 2023), as well as employing LLMs for explainability (Wang et al., 2023; Huang et al., 2023a). LLM-based solutions have also been proposed for misinformation detection (Jiang et al., 2024; Pelrine et al., 2023; Hu et al., 2024; Nakshatri et al., 2023; Sundriyal et al., 2023; Su et al., 2023a; Li et al., 2023c; Chen et al., 2023a; Choi and Ferrara, 2024; Wang and Shu, 2023; Leite et al., 2023; Vykopal et al., 2023), with a focus on detecting machine-generated fake news (Huang et al., 2023c; Pan et al., 2023; Su et al., 2023b; Xu et al., 2023; Chen and Shu, 2023) and in adversarial settings (Han et al., 2023; Lucas et al., 2023; Wu and Hooi, 2023). In this work, we investigate LLMs’ opportunities and risks in social bot detection (Luceri et al., 2024), highlighting the potential of LLMs as stateof-the-art bot detectors as well as the dual-use risks for designing advanced and evasive social bots.

heading: 7 Conclusion
text: We propose to investigate the opportunities and risks of LLMs in social media bot detection. As promising opportunities, we propose a mixture-ofheterogeneous-experts framework to adapt LLMs for bot detection through in-context learning or instruction tuning. As tangible risks, we propose
text- and graph-based strategies to manipulate the information of bot accounts with the help of LLMs aiming to evade detection. Extensive experiments demonstrate that LLM-based bot detectors achieve state-of-the-art performance on two widely adopted bot detection datasets, but it is easier than ever to deploy an adversarial LLM-based bot that successfully evades detection, especially for existing non-LLM social bot detection models.

heading: Limitations
text: While our proposed LLM-based bot detectors and LLM-guided bot manipulations are generic and platform-agnostic, the experiments in this work focus primarily on the Twitter/X platform. This is due to the availability of annotated social media data while we expect to expand our experiments and analysis to other social media platforms such as TikTok, Reddit, and more, in future work.
We employ Twibot-20 and Twibot-22, two widely adopted datasets collected in and before 2022, to evaluate our proposed detectors and manipulation strategies. However, social media bot accounts are constantly evolving to evade detection (Cresci et al., 2017): we could not experiment with more up-to-date bot accounts again due to data availability, for example, the X platform has cancelled its academic research API access. We hope to test out LLM-based detectors and manipulation strategies with more up-to-date data with research access to social media data.

heading: Ethics Statement
text: The adversarial nature of social bot detection involves threat modeling and the development of evasive bots. This research is essential to model LLM risks and develop defense measures, while it also increases the risks of dual-use. We as authors aim to mitigate such dual use by employing controlled access to the social media data and trained models, ensuring that it is only employed for research purposes.
Language models have been extensively documented to have inherent social biases (Blodgett et al., 2020; Jin et al., 2021; Bender et al., 2021; Shaikh et al., 2023), and such biases could have an impact on downstream tasks such as hate speech detection (Xia et al., 2020) and misinformation (Feng et al., 2023). We expect social media bot detection to be no exception. We hypothesize that LLM-based bot detectors might underserve cer-
tain users and communities, potentially informed by LLMs’ internal biases, stereotypes, and spurious correlations. We argue that the decisions of LLM-based bot detectors should be interpreted as an initial screening of malicious accounts, while content moderation decisions should be made with humans in the loop. Future work could also investigate the fairness implications of social media bot detectors based on LLMs and other machine learning models.

heading: Acknowledgements
text: This material is based upon work supported by the National Science Foundation under CAREER Grant No. IIS2142739, NSF Grant No. IIS2203097, and the DARPA CMO under Contract No. HR001120C0124. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the funding agencies. We also gratefully acknowledge support from Alfred P. Sloan Foundation Fellowship.

heading: A Experiment Details

heading: A.1 Dataset Details
text: We employ two widely adopted datasets in social media bot detection, Twibot-20 (Feng et al., 2021b) and Twibot-22 (Feng et al., 2022b), to evaluate LLM-based bot detectors and LLM-based manipulation strategies. For Twibot-20, we employ the original test split of 1,183 accounts for evaluation. For Twibot-22, we employ the test split of 340 accounts employed in Tan et al. (2023). For instruction tuning LLMs on both datasets, we downsample the original training split into 1,000 users due to limits in computational budget.

heading: A.2 LLM Details
text: We employ three LLMs to evaluate their opportunities and risks in social bot detection: 1) Mistral-7B, through the MISTRALAI/MISTRAL7B-INSTRUCT-V0.1 checkpoint publicly available on Huggingface (Wolf et al., 2019); 2) LLaMA270B, through the META-LLAMA/LLAMA-2-70BCHAT-HF checkpoint publicly available on Huggingface; 3) ChatGPT, through the GPT-3.5TURBO-INSTRUCT checkpoint with the OpenAI API.

heading: A.3 Baseline Details
text: • BIC (Leite et al., 2023) incorporates text and graph modalities in social networks using a textgraph interaction module and models user behavior consistency with a semantic consistency module.
• LMBot (Cai et al., 2024) distills graph knowledge into encoder-only language models with iterative distillation between graph-based social bot detection model and language model.
• SGBot (Yang et al., 2020) is a feature-based method that identifies bots using random forest based on 8 types of user metadata and 12 derived features.
• BotPercent (Tan et al., 2023) is an amalgamation of Twitter bot detection datasets and feature-, text-, and graph-based models to probe the percentage of bot accounts in Twitter communities.
• RoBERTa (Liu et al., 2019) leverages the pretrained language model RoBERTa to encode user tweets and descriptions, then feed them into an MLP classifier.
• Botometer (Yang et al., 2022) is a public website to check the activity of a Twitter account and gives the score of how likely the user is a social bot. Botometer’s classification system leverages more than 1,000 features using available metadata and information extracted from interaction patterns and content.
• BotBuster (Ng and Carley, 2023) is a social bot detection system that processes user metadata and textual information using the mixtureof-expert architecture to adapt to multiple social platforms.
• LOBO (Echeverrï£¡ a et al., 2018) is a featurebased social bot detection method that extracts 26 features and adopts random forest for classification.
• RGT (Feng et al., 2022a) is a graph-based social bot detection method that models the intrinsic influence and relation heterogeneity in social networks.
A.4 Implementation Details
For in-context learning, we employ 16 in-context examples by default. For account metadata, we employ five entries: follower count, following count, tweet count, verified, and active years as they are most helpful in identifying social bots. For structure-based detectors, we include a maximum of five followers/followings for each account. For classifier guidance, we employ a finetuned ROBERTA-BASE language model (Liu et al., 2019) as the classifier based on user descriptions. For add neighbors, we provide a total of five accounts for LLMs to choose from. The ensemble of LLM-based detectors is a majority vote based on five detectors, metadata, text, meta+text, structurerandom, and structure-attention.

heading: B Analysis (cont.)
text: Qualitative Analysis We present qualitative examples of LLM-based paraphrasing of bot posts in Tables 4 to 8. It is demonstrated that LLMs could enrich the context of bot posts to seem genuine and add emojis and attributes (e.g. WSJ reporter) to seem personal and credible. However, LLMs also change the language of the original bot posts while using the word “regular” too often in generated posts. This indicates that while LLM-based textual manipulations could make bot accounts seem more
genuine, they also introduce new biases and signals for future bot detectors to leverage.
Fine-tuning MISTRAL-7B We conduct additional experiments to instruction-tune Mistral-7B with parameter-efficient training and present the results in the following table, comparing instruction tuning with in-context learning (ICL) using Mistral-7B. Table 3 echoes the finding with ChatGPT: Mistral-7B with instruction tuning also significantly boosts its bot detection utility compared to off-the-shelf prompting, albeit the improvements are generally less than ChatGPT.

heading: C Prompt Format
text: We provide specific prompt templates for the proposed approaches in Tables 9 to 20. Note that LLMs might be sensitive to varying prompt formats (Sclar et al., 2023) and the most optimal prompt for bot detection might vary for future LLMs.

heading: Method Acc, 20 F1, 20 Precision, 20 Recall, 20 Acc, 22 F1, 22 Precision, 22 Recall, 22

heading: Future Work
text: , 2023) and the most optimal prompt for bot detection might vary for future LLMs