When Detection Fails: The Power of Fine-Tuned Models to Generate Human-Like Social Media Text (2025)

Abstract: Detecting AI-generated text is a difficult problem to begin with; detecting AI-generated text on social media is made even more difficult due to the short text length and informal, idiosyncratic language of the internet. It is nonetheless important to tackle this problem, as social media represents a significant attack vector in online influence campaigns, which may be bolstered through the use of mass-produced AI-generated posts supporting (or opposing) particular policies, decisions, or events. We approach this problem with the mindset and resources of a reasonably sophisticated threat actor, and create a dataset of 505,159 AI-generated social media posts from a combination of open-source, closed-source, and fine-tuned LLMs, covering 11 different controversial topics. We show that while the posts can be detected under typical research assumptions about knowledge of and access to the generating models, under the more realistic assumption that an attacker will not release their fine-tuned model to the public, detectability drops dramatically. This result is confirmed with a human study. Ablation experiments highlight the vulnerability of various detection algorithms to fine-tuned LLMs. This result has implications across all detection domains, since fine-tuning is a generally applicable and realistic LLM use case.

## introduction large language models (llms) are able to produce increasingly complex and fluent output, creating an information environment where humans can no longer reliably distinguish between text that was written by other humans, and that which was generated by llms. this increases our vulnerability to various tactics of opinion manipulation, particularly in online spaces such as social media. these tactics include disinformation and misinformation campaigns, as well as astroturfing and "flooding the zone" attacks. while growing research effort has focused generally on the development of methods for detecting ai-generated text (aigt), very little work has focused on the pressing question of aigt detection in the social media domain. furthermore, much of the research has relied on assumptions that are highly unlikely to hold in real-world contexts, such as the assumption that the defender has knowledge of (or even access to) the llm that was used to generate the text they are trying to detect. in the current work, we are particularly motivated by the concept of astroturfing, or the artificial creation of an impression of widespread support for (or opposition to) a product, policy, or concept @xcite . by flooding the information space with posts advocating a particular stance on some issue, ai can be used to influence and change people's beliefs by exploiting their natural tendency toward "crowd mentality" and desire for social acceptance. crucially, astroturfing is distinct from disinformation, in that the sentiment being expressed is not necessarily false. this means that traditional methods of disinformation detection are not effective against such an attack. rather, the deception involved in astroturfing is the inauthenticity of the apparent crowd @xcite . therefore, the ability to detect whether a social media post on some topic originates from a real human or from ai becomes an important signal for authenticity determination. our work attempts to make realistic assumptions about the motivations and resources available to both attackers (those spreading the aigt) and defenders (those detecting the aigt) in this scenario. in contrast to previous work, which sampled social media posts on a broad variety of everyday topics @xcite , we assume that both attackers and defenders will be more motivated to participate in/monitor online discussions on controversial or political topics. we assume that the attacker will have a particular stance they want to communicate on a given topic, rather than simply summarizing or paraphrasing existing posts. we also assume they will take reasonable, low-cost steps to produce realistic text (such as fine-tuning the model to mimic the intended social media style), and that they will not release any fine-tuned models for the defender to access. while it is reasonable to assume that an actor will attempt to mimic the intended distribution through fine-tuning, we do not make any further assumptions that the attacker is actively trying to evade detection, which may require more sophisticated knowledge of modern detection schemes. therefore, we examine four related research questions: we run a human study with 250 participants and find that texts from fine-tuned models are essentially undetectable to human readers (54% detection accuracy) (section 4.3). in addition to these research contributions, we also make our dataset of 505,159 english-language ai-generated texts available to other researchers. ## background here, we briefly summarize the state-of-the-art nlp approaches to detecting aigt, along with their strengths and weaknesses; more in-depth background is available in recent survey papers @xcite @xcite @xcite @xcite . one category of detection methods is metricbased approaches that rely on detecting the statistical regularities of aigt resulting from the generation process. these include measures of the log-likelihood or rank of each token, with respect to the model's probability distribution over tokens at each step, or measures of the overall entropy or perplexity of the text @xcite . the detectgpt algorithm @xcite extends these ideas to estimate the local curvature of the model's log probability function, laying the groundwork for fast-detectgpt @xcite . these are all so-called "zeroshot" methods that do not require training data (although in practice, some in-domain data is needed to calibrate the decision threshold). they also rely on information about the probability distribution of the generating model, which ideally is obtained through white-box access to that model. other methods use probability values from opensource llms as proxies for the unknown true probability distribution of the generating model @xcite @xcite . the binoculars method @xcite combines the benefits of black-box detection and "zeroshot" classification by measuring cross-entropy between two open-source models as a single feature to distinguish aigt from human-written text. in contrast to the feature-engineering methods described above, another broad set of aigtdetectors involves fine-tuning pretrained language models (plms), such as roberta, for the classification task. such methods are limited by the fact that they require larger quantities of labeled training data, and classifiers trained on older llms do not generalize to text from newer llms @xcite . however, @xcite found that, compared to metric-based methods, a fine-tuned plm detector performed better when tested on unseen domains and unseen generating models. very little work has focused specifically on social media text @xcite @xcite . one recent exception is the multisocial dataset, comprising 472,097 texts from seven multilingual llms, and covering five different social media platforms @xcite . this dataset represents a valuable contribution to an understudied field. however, the data generation strategy involved broadly sampling social media posts and then generating aigt paraphrases of the posts, which does not accurately represent known threat models of ai-generated content on social media @xcite . here, we choose specific topics more likely to be targeted by influence campaigns, compare different methods of prompting to generate the aigt, and explore the use of finetuned generating models. ## conclusion a realistically motivated attacker is likely to finetune a model for their specific style and use case, as it is cheap and easy to do so. with minimal effort, time, and money, we produced fine-tuned generators that are capable of much more realistic socialmedia tweets, based on both linguistic features and detection accuracy, and verified through human annotations. although motivated by the spread of ai content on social media, and the associated risks of astroturfing and influence campaigns, we stress that the main findings extend across all text domains. indeed, fine-tuning models for style-specific content generation is a generally applicable method, and one that is likely already in use by many generative ai users -calling into question whether existing methods of detecting aigt are as effective in the real world as in the research lab. the fast-detectgpt matrix shows measurement models (columns) against generating models (rows). measurement models may be equal to the generating model (green), share a common base model (yellow), share a common model family (orange), or share a common fine-tuning dataset (purple). for each generating llm, the best-performing detector is in bold and the second-best is underlined. outside model families, purple indicates that the training set has been generated by a model that was fine-tuned on the same data as the generator.