How to Generalize the Detection of AI -Generated Text: Confounding Neurons (2025)

Abstract: Detectors of LLM-generated text suffer from poor domain shifts generalization ability. Yet, reliable text detection methods in the wild are of paramount importance for plagiarism detection, integrity of the public discourse, and AI safety. Linguistic and domain confounders introduce spurious correlations, leading to poor out-of-distribution (OOD) performance. In this work we introduce the concept of confounding neurons, individual neurons within transformers-based detectors that encode dataset-specific biases rather than task-specific signals. Leveraging confounding neurons, we propose a novel post-hoc, neuron-level intervention framework to disentangle AI-generated text detection factors from data-specific biases. Through extensive experiments we prove its ability to effectively reduce topic-specific biases, enhancing the model’s ability to generalize across domains.

## introduction the rapid development of large language models (llms) has revolutionized natural language processing (nlp), allowing machines to produce text that mirrors human writing in coherence and contextual relevance. however, as llms become increasingly sophisticated, identifying ai-generated text poses a critical challenge @xcite . this task is particularly pressing in contexts such as academic integrity, misinformation detection, authorship attribution, and cybersecurity, where the misuse of ai-generated content raises ethical and societal issues @xcite . despite advancements in ai detection methods, state-of-the-art (sota) systems still exhibit significant generalization failures, especially when applied across diverse domains, languages, and models @xcite . detection methodologies for ai-generated text, based on fine-tuned transformer models (e.g., figure 1 : accuracy gain in a human-vs-llm detector bert-based removing the top-k (@xmath0. removing as few as 20 (∼ 0.05%) confounding neurons in the feed-forward mlps intermediate layers results in up to a 6.9% improvement in the test sets (in-domain, out-of-sample (oos), and four outof-distribution (ood)). the shaded area around zero corresponds to the random baseline. roberta, xlm-r), achieve high accuracy (>99%) on controlled datasets, their performance on out-ofdistribution (ood) data remains unreliable, limiting their applicability in real-world scenarios @xcite . recent benchmarks, such as m4 (multi-generator, multi-domain, multi-lingual) @xcite and the genai content detection tasks @xcite , have underscored the fragility of current detectors, particularly in crossdomain generalization @xcite . detectors struggle with generalization due to linguistic and domain confounders, which introduce spurious correlations that bias detection models @xcite . linguistic confounders, such as sentence length and lexical diversity, reflect training data artifacts rather than intrinsic ai text features, leading to poor out-ofdistribution (ood) performance @xcite . domain confounders further complicate detection by linking ai-generated text with specific topics or styles rather than universal generation patterns @xcite . for instance, detectors trained on academic ai text often fail with news articles, revealing a lack of cross-domain robustness @xcite . we propose a neuron-level framework of training-based ai text detectors, leveraging the concept of confounding neurons: specific neuronal activations that encode dataset-specific biases @xcite . by systematically identifying these neurons, we can analyze how and where spurious correlations emerge, enabling the development of generalizable detection strategies that prioritize intrinsic textual features rather than dataset-dependent artifacts. we show an example of the effectiveness of our approach in figure 1 . emerging research indicates that llms encode knowledge, writing styles, and topic preferences within specific neurons @xcite @xcite . while prior work has examined neurons in the context of knowledge storage and language generation, their role in ai-generated text detection remains largely unexplored. given that ai text detectors unintentionally encode dataset-specific biases, understanding their neuronal activations is crucial for disentangling the detection task from these confounding factors. we focus on computationally inexpensive and validated techniques within knowledge editing @xcite to build a framework for intervening at the neuronal level, enabling more reliable detection systems that generalize across text distributions. contributions this study advances robust llm-generated text detection through the following contributions: ## related works detection methods are mainly categorized into statistical methods, neural-based and llm-based detectors. statistical methods detect ai-generated text by analyzing linguistic features such as perplexity, n-gram frequency, or token distribution (hamed and @xcite . these methods are computationally efficient and perform well for simple llms, but their effectiveness decreases when faced with larger more advanced models @xcite . neural-based detectors, employing transformer architectures like bert @xcite , roberta @xcite , and xlm-r @xcite , achieve high accuracy (often exceeding 99%) on controlled datasets @xcite . however, their performance degrades significantly on out-of-distribution (ood) data, revealing limited generalization @xcite . a fundamental challenge for detection systems is achieving ood robustness. despite their high accuracy within specific domains, neural detectors struggle with diverse text types @xcite , as linguistic and domain confounders introduce spurious correlations that hinder generalization across domain shifts @xcite . ## methods the guiding hypothesis is that the cross-domain fragility of modern ai-text detectors originates in a small, localized subset of neurons whose activity encodes linguistic and domain confounders rather than generation source-specific signals. if these neurons are identified and "deactivated" after training in a post-hoc approach, the detector should preserve in-domain accuracy while exhibiting better generalization to unseen text distributions. to examine this hypothesis, we propose a model-agnostic framework (figure 2 ) based on neuron-level intervention: the framework allows for a controlled comparison between m 0 and m p on identical inputs; any gain in ood performance can thus be attributed to the removal of the confounding neurons. the remainder of this section details each stage, and §4 reports the empirical findings. ## experiments in this section, we evaluate the proposed neuronlevel intervention framework for llm-generated text detection 1 . the experiments are designed to assess the efficacy of our approach in addressing domain generalization challenges. we aim to answer three research questions: rq1 localisation & distribution: where are confounding signals concentrated, and how do they spread across layers? finding: just 20 neurons in the early transformer blocks govern up to +7% accuracy gains on ood text, while leaving in-domain performance intact, meanwhile, task-relevant "detection" neurons cluster almost exclusively in the final layers (fig. 1 , fig. 4 ).