M - R ange D etector: Enhancing Generalization in Machine-Generated Text Detection through Multi-Range Attention Masks (2025)

Abstract: The increasing capability and widespread usage of large language models (LLMs) highlight the desirability of automatic detection of machine-generated text. Existing supervised detectors often overfit within their training domains, as they have primarily learned domain-specific textual features, such as word frequency, syntax, and semantics. In this paper, we introduce a domain-independent feature, namely the difference of writing strategy between LLMs and human, to improve the out-of-domain generalization capability of detectors. LLMs focus on the preceding range tokens when generating a token, while human consider multiple ranges, including bidirectional, global, and local contexts. The attention mask influences the range of tokens to which the model can attend. Therefore, we propose a method called M-RangeDetector, which integrates four distinct attention masking strategies into a Multi-Range Attention module, enabling the model to capture diverse writing strategies. Specifically, with the global mask, band mask, dilated mask, and random mask, our method learns various writing strategies for machine-generated text detection. The experimental results on three datasets demonstrate the superior generalization capability of our method.

## introduction large language models (llms), such as chat-gpt (openai, 2022), claude @xcite , deepseek @xcite , qwen @xcite , and llama @xcite , have been rapidly advancing. the text generated by these models is becoming increasingly fluent and humanlike, facilitating tasks such as question-answering and news reporting. however, this progress has also raised increasing concerns about the misuse of llms, including the spread of misinformation and academic dishonesty @xcite . this highlights the need for automated detection of machine-generated text. machine-generated text detection is a binary classification task that determines whether a given text is authored by human or generated by llms. early supervised methods @xcite use @xcite as an encoder to obtain token logits or representations for classification. due to the growing capabilities of llms in semantic understanding, more recent approaches @xcite have replaced roberta with llms, and some methods @xcite directly use llms to generate "machine" or "human" labels for detection. @xcite improves detection accuracy by conducting finegrained classification of texts generated by different models and human-written texts. these methods perform exceptionally well in specific training domains but struggle in out-of-distribution scenarios. this issue arises because existing methods primarily capture domain-specific features, such as word frequency, syntax, semantics, and sentiments @xcite . as these features can vary across domains, detectors often struggle with misclassification when applied to texts from new domains. in contrast, capturing domain-independent features is an effective method for improving the generalization capability of machine-generated text detection. there exists an inherent distinction between the writing strategies of llms and human, specifically in the range of tokens they attend to when generating a token, no matter the domain to which a text belongs. as illustrated in figure 1 , when generating the token "champ", llms are restricted to attending only to the preceding range of tokens, due to their masked attention mechanism. in contrast, when human write the word "champ", they may take into account diverse ranges of words, such as global, local, and bidirectional contexts, depending on their intentions. we propose the m-rangedetector method to enhance the generalization capability of machinegenerated text detection by modeling diverse writing strategies as domain-independent features. for a given text, we first obtain token representations using a proxy llm and then apply a multi-range attention module to extract diversified writing strategy features for each token. these features are concatenated and fed into a classifier to determine whether the text is machine-generated. specifically, the module integrates four distinct attention masking strategies to constrain different ranges during the computation of attention scores. the global mask captures relationships across all positions in the input sequence, the band mask focuses on local relationships between tokens on both sides, the dilated mask restricts a token's interaction with other tokens that are separated by a fixed interval, and the random mask randomly selects subsets of tokens from any position. the module can learn writing strategy features by calculating attention scores for each token within diverse specified ranges. this constraint ensures that the calculation of attention scores focuses on a range that closely aligns with the range a human would attend to during writing. notably, our method does not require fine-tuning the proxy llm for token representation. instead, it only requires training the multi-range attention module and the classifier, which demonstrates that our approach is a resource-efficient solution for detecting machine-generated text. our approach outperforms existing methods across multiple widely used datasets, including the ghostbuster dataset, the m4 monolingual and multilingual datasets, and the outfox dataset. our contributions can be summarized as follows: ## method in this section, we provide a comprehensive overview of our method. in section 3.1, we define the task of detecting machine-generated text and outline the key steps for utilizing the proxy llm to obtain token representations. in section 3.2, we introduce the multi-range attentions module that integrates four distinct attention masking strategies to restrict different ranges during attention computation. the architecture of m-rangedetector is given in figure 2 . ## experiments we first evaluate the effectiveness of our method in in-domain and out-of-domain detection scenarios with unknown domains. next, we assess the model's performance in detecting machinegenerated text from unseen generators and in multilingual settings. finally, we explore additional outof-domain detection scenarios, including generating the same text with prompts different from those used in the training set, using three datasets containing texts written by non-native english speakers, and evaluate the robustness of our approach against paraphrase attacks, as well as its performance in the authorship attribution task. ## conclusion in this study, we propose the different writing strategies between llms and human as domainindependent features for enhancing generalization in machine-generated text detection. we introduce a novel approach called m-rangedetector, which incorporates a multi-range attention module to learn different writing strategies for classification. the module incorporates four distinct attention masking strategies to constrain different ranges during the computation of attention scores. specifically, it employs global, band, dilated, and random masking strategies to capture the contextual representations of tokens across various token ranges, including global, local, and bidirectional ranges. our method achieves state-of-the-art performance on three widely used benchmarks. experimental results demonstrate that our approach not only exhibits exceptional generalization ability in unseen domains but also performs well on newly emerging llms and across multiple languages.