M-RangeDetector: Enhancing Generalization in Machine-Generated Text Detection through Multi-Range Attention Masks
Authors: Kaijie Jiao, Quan Wang, Licheng Zhang, Zikang Guo, Zhendong Mao
Accepted at: acl25

M-RangeDetector: Enhancing Generalization in Machine-Generated Text Detection through Multi-Range Attention Masks

Kaijie Jiao1, Quan Wang2, Licheng Zhang1, Zikang Guo1, Zhendong Mao1*
1University of Science and Technology of China, Hefei, China
2MOE Key Laboratory of Trustworthy Distributed Computing and Service, Beijing University of Posts and Telecommunications, Beijing, China
jiaokaijie@mail.ustc.edu.cn, zdmao@ustc.edu.cn

Abstract

The increasing capability and widespread usage of large language models (LLMs) highlight the desirability of automatic detection of machine-generated text. Existing supervised detectors often overfit within their training domains, as they have primarily learned domain-specific textual features, such as word frequency, syntax, and semantics. In this paper, we introduce a domain-independent feature, namely the difference of writing strategy between LLMs and human, to improve the out-of-domain generalization capability of detectors. LLMs focus on the preceding range tokens when generating a token, while human consider multiple ranges, including bidirectional, global, and local contexts. The attention mask influences the range of tokens to which the model can attend. Therefore, we propose a method called M-RangeDetector, which integrates four distinct attention masking strategies into a Multi-Range Attention module, enabling the model to capture diverse writing strategies. Specifically, with the global mask, band mask, dilated mask, and random mask, our method learns various writing strategies for machine-generated text detection. The experimental results on three datasets demonstrate the superior generalization capability of our method.

1 Introduction

Large language models (LLMs), such as Chat-GPT (OpenAI, 2022), Claude (Anthropic, 2024), Deepseek (Guo et al., 2025), Qwen (Yang et al., 2024), and Llama (Dubey et al., 2024), have been rapidly advancing. The text generated by these models is becoming increasingly fluent and human-like, facilitating tasks such as question-answering and news reporting. However, this progress has also raised increasing concerns about the misuse of LLMs, including the spread of misinformation and academic dishonesty (Crothers et al., 2023). This highlights the need for automated detection of machine-generated text.

Machine-generated text detection is a binary classification task that determines whether a given text is authored by human or generated by LLMs. Early supervised methods (Guo et al., 2023) use RoBERTa (Yinhan et al., 2019) as an encoder to obtain token logits or representations for classification. Due to the growing capabilities of LLMs in semantic understanding, more recent approaches (Verma et al., 2023; Sarvazyan et al., 2024) have replaced RoBERTa with LLMs, and some methods (Chen et al., 2023; Wang et al., 2024a) directly use LLMs to generate "machine" or "human" labels for detection. (Guo et al., 2024) improves detection accuracy by conducting fine-grained classification of texts generated by different models and human-written texts. These methods perform exceptionally well in specific training

![Illustration of our motivation. Both LLMs and human are writing the text, "The Eiffel Tower is a wrought-iron lattice tower on the Champ de Mars in Paris, France." When reaching the word "Champ", they exhibit different writing strategies. Left: LLMs focus solely on the preceding range of tokens. Right: In contrast, human writing strategies may involve global planning, local writing, iterative revision, and bidirectional thinking. As a result, human do not only consider the preceding tokens but may also take into account words from the global, local, and bidirectional contexts.](page_1012_370_388_246.png)

Figure 1: Illustration of our motivation. Both LLMs and human are writing the text, "The Eiffel Tower is a wrought-iron lattice tower on the Champ de Mars in Paris, France." When reaching the word "Champ", they exhibit different writing strategies. Left: LLMs focus solely on the preceding range of tokens. Right: In contrast, human writing strategies may involve global planning, local writing, iterative revision, and bidirectional thinking. As a result, human do not only consider the preceding tokens but may also take into account words from the global, local, and bidirectional contexts.
domains but struggle in out-of-distribution scenarios. This issue arises because existing methods primarily capture domain-specific features, such as word frequency, syntax, semantics, and sentiments (Guo et al., 2023; Wang et al., 2023). As these features can vary across domains, detectors often struggle with misclassification when applied to texts from new domains. In contrast, capturing domain-independent features is an effective method for improving the generalization capability of machine-generated text detection.

There exists an inherent distinction between the writing strategies of LLMs and human, specifically in the range of tokens they attend to when generating a token, no matter the domain to which a text belongs. As illustrated in Figure 1, when generating the token "Champ", LLMs are restricted to attending only to the preceding range of tokens, due to their masked attention mechanism. In contrast, when human write the word "Champ", they may take into account diverse ranges of words, such as global, local, and bidirectional contexts, depending on their intentions.

We propose the M-RangeDetector method to enhance the generalization capability of machine-generated text detection by modeling diverse writing strategies as domain-independent features. For a given text, we first obtain token representations using a Proxy LLM and then apply a Multi-Range Attention module to extract diversified writing strategy features for each token. These features are concatenated and fed into a classifier to determine whether the text is machine-generated. Specifically, the module integrates four distinct attention masking strategies to constrain different ranges during the computation of attention scores. The global mask captures relationships across all positions in the input sequence, the band mask focuses on local relationships between tokens on both sides, the dilated mask restricts a token’s interaction with other tokens that are separated by a fixed interval, and the random mask randomly selects subsets of tokens from any position. The module can learn writing strategy features by calculating attention scores for each token within diverse specified ranges. This constraint ensures that the calculation of attention scores focuses on a range that closely aligns with the range a human would attend to during writing. Notably, our method does not require fine-tuning the Proxy LLM for token representation. Instead, it only requires training the Multi-Range Attention module and the classifier, which demonstrates that our approach is a resource-efficient solution for detecting machine-generated text. Our approach outperforms existing methods across multiple widely used datasets, including the Ghostbuster dataset, the M4 monolingual and multilingual datasets, and the OUTFOX dataset. Our contributions can be summarized as follows:

• In this paper, we introduce a domain-independent feature, namely the difference of writing strategy between LLMs and human, to improve the out-of-domain generalization capability of the detector.

• We introduce a Multi-Range Attention module that integrates four distinct attention masking strategies, including global, band, dilated, and random masking, to constrain the attention computation to different ranges, thereby capturing various writing strategies.

• The experimental results demonstrate that our method not only exhibits exceptional generalization ability in unseen domains but also performs well on newly emerging LLMs and multiple languages.

2 Related Work

LLM-Generated Text Mainstream LLMs, such as ChatGPT and Llama, primarily employ the autoregressive structure within the Transformer architecture (Vaswani, 2017) for continuous next-token prediction. This enables the generation of high-quality text that closely mimics human-like writing styles (Shlegeris et al., 2022; Mei et al., 2017).

Various Modified Attention Mechanisms In addition to the standard attention mechanism, several modified attention mechanisms have been proposed to improve model efficiency and performance. We introduce four attention mechanisms to control the range of token representations attending to other tokens. Global attention (Yu et al., 2018; Ronen et al., 2022) captures global contextual information by computing relationships between all positions in the input sequence. Band attention (Beltagy et al., 2020; Li et al., 2020) reduces the computational cost by focusing on local token relationships, making it suitable for tasks with strong local dependencies. Dilated Attention (Hoang et al., 2022) introduces flexibility by selecting tokens in a skipping manner, allowing the model to capture
Figure 2: The architecture of M-RangeDetector. Left: We utilize Llama or Qwen as a Proxy LLM to obtain token representations. Right: We integrate the standard attention mechanism with four distinct attention masking strategies into a Multi-Range Attention module, composed of global, band, dilated, and random attention masks. The module is designed to capture distinct writing strategy features.

long-range dependencies efficiently. Random attention (Cattaneo et al., 2020) randomly selects subsets of tokens from any position.

Machine-Generated Text Detection Existing methods for detecting machine-generated text can be classified into zero-shot methods and supervised methods. Zero-shot methods rely on statistical features such as entropy, and perplexity for machine-generated text detection. For instance, GPT-Zero (Tian, 2023) classifies text based on perplexity and burstiness. DetectGPT (Mitchell et al., 2023) demonstrates that text generated by LLMs tends to fall within the negative curvature region of the log-probability function. Binoculars (Hans et al., 2024) employs two LLMs to compute the log perplexity of the text, utilizing cross-perplexity to detect LLM-generated content. These methods are training-free and easy to use, but their effectiveness is relatively limited (Taguchi et al., 2024).

Supervised approaches, such as fine-tuning RoBERTa (Yinhan et al., 2019) with external classifiers, are widely used to distinguish human-written text from machine-generated text (Yinhan et al., 2019; Guo et al., 2023). T5-Sentinel (Chen et al., 2023) addresses text detection by leveraging T5’s (Ni et al., 2021) next-token prediction capabilities. DeTeCtive (Guo et al., 2024) introduces a multi-task auxiliary and multi-layer contrastive learning framework to learn writing styles from different models and human, enhancing generalization ability. Ghostbuster (Verma et al., 2023) incorporates multiple weaker language models (ranging from unigram models to LLMs) to capture token-level output probabilities and uses structured search for feature construction. LLMIIXTIC (Sarvazyan et al., 2024) combines token-level probabilities from four Llama (Touvron et al., 2023) models for classification. Recent studies (Verma et al., 2023; Sarvazyan et al., 2024) have increasingly explored leveraging LLMs for feature extraction instead of relying on traditional models like RoBERTa.

3 Method

In this section, we provide a comprehensive overview of our method. In Section 3.1, we define the task of detecting machine-generated text and outline the key steps for utilizing the Proxy LLM to obtain token representations. In Section 3.2, we introduce the Multi-Range Attentions module that integrates four distinct attention masking strategies to restrict different ranges during attention computation. The architecture of M-RangeDetector is given in Figure 2.

3.1 Preliminary

Task Formulation This paper addresses the task of detecting machine-generated text across diverse domains, languages, and generators. Given a query text \( T \) consisting of \( L \) words, \( T = \{ t_1, t_2, \ldots, t_L \} \), our objective is to determine whether the text is authored by human or generated by LLMs.

Proxy LLM A sequence \( T \) is fed into the Proxy LLM (e.g., Llama or Qwen). The model generates hidden states for each token at each layer. We focus on the hidden states from the final layer, denoted as \( H = \{ h_1, h_2, \ldots, h_n \} \), where \( n \) represents the number of tokens after tokenization. These hidden states serve as high-level representations of the tokens. The key computational steps and equations for the hidden state \( H \) are as follows:
First, since the Proxy LLM we use is an autoregressive model, we obtain the masked attention scores \( A_m \), formulated as follows:

\[
A_m (Q, K, V, M) = \operatorname{Softmax} \left( \frac{Q K^T}{\sqrt{d_k}} + f_m(M) \right) V,
\] (1)

where \( Q \), \( K \), and \( V \) denote the query, key, and value matrices, respectively, which are obtained through linear transformations. \( d_k \) represents the dimensionality of the key vectors. \( M \) is a masking matrix where all elements in the lower triangle are set to 1, while those in the upper triangle are 0. The function \( f_m \) operates on the elements of matrix \( M \), defined as follows:

\[
f_m(M_{ij}) = \begin{cases}
0, & \text{if } M_{ij} = 1 \\
-\infty, & \text{otherwise}
\end{cases}.
\] (2)

The \( H \) is recursively computed through a function \( f \) applied to the attention results at each layer:

\[
H^{(i)} = f \left( A_m^{(i)}, H^{(i-1)} \right), \quad i = 1, 2, \ldots, l,
\] (3)

where \( l \) represents the number of layers in the Proxy LLM, and \( A_m^{(i)} \) denotes the attention output computed at layer \( i \) as defined in Eq. 1. The initial hidden state is set as \( H^{(0)} = X \), corresponding to the input embeddings. The function \( f \) encompasses essential operations such as multi-head attention, layer normalization, and residual connections.

3.2 Multi-Range Attentions Module

Following the steps outlined in Section 3.1, we obtain token representations. However, a token’s representation is generated by the Proxy LLM using a masking module that relies solely on preceding tokens. This approach learns contextual information from only a single range of tokens. To address this limitation of the Proxy LLM, we introduce four different attention masks: Global Mask, Band Mask, Dilated Mask, and Random Mask, which are distinct from those used by the Proxy LLM. We integrate these masks with the standard attention mechanism to implement four distinct types of attention. As shown in Figure 2, these components together form the Multi-Range Attention module. The configurations of attention masks in the processing of \( H \) are detailed in Algorithm 1.

Specifically, the \( H \) from the final layer, obtained in Section 3.1, are fed into four attention modules in parallel, yielding four diversified text representations: \( r_1, r_2, r_3, r_4 \). The computation of \( r_i \) is formulated as follows for \( i = (1, 2, 3, 4) \):

\[
r_i = \operatorname{AvgPool} (A_m (Q, K, V, M_i)),
\] (4)

where \( Q \), \( K \), and \( V \) are obtained by applying a nonlinear transformation to \( H \), \( M_1, M_2, M_3, M_4 \) correspond to the Global Mask, Band Mask, Dilated Mask, and Random Mask. \( A_m \) represents the masked attention, as defined in Equation 1. AvgPool is applied to aggregate the representations of all \( n \) tokens into a single text representation.

Additionally, the \( H \) are transformed into a unified text representation via the AvgPool operation for subsequent processing, referred to as the raw text representation \( r_s \).

Finally, the four diversified text representations, along with the raw text representation, are concatenated to obtain the fused text representation: \( R = \{ r_s, r_1, r_2, r_3, r_4 \} \). The fused representation \( R \) is then fed into a classifier to determine whether the text is machine-generated. After backpropagation, the classification loss \( \mathcal{L}_f \) is computed. Meanwhile, to further differentiate the parameters of the four attention mechanisms in the Multi-Range Attention Module, each of \( r_1, r_2, r_3, r_4 \) is individually input into a pseudo classifier, whose output does not play a decisive role. After backpropagation, the four pseudo-classifiers yield the corresponding losses \( \mathcal{L}_1, \mathcal{L}_2, \mathcal{L}_3, \mathcal{L}_4 \). Due to the impact of individual classification losses, each attention module undergoes more significant gradient updates, leading to a more diverse semantic space for each diversified text representation. The final loss function is computed as:

\[
\mathcal{L} = \alpha_f \cdot \mathcal{L}_f + \sum_{i=1}^4 \alpha_i \cdot \mathcal{L}_i,
\] (5)

Algorithm 1 The configurations of Global, Band, Dilated, and Random attention masks during the processing of \( H \).

Require: Input token representations \( H \), mask type \( \mathcal{M} \in \{ \text{Global, Band, Dilated, Random} \} \), \( M \in \mathbb{R}^{n \times n} \).
1: Initialize mask matrix \( M \leftarrow \mathbf{0} \in \mathbb{R}^{n \times n} \)
2: if \( \mathcal{M} = \text{Global Mask} \) then
3:   Select global attention tokens \( G \subseteq \{1, 2, ..., n\} \), where \( |G| < n \)
4:     \( M_{i,j} \leftarrow 1 \) if \( i \in G \) or \( j \in G \)
5:     \( M_{i,j} \leftarrow 0 \) otherwise
6: else if \( \mathcal{M} = \text{Band Mask} \) then
7:   Define band width \( w \)
8:     \( M_{i,j} \leftarrow 1 \) if \( |i - j| \leq w \)
9: else if \( \mathcal{M} = \text{Dilated Mask} \) then
10:   Define dilation rate \( d \)
11:   \( M_{i,j} \leftarrow 1 \) if \( |i - j| \mod d = 0 \)
12: else if \( \mathcal{M} = \text{Random Mask} \) then
13:   Define sparsity ratio \( r \)
14:   For each row \( i \), randomly select \( rn \) indices and set \( M_{i,j} \leftarrow 1 \) for selected \( j \)
15: end if
16: return \( M \)
where cross-entropy loss is used for each loss, \( \alpha_f \) and \( \alpha_i \) (for \( i = 1, 2, 3, 4 \)) are hyperparameters. The hyperparameters \( \alpha_f \) and \( \alpha_i \) are selected based on validation performance and are discussed in detail in Appendix B.

Notably, our method does not require fine-tuning the Proxy LLM. Instead, it only necessitates training the Differentiated Attentions module and the classifiers, which involve a minimal number of parameters. This makes our approach resource-efficient for detecting machine-generated text.

4 Experiments

We first evaluate the effectiveness of our method in in-domain and out-of-domain detection scenarios with unknown domains. Next, we assess the model’s performance in detecting machine-generated text from unseen generators and in multilingual settings. Finally, we explore additional out-of-domain detection scenarios, including generating the same text with prompts different from those used in the training set, using three datasets containing texts written by non-native English speakers, and evaluate the robustness of our approach against paraphrase attacks, as well as its performance in the authorship attribution task.

4.1 Experimental Setup

4.1.1 Dataset

We utilize four widely recognized and challenging datasets to conduct comprehensive experiments.

Ghostbuster The Ghostbuster (Verma et al., 2023) dataset covers three domains and two models, providing a diverse range of detection scenarios. These include in-domain and out-of-domain settings, such as unknown domains, unseen generators, and varied prompts, as well as cases involving text written by non-native English speakers.

M4 The M4 (Wang et al., 2023) dataset consists of M4-monolingual and M4-multilingual, encompassing data from eight models, six domains, and nine languages. The OOD detection scenarios in M4-monolingual and M4-multilingual include cases involving unknown models and domains. Additionally, M4-multilingual also includes scenarios with unknown languages.

OUTFOX The machine-generated text in the testing data is paraphrased using OUTFOX (Koike et al., 2024) or DIPPER (Krishna et al., 2024).

TuringBench The TuringBench (Uchendu et al., 2021) dataset is designed for the authorship attribution task, which involves classifying texts into 20 distinct categories. Further details about these datasets can be found in Appendix A.

4.1.2 Evaluation metrics

In line with existing works, we employ Accuracy (Acc), the F1-score, and Average Recall (AvgRec) as our primary evaluation metrics. Accuracy is simple and effective for evaluating overall performance but fails to reflect minority class performance in imbalanced datasets. The F1-score considers both the precision and recall of the model, evaluating overall model performance by computing the harmonic mean of these two. AvgRec, the average recall for human-written (HumanRec) and machine-generated (MachineRec) text.

4.1.3 Baseline methods

We primarily compare our method with the Ghostbuster approach, which uses larger parameter LLMs for feature extraction, while we utilize smaller parameter LLMs. Additionally, we compare our method with DeTeCtive, which employs a multi-layer contrastive framework to learn the differences in writing styles between machines and human, whereas we focus on learning writing strategies. Furthermore, we perform a comprehensive comparison with several widely adopted methods.

Ghostbuster Ghostbuster (Verma et al., 2023) integrates multiple weaker language models, ranging from unigram models to Davinci with 17.5 billion parameters, to capture token-level probabilities as text features. It employs a structured search approach for feature construction and is the state-of-the-art model on the Ghostbuster dataset.

DeTeCtive DeTeCtive (Guo et al., 2024) employs multi-level contrastive learning to capture the differences in writing styles between machine-generated and human-written texts for AI-generated text detection. It is the state-of-the-art on both the M4 monolingual and multilingual datasets.

RoBERTa Directly fine-tuning RoBERTa (Yinhan et al., 2019) with an external classifier is a common approach for implementing binary classification tasks (Guo et al., 2023).

T5-Sentinel T5-Sentinel (Chen et al., 2023) addresses text detection by leveraging T5’s next-token prediction capabilities.
<table>
  <tr>
    <th rowspan="2">Model</th>
    <th colspan="3">In-Domain</th>
    <th colspan="4">Out-of-Domain</th>
  </tr>
  <tr>
    <th>All Domains</th>
    <th>News</th>
    <th>Creative Writing</th>
    <th>Student Essays</th>
    <th>News</th>
    <th>Creative Writing</th>
    <th>Student Essays</th>
    <th>Average</th>
  </tr>
  <tr>
    <td><i>Binoculars</i> (Hans et al., 2024)</td>
    <td>92.7</td>
    <td>97.4</td>
    <td>92.4</td>
    <td>87.9</td>
    <td>97.4</td>
    <td>92.4</td>
    <td>87.9</td>
    <td>92.6</td>
  </tr>
  <tr>
    <td><i>DetectGPT</i> (Mitchell et al., 2023)</td>
    <td>57.4</td>
    <td>56.6</td>
    <td>48.2</td>
    <td>67.3</td>
    <td>56.6</td>
    <td>48.2</td>
    <td>67.3</td>
    <td>57.4</td>
  </tr>
  <tr>
    <td><i>FastDetectGPT</i> (Bao et al., 2023)</td>
    <td>90.8</td>
    <td>92.5</td>
    <td>88.5</td>
    <td>91.2</td>
    <td>92.5</td>
    <td>88.5</td>
    <td>91.2</td>
    <td>90.7</td>
  </tr>
  <tr>
    <td><i>GPTZero</i> (Tian, 2023)</td>
    <td>93.1</td>
    <td>91.5</td>
    <td>93.1</td>
    <td>83.9</td>
    <td>91.5</td>
    <td>93.1</td>
    <td>83.9</td>
    <td>89.5</td>
  </tr>
  <tr>
    <td><i>RoBERTa</i> (Guo et al., 2023)</td>
    <td>98.1</td>
    <td>99.4</td>
    <td>97.6</td>
    <td>97.4</td>
    <td>88.3</td>
    <td>95.7</td>
    <td>71.4</td>
    <td>85.1</td>
  </tr>
  <tr>
    <td><i>T5 — Sentinel</i> (Chen et al., 2023)</td>
    <td>96.6</td>
    <td>97.8</td>
    <td>95.6</td>
    <td>96.2</td>
    <td>89.6</td>
    <td>95.6</td>
    <td>87.9</td>
    <td>91.0</td>
  </tr>
  <tr>
    <td><i>Ghostbuster</i> (Verma et al., 2023)</td>
    <td>99.0</td>
    <td>99.5</td>
    <td>98.4</td>
    <td>99.5</td>
    <td>97.9</td>
    <td>95.3</td>
    <td>97.7</td>
    <td>97.0</td>
  </tr>
  <tr>
    <td><i>M — RangeDetector (Ours)</i></td>
    <td><b>99.8</b></td>
    <td><b>100.0</b></td>
    <td><b>99.5</b></td>
    <td><b>100.0</b></td>
    <td><b>98.6</b></td>
    <td><b>99.0</b></td>
    <td><b>97.7</b></td>
    <td><b>98.4</b></td>
  </tr>
</table>

Table 1: The F1 scores in both In-Domain and Out-of-Domain scenarios on the Ghostbuster dataset. The Out-of-Domain condition refers to using two of the three domains (news, creative writing, or student essays) as the training set while the remaining domain is used as the test set. Other results are derived from the (Verma et al., 2023). The best number is highlighted in bold, while the second-best one is underlined. The results shown above are obtained under the zero-shot setting. In contrast, the following results are based on supervised methods.

DetectGPT DetectGPT (Mitchell et al., 2023) utilizes probability curvature to detect whether a text is generated by LLMs. It observes that text sampled from LLMs tends to occupy the region of negative curvature in the model’s logarithmic probability function.

Binoculars Binoculars (Hans et al., 2024) determines whether a text is machine-generated by computing the ratio between the raw perplexity and cross-perplexity, thus mitigating the influence of prompt words on perplexity and enhancing detection accuracy and robustness.

GPT-Zero GPT-Zero (Tian, 2023) is a commercial model that classifies text based on perplexity and burstiness.

4.1.4 Implementation details

In all experiments for our method, we freeze the Proxy LLM and only train the parameters of the Multi-Range Attention module and the classifier. It is worth noting that, with the exception of the experiments in Section 4.5, all other experiments are conducted using Llama-3 with a 1 billion (B) parameter size as the Proxy LLM. All experiments use the AdamW optimizer with a cosine annealing learning rate schedule. The peak learning rate is set to \(3 \times 10^{-5}\), with a linear warm-up of 2000 steps and weight decay set to \(1 \times 10^{-5}\). The maximum input token length is set to 512. We train for 20 epochs on a single NVIDIA A800 GPU, with a batch size of 64. For all comparison experiments, we either directly use the reported results from (Verma et al., 2023) and (Guo et al., 2024) or train and test using their open-source code with default settings, reporting the final results.

<table>
  <tr>
    <th>Method</th>
    <th colspan="2">M4-monolingual</th>
    <th colspan="2">M4-multilingual</th>
  </tr>
  <tr>
    <th></th>
    <th>AvgRec</th>
    <th>F1</th>
    <th>AvgRec</th>
    <th>F1</th>
  </tr>
  <tr>
    <td><i>Binoculars</i> (Hans et al., 2024)</td>
    <td>89.89</td>
    <td>89.89</td>
    <td>80.63</td>
    <td>82.43</td>
  </tr>
  <tr>
    <td><i>RoBERTa</i> (Guo et al., 2023)</td>
    <td>88.70</td>
    <td>88.44</td>
    <td>80.01</td>
    <td>84.44</td>
  </tr>
  <tr>
    <td><i>T5 — Sentinel</i> (Chen et al., 2023)</td>
    <td>84.01</td>
    <td>81.08</td>
    <td>76.21</td>
    <td>68.99</td>
  </tr>
  <tr>
    <td><i>DeTeCTive</i> (Guo et al., 2024)</td>
    <td><b>98.44</b></td>
    <td><b>98.38</b></td>
    <td><b>93.42</b></td>
    <td><b>93.05</b></td>
  </tr>
  <tr>
    <td><i>M — RangeDetector</i></td>
    <td><b>98.42</b></td>
    <td><b>98.41</b></td>
    <td><b>97.06</b></td>
    <td><b>96.98</b></td>
  </tr>
</table>

Table 2: Experimental results on M4-monolingual and M4-multilingual.

4.2 Main Results and Analysis

We conduct extensive experiments on the Ghostbuster dataset to assess the generalization ability of M-RangeDetector across unknown domains. As shown in Table 1, we compare performance in both In-Domain and Out-of-Domain detection scenarios. Our method achieves an F1 score of 99.8% (All domains) and 98.4% (Average), outperforming the previous best approach by 0.8% and 1.4%, respectively. We further validate the superiority of our method on the M4-Mono dataset, which covers a broader range of domains and generators. As shown in Table 2, our method achieves an F1 score of 98.41%, matching the highest result attained by DeTeCTive. This result further demonstrates the generalization ability of our approach in OOD scenarios, including both unknown domains and newly emerging LLMs. Additionally, compared to the DeTeCTive method, our method not only demonstrates strong performance in English but also achieves an F1 score of 96.98% on the M4-multilingual dataset, exceeding DeTeCTive by 3.93%. This result highlights the generalization capability of our approach in OOD scenarios, including unseen languages. Overall, the comprehensive experimental results confirm the generalization ability of our method in OOD detection scen-
<table>
  <tr>
    <th>Model</th>
    <th>Prompts (F1)</th>
    <th>Claude Langs (F1)</th>
    <th>TOEFL 11 (Acc.)</th>
    <th>TOEFL 91 (Acc.)</th>
  </tr>
  <tr>
    <td><i>Binoculars</i></td>
    <td>84.4</td>
    <td>64.0</td>
    <td>95.5</td>
    <td>100.0</td>
  </tr>
  <tr>
    <td><i>DetectGPT</i></td>
    <td>70.8</td>
    <td>64.2</td>
    <td>98.6</td>
    <td>100.0</td>
  </tr>
  <tr>
    <td><i>FastDetectGPT</i></td>
    <td>94.6</td>
    <td>84.0</td>
    <td>90.2</td>
    <td>90.9</td>
  </tr>
  <tr>
    <td><i>GPTZero</i></td>
    <td>96.1</td>
    <td>75.6</td>
    <td>99.2</td>
    <td>100.0</td>
  </tr>
  <tr>
    <td><i>RoBERTa</i></td>
    <td>97.4</td>
    <td>87.8</td>
    <td>98.6</td>
    <td>98.1</td>
  </tr>
  <tr>
    <td><i>T5 – Sentinel</i></td>
    <td>94.6</td>
    <td>84.1</td>
    <td>98.9</td>
    <td>99.6</td>
  </tr>
  <tr>
    <td><i>Ghostbuster</i></td>
    <td>99.5</td>
    <td>92.2</td>
    <td>95.5</td>
    <td>99.9</td>
  </tr>
  <tr>
    <td><i>M – RangeDetector</i></td>
    <td>99.7</td>
    <td>96.5</td>
    <td>99.9</td>
    <td>100.0</td>
  </tr>
</table>

Table 3: The additional generalization results include diverse prompting strategies, the unseen generator Claude, and three datasets containing texts written by non-native English speakers. All configurations remain identical to those in (Verma et al., 2023).

<table>
  <tr>
    <th>Attacker Detector</th>
    <th colspan="2">Non-attacked</th>
    <th colspan="2">DIPPER</th>
    <th colspan="2">OUTFOX</th>
  </tr>
  <tr>
    <th></th>
    <th>AvgRec</th>
    <th>F1</th>
    <th>AvgRec</th>
    <th>F1</th>
    <th>AvgRec</th>
    <th>F1</th>
  </tr>
  <tr>
    <td><i>Binoculars</i></td>
    <td>49.3</td>
    <td>33.0</td>
    <td>55.4</td>
    <td>45.2</td>
    <td>89.1</td>
    <td>89.0</td>
  </tr>
  <tr>
    <td><i>FastDetectGPT</i></td>
    <td>75.1</td>
    <td>74.6</td>
    <td>88.2</td>
    <td>88.2</td>
    <td>94.9</td>
    <td>94.9</td>
  </tr>
  <tr>
    <td><i>RoBERTa</i></td>
    <td>90.8</td>
    <td>90.7</td>
    <td>94.3</td>
    <td>94.4</td>
    <td>73.9</td>
    <td>68.3</td>
  </tr>
  <tr>
    <td><i>T5 – Sentinel</i></td>
    <td>99.0</td>
    <td>98.9</td>
    <td>96.1</td>
    <td>96.1</td>
    <td>94.8</td>
    <td>94.8</td>
  </tr>
  <tr>
    <td><i>OUTFOX</i></td>
    <td>96.5</td>
    <td>96.4</td>
    <td>82.4</td>
    <td>79.0</td>
    <td>61.8</td>
    <td>39.4</td>
  </tr>
  <tr>
    <td><i>DeTective</i></td>
    <td>99.1</td>
    <td>99.1</td>
    <td>97.7</td>
    <td>97.5</td>
    <td>97.0</td>
    <td>96.9</td>
  </tr>
  <tr>
    <td><i>M – RangeDetector</i></td>
    <td>99.4</td>
    <td>99.4</td>
    <td>99.2</td>
    <td>99.2</td>
    <td>99.1</td>
    <td>99.1</td>
  </tr>
</table>

Table 4: The results of our approach against paraphrase attacks, including DIPPER attack and OUTFOX attack on the OUTFOX dataset.

narios, including unknown domains, newly emerging LLMs, and unseen languages. We compare our method against all participating approaches in the SemEval competition\footnote{SemEval-2024, Task 8: https://www.codabench.org/competitions/1752}, which focuses on the M4 dataset, to thoroughly demonstrate the superior performance of our method on the M4 dataset. Detailed results are provided in Appendix C.

4.3 More comprehensive experiments

As shown in Table 3, we further conduct additional generalization experiments, evaluating model performance under diverse prompting strategies and assessing its ability to detect texts generated by the unseen generator, Claude. We further evaluate model accuracy on three datasets of texts written by non-native English speakers. In all cases, our method achieves state-of-the-art performance, demonstrating its strong generalization across machine-generated texts with diverse prompts and human-written texts from different linguistic backgrounds.

We evaluate the robustness of our approach against paraphrase attacks using the OUTFOX dataset. As shown in Table 4, in the non-attacked text detection scenario, our method achieves the highest accuracy among all approaches, with both AvgRec and F1 scores reaching 99.4%. Under

<table>
  <tr>
    <th>Method</th>
    <th>M4-monolingual</th>
    <th colspan="2"></th>
  </tr>
  <tr>
    <th></th>
    <th>AvgRec</th>
    <th>F1</th>
  </tr>
  <tr>
    <td><i>M – RangeDetector</i></td>
    <td>98.42</td>
    <td>98.41</td>
  </tr>
  <tr>
    <td>w/o All Attentions</td>
    <td>92.14</td>
    <td>92.20</td>
  </tr>
  <tr>
    <td>w/o AvgPool</td>
    <td>97.82</td>
    <td>97.85</td>
  </tr>
  <tr>
    <td>w/o Global Attention</td>
    <td>96.75</td>
    <td>96.83</td>
  </tr>
  <tr>
    <td>w/o Band Attention</td>
    <td>96.15</td>
    <td>96.24</td>
  </tr>
  <tr>
    <td>w/o Dilated Attention</td>
    <td>96.66</td>
    <td>96.74</td>
  </tr>
  <tr>
    <td>w/o Random Attention</td>
    <td>96.25</td>
    <td>96.28</td>
  </tr>
  <tr>
    <td>w/o Pseudo Classifier</td>
    <td>97.63</td>
    <td>97.68</td>
  </tr>
</table>

Table 5: Ablation studies for the Multi-Range Attention module and the Pseudo Classifiers.

DIPPER and OUTFOX attacks, our method maintains exceptional performance, achieving AvgRec and F1 scores of 99.2% and 99.1%, respectively. In contrast, other methods experience significant performance degradation under attack. These results demonstrate the superior robustness of our approach across various attack scenarios. We conduct additional experiments on the GenAI workshop dataset (Wang et al., 2025) to validate our method’s effectiveness in detecting text generated by recent LLMs. To evaluate whether **M-RangeDetector** has learned fine-grained features for distinguishing between generators, we conducted an authorship attribution experiment on the TuringBench dataset. Detailed results are provided in Appendix D.1 and D.2.

4.4 Ablation study

To systematically evaluate the contribution of each component in our method, we conduct a series of ablation studies on the M4-monolingual dataset, as summarized in Table 5. After removing AvgPool, the model achieves an F1 score of 97.85%, which is only 0.6% lower than the full model (M-RangeDetector). However, when we remove the multi-range attention module (w/o All Attentions), the performance drops by 6% compared to M-RangeDetector. This demonstrates that the multi-range attention module, which captures distinct writing strategies, plays a crucial role in enhancing the model’s generalization capability, rather than relying on the ProxyLLM hidden states used in existing methods. To further investigate the effectiveness of the various attention mechanisms within the module, we systematically remove Global Attention, Band Attention, Dilated Band Attention, and Random Attention individually. The observed performance degradation across all cases underscores the critical role of each attention mechanism within the module. Notably, after removing the pseudo-
classifier, we observe a noticeable decline in performance, suggesting that the pseudo-classifier plays a role in enhancing the diversity of the semantic space for each diversified text representation.

4.5 Impact of Proxy LLM Scale and Type
To investigate whether different architectures of the Proxy LLM significantly impact the performance of our method, we conduct comprehensive experiments on the Ghostbuster dataset in out-of-domain scenarios, assessing the average F1 score across three settings using the Llama-3 and Qwen-2.5 model series. The Llama-3 models feature parameter sizes of 1B, 3B, and 8B, while the Qwen-2.5 models comprise 1.5B, 7B, and 14B. As shown in Figure 3, the results indicate that the Proxy LLM, whether based on Llama or Qwen, surpasses the previous SOTA model, Ghostbuster, which utilizes davinci with 17.5 billion parameters. Furthermore, the minimal performance gap between Llama and Qwen indicates that architectural differences among LLMs have a negligible impact on the effectiveness of our method.

Additionally, we investigate whether scaling up LLMs improves detection performance. A horizontal comparison reveals that increasing model size has a negligible impact, indicating that model scale is not a critical factor in determining the effectiveness of our method.

4.6 Exploratory on Attention Score
We perform an analysis of the attention scores from different attention in the Multi-Range Attention module. We introduce a special token, \( T_s \), at the end of the original text and assign it global attention, enabling \( T_s \) to interact with all tokens (similarly to the [CLS] token in BERT (Devlin, 2018)). We only analyze the attention scores between \( T_s \) and other tokens. We define the breadth of attention, denoted as \( \theta \), to calculate the proportion of preceding tokens attended to when generating the last token. Specifically, the breadth of attention is the ratio of the number of tokens with attention scores greater than \( 3 \times 10^{-5} \) to \( T_s \) relative to the total number of tokens. A larger \( \theta \) value indicates that the model attends to a broader range of tokens. The value of \( \theta \) is computed as follows:

\[
\theta = \frac{\sum_{j=1}^{N} \mathbb{I}(A_{i,j} > 3 \times 10^{-5})}{N}
\]

where \( A_{i,j} \) is the attention score assigned by the model to token \( i \) when attending to token \( j \), token \( i \) is \( T_s \), \( N \) is the total number of tokens, \( \mathbb{I}(\cdot) \) is the indicator function, which equals 1 if the condition inside holds true and 0 otherwise. A random sample of 1000 instances is drawn from the Ghostbuster dataset for experimentation. Each sample is processed through the model and computed using Equation 6, yielding four \( \theta \) values. The results are then visualized using box plots. As shown in Figure 4, the within-group results indicate that, across all four diversified attentions, the \( \theta \) for human-written text is lower than that for machine-generated text. We hypothesize that this discrepancy arises because human writing tends to focus on a limited number of key tokens, while machine-generated text, due to the central role of the attention mechanism in LLMs, exhibits a broader attention pattern. In the between-group results, we observe that the four attention mechanisms yield distinct distributions of the \( \theta \) value within the same category. This suggests that the

Figure 3: The experimental results assessing the impact of Proxy LLM scale and type. We used LLaMA and Qwen models of varying sizes, evaluated across out-of-domain scenarios on the Ghostbuster dataset.

Figure 4: The figure compares \( \theta \) value distributions between human-written and machine-generated texts across four attention mechanisms, using box plots to highlight the differences.
four attention masks have learned different token representation patterns.

5 Conclusion

In this study, we propose the different writing strategies between LLMs and human as domain-independent features for enhancing generalization in machine-generated text detection. We introduce a novel approach called M-RangeDetector, which incorporates a Multi-Range Attention module to learn different writing strategies for classification. The module incorporates four distinct attention masking strategies to constrain different ranges during the computation of attention scores. Specifically, it employs global, band, dilated, and random masking strategies to capture the contextual representations of tokens across various token ranges, including global, local, and bidirectional ranges. Our method achieves state-of-the-art performance on three widely used benchmarks. Experimental results demonstrate that our approach not only exhibits exceptional generalization ability in unseen domains but also performs well on newly emerging LLMs and across multiple languages.

Limitations

Despite its effectiveness, M-RangeDetector has certain limitations. While contextual representations from tokens across diverse ranges differ significantly between human and LLMs, the results of the Authorship Attribution experiment indicate that although our method can reliably detect whether a text is machine-generated, it is not capable of accurately identifying the specific LLM that produced it. Furthermore, we have not conducted a detailed analysis of the specific contributions of different masking strategies to the model, nor have we investigated whether incorporating additional masking strategies can capture a broader range of writing strategies. In the future, we aim to develop more generalizable models for the Authorship Attribution task and further explore the impact of various attention masking strategies on the performance of our method. We will continue to advance research in machine-generated text detection tasks.

Acknowledgements

This work is supported by the National Natural Science Foundation of China No.62222212, 62232006 and 62376033.

References

Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.

Anthropic. 2024. Claude 3.5 sonnet.

Guangsheng Bao, Yanbin Zhao, Zhiyang Teng, Linyi Yang, and Yue Zhang. 2023. Fast-detectgpt: Efficient zero-shot detection of machine-generated text via conditional probability curvature. arXiv preprint arXiv:2310.05130.

Iz Beltagy, Matthew E Peters, and Arman Cohan. 2020. Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150.

Matias D Cattaneo, Xinwei Ma, Yusufcan Masatlioglu, and Elchin Suleymanov. 2020. A random attention model. Journal of Political Economy, 128(7):2796–2836.

Yutian Chen, Hao Kang, Vivian Zhai, Liangze Li, Rita Singh, and Bhiksha Raj. 2023. Gpt-sentinel: Distinguishing human and chatgpt generated content. arXiv preprint arXiv:2305.07969.

Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. 2024. Scaling instruction-finetuned language models. Journal of Machine Learning Research, 25(70):1–53.

Evan N Crothers, Nathalie Japkowicz, and Herna L Viktor. 2023. Machine-generated text: A comprehensive survey of threat models and detection methods. IEEE Access, 11:70977–71002.

Jacob Devlin. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.

Abhimanyu Dubey, Abhinav Jauhari, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783.

Angela Fan, Yacine Jernite, Ethan Perez, David Grangier, Jason Weston, and Michael Auli. 2019. Eli5: Long form question answering. arXiv preprint arXiv:1907.09190.

Biyang Guo, Xin Zhang, Ziyuan Wang, Minqi Jiang, Jinran Nie, Yuxuan Ding, Jianwei Yue, and Yupeng Wu. 2023. How close is chatgpt to human experts? comparison corpus, evaluation, and detection. arXiv preprint arXiv:2301.07597.

Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948.
Xun Guo, Shan Zhang, Yongxin He, Ting Zhang, Wanquan Feng, Haibin Huang, and Chongyang Ma. 2024. Detective: Detecting ai-generated text via multi-level contrastive learning. arXiv preprint arXiv:2410.20964.

Abhimanyu Hans, Avi Schwarzschild, Valeriia Cherepanova, Hamid Kazemi, Aniruddha Saha, Micah Goldblum, Jonas Geiping, and Tom Goldstein. 2024. Spotting llms with binoculars: Zero-shot detection of machine-generated text. arXiv preprint arXiv:2401.12070.

Dinh-Hieu Hoang, Gia-Han Diep, Minh-Triet Tran, and Ngan T H Le. 2022. Dam-al: Dilated attention mechanism with attention loss for 3d infant brain image segmentation. In Proceedings of the 37th ACM/SIGAPP Symposium on Applied Computing, pages 660–668.

Dongyeop Kang, Waleed Ammar, Bhavana Dalvi, Madeleine Van Zuylen, Sebastian Kohlmeier, Eduard Hovy, and Roy Schwartz. 2018. A dataset of peer reviews (peerread): Collection, insights and nlp applications. arXiv preprint arXiv:1804.09635.

Ryuto Koike, Masahiro Kaneko, and Naoaki Okazaki. 2024. Outfox: Llm-generated essay detection through in-context learning with adversarially generated examples. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 21258–21266.

Mahnaz Koupae and William Yang Wang. 2018. Wikihow: A large scale text summarization dataset. arXiv preprint arXiv:1810.09305.

Kalpesh Krishna, Yixiao Song, Marzena Karpinska, John Wieting, and Mohit Iyyer. 2024. Paraphrasing evades detectors of ai-generated text, but retrieval is an effective defense. Advances in Neural Information Processing Systems, 36.

Nicolas Küchler, Emanuel Opel, Hide Lycklama, Alexander Viand, and Anwar Hithnawi. 2024. Cohere: Managing differential privacy in large scale systems. In 2024 IEEE Symposium on Security and Privacy (SP), pages 991–1008. IEEE.

Jiaojiao Li, Ruxing Cui, Bo Li, Rui Song, Yunsong Li, Yuchao Dai, and Qian Du. 2020. Hyperspectral image super-resolution by band attention through adversarial learning. IEEE Transactions on Geoscience and Remote Sensing, 58(6):4304–4318.

Hongyuan Mei, Mohit Bansal, and Matthew R Walter. 2017. Coherent dialogue with attention-based language models. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, pages 3252–3258.

Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D Manning, and Chelsea Finn. 2023. Detectgpt: Zero-shot machine-generated text detection using probability curvature. In International Conference on Machine Learning, pages 24950–24962. PMLR.

Niklas Muenninghoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M Saiful Bari, Sheng Shen, Zheng-Xin Yong, Hailey Schoelkopf, et al. 2022. Crosslingual generalization through multitask finetuning. arXiv preprint arXiv:2211.01786.

Jianmo Ni, Gustavo Hernandez Abrego, Noah Constant, Ji Ma, Keith B Hall, Daniel Cer, and Yinfei Yang. 2021. Sentence-t5: Scalable sentence encoders from pre-trained text-to-text models. arXiv preprint arXiv:2108.08877.

OpenAI. 2022. Chatgpt: Optimizing language models for dialogue.

Alec Radford. 2018. Improving language understanding by generative pre-training.

Roi Ronen, Shahar Tsiper, Oron Anschel, Inbal Lavi, Amir Markovitz, and R Manmatha. 2022. Glass: Global to local attention for scene-text spotting. In European Conference on Computer Vision, pages 249–266. Springer.

Areg Mikael Sarvazyan, José-Ángel González, and Marc Franco-Salvador. 2024. Genaios at semeval-2024 task 8: Detecting machine-generated text by mixing language model probabilistic features. In Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024), pages 101–107.

Buck Shlegeris, Fabien Roger, Lawrence Chan, and Euan McLean. 2022. Language models are better than humans at next-token prediction. arXiv preprint arXiv:2212.11281.

Ayumi Taguchi, Daisuke Yoshimoto, Misako Kusakabe, Satoshi Baba, Akira Kawata, Yuichiro Miyamoto, Mayuyo Mori, Kenbun Sone, Yasushi Hirota, and Yutaka Osuga. 2024. Impact of human papillomavirus types on uterine cervical neoplasia. Journal of Obstetrics and Gynaecology Research, 50(8):1283–1288.

Edward Tian. 2023. Gptzero: An ai text detector.

Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288.

Adaku Uchendu, Zeyu Ma, Thai Le, Rui Zhang, and Dongwon Lee. 2021. Turingbench: A benchmark environment for turing test in the age of neural text generation. arXiv preprint arXiv:2109.13296.

A Vaswani. 2017. Attention is all you need. Advances in Neural Information Processing Systems.

Vivek Verma, Eve Fleisig, Nicholas Tomlin, and Dan Klein. 2023. Ghostbuster: Detecting text ghost-written by large language models. arXiv preprint arXiv:2305.15047.
Rongsheng Wang, Haoming Chen, Ruizhe Zhou, Han Ma, Yaofei Duan, Yanlan Kang, Songhua Yang, Baoyu Fan, and Tao Tan. 2024a. Llm-detector: Improving ai-generated chinese text detection with open-source llm instruction tuning. arXiv preprint arXiv:2402.01158.

Yuxia Wang, Jonibek Mansurov, Petar Ivanov, Jinyan Su, Artem Shelmanov, Akim Tsvigun, Osama Mohammed Afzal, Tarek Mahmoud, Giovanni Puccetti, Thomas Arnold, et al. 2024b. Semeval-2024 task 8: Multidomain, multimodel and multilingual machine-generated text detection. arXiv preprint arXiv:2404.14183.

Yuxia Wang, Jonibek Mansurov, Petar Ivanov, Jinyan Su, Artem Shelmanov, Akim Tsvigun, Chenxi Whitehouse, Osama Mohammed Afzal, Tarek Mahmoud, Toru Sasaki, et al. 2023. M4: Multi-generator, multi-domain, and multi-lingual black-box machine-generated text detection. arXiv preprint arXiv:2305.14902.

Yuxia Wang, Artem Shelmanov, Jonibek Mansurov, Akim Tsvigun, Vladislav Mikhailov, Rui Xing, Zhuohan Xie, Jahui Geng, Giovanni Puccetti, Ekaterina Artemova, et al. 2025. Genai content detection task 1: English and multilingual machine-generated text detection: Ai vs. human. arXiv preprint arXiv:2501.11012.

An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. 2024. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115.

Liu Yinhao, Ott Myle, Goyal Naman, Du Jingfei, Joshi Mandar, Chen Danqi, Levy Omer, and Lewis Mike. 2019. Roberta: A robustly optimized bert pretraining approach (2019). arXiv preprint arXiv:1907.11692, pages 1–13.

Adams Wei Yu, David Dohan, Minh-Thang Luong, Rui Zhao, Kai Chen, Mohammad Norouzi, and Quoc V Le. 2018. Qanet: Combining local convolution with global self-attention for reading comprehension. In International Conference on Learning Representations.

A Dataset Details

A.1 GhostBuster Dataset

The Ghostbuster dataset (Verma et al., 2023) serves as a benchmark for detecting AI-generated text across various domains. It spans three distinct domains: student essays, creative writing, and news articles, and includes two generative models, ChatGPT (OpenAI, 2022) and Claude (Anthropic, 2024). The dataset features five detection scenarios: In-Domain, Out-of-Domain, Prompts, Claude, and non-native English speakers. In the In-Domain scenario, data from all three domains (ChatGPT) are used for training and evaluated on a test set from the same distribution. News, Creative Writing, and Student Essays represent single-domain training and testing on the corresponding test set. In the Out-of-Domain scenario, two of the three domains (news, creative writing, or student essays) are used for training, with the remaining domain reserved for testing. The Avg reports the average results of the three individual out-of-domain experiments. The Prompts scenario tests text generated from prompts different from those used in training, essentially an out-of-domain detection challenge. The Claude scenario involves training on ChatGPT-generated texts and testing on Claude-generated texts to test the unseen model detection. Finally, the non-native English speakers category includes three datasets: Lang8, TOEFL-11, and TOEFL-91, which are written by non-native speakers, testing the ability to detect texts written by different linguistic backgrounds.

A.2 M4

The M4 (Wang et al., 2023) dataset is a large-scale, multi-domain, multi-model, and multilingual collection. It includes text from sources such as Wikipedia, WikiHow (Koupaee and Wang, 2018), Reddit (Fan et al., 2019), arXiv, and PeerRead (Kang et al., 2018). Using human-written prompts, models like ChatGPT (OpenAI, 2022), davinci-003 (Radford, 2018), Llama (Touvron et al., 2023), FLAN-T5 (Chung et al., 2024), Cohere (Küchler et al., 2024), Dolly-v2, and BLOOMz(Muennighoff et al., 2022) generate content in nine languages, including English, Chinese, and Russian. In the monolingual scenario, the test set features unseen domains and unseen AI-generated texts from GPT-4 (Achiam et al., 2023), which are further paraphrased by OUTFOX (Koike et al., 2024) to increase detection difficulty. In the multilingual scenario, the test set introduces novel languages not present in the training or validation sets, with AI-generated texts also paraphrased.

A.3 TuringBench

The dataset within TuringBench (Uchendu et al., 2021) comprises 200,000 samples across 20 distinct labels, including human-written texts and outputs from various AI text-generators such as GPT-1, GPT-2, GPT-3, GROVER, CTRL, XLM, XLNET, and others. The dataset was constructed by collecting 10,000 news articles, primarily in the domain of politics, from sources like CNN. These
articles, ranging from 200 to 400 words, were used to prompt AI text-generators to produce an additional 10,000 articles each, resulting in a total of 200,000 articles. The dataset supports two benchmark tasks: the Turing Test (TT), which is a binary classification problem to differentiate between human and machine-generated texts, and Authorship Attribution (AA), a multi-class classification problem aimed at identifying the specific neural language model that generated the texts.

B Hyperparameters Settings

We outline our hyperparameter settings for the weighted loss function. Our overall loss function is defined as:

\[
L = \alpha_f \cdot L_f + \sum_{i=1}^{4} \alpha_i \cdot L_i
\]

To ensure numerical stability and effective optimization, we first constrain the sum of all five hyperparameters to equal 1. This prevents any individual loss term from dominating excessively while maintaining a balanced gradient.

\[
\alpha_f + \sum_{i=1}^{4} \alpha_i = 1.
\]

Through empirical analysis, we found that \( L_f \) plays a more crucial role in optimization than \( L_i \). To reflect this importance, we set \( \alpha_f = 0.5 \) and evenly distributed the remaining weight among \( L_i \), assigning \( \alpha_i = 0.125 \) for \( i = 1, 2, 3, 4 \). Furthermore, to validate the stability of these choices, we conducted additional experiments by varying \( \alpha_f \) within the range of 0.3, with corresponding adjustments to \( \alpha_i \). Our results indicate that setting \( \alpha_f = 0.5 \pm 0.1, \alpha_i = 0.125 \pm 0.025 \) for \( i = 1, 2, 3, 4 \) yields higher detection performance. Based on these findings, we ultimately set \( \alpha_f = 0.5, \alpha_i = 0.125 \).

C SemEval Competition Results

As shown in Table 2 in Section 4.2, we compare our model with Detective on AvgRecall and F1 scores. The results demonstrate that our method achieves SOTA performance in both monolingual and multilingual tasks, with a 3.93% improvement over Detective in the multilingual task. To further highlight the capabilities of our approach, we also compared it with the results of all participants in the SemEval competition, where the final ranking was based on the accuracy (Acc) of the Machine-Generated Text detection task. As shown in Tables 6 and 7 (due to space limitations, we only present the results of the top six participants and the baseline, for detailed results, refer to (Wang et al., 2024b)), our method also achieves SOTA performance in terms of accuracy (Acc). In the monolingual task, our method outperforms the top-ranked model by 0.84%, and in the multilingual task, it surpasses the top model by 0.99%. These results further demonstrate that our method achieves SOTA performance among all existing methods.

<table>
  <tr>
    <th>Team</th>
    <th>Prec</th>
    <th>Recall</th>
    <th>F1-score</th>
    <th>Acc</th>
  </tr>
  <tr>
    <td>M-RangeDetector</td>
    <td><b>98.19</b></td>
    <td>98.63</td>
    <td><b>98.41</b></td>
    <td><b>98.42</b></td>
  </tr>
  <tr>
    <td>dianchi</td>
    <td>96.21</td>
    <td>99.19</td>
    <td>97.68</td>
    <td>97.53</td>
  </tr>
  <tr>
    <td>Genaios</td>
    <td>96.11</td>
    <td>98.03</td>
    <td>97.06</td>
    <td>96.88</td>
  </tr>
  <tr>
    <td>USTC-BUPT</td>
    <td>95.75</td>
    <td>96.86</td>
    <td>96.30</td>
    <td>96.10</td>
  </tr>
  <tr>
    <td>mail6dijj</td>
    <td>94.87</td>
    <td>97.18</td>
    <td>96.02</td>
    <td>95.76</td>
  </tr>
  <tr>
    <td>howudoin</td>
    <td>93.48</td>
    <td>98.12</td>
    <td>95.74</td>
    <td>95.42</td>
  </tr>
  <tr>
    <td>idontknow</td>
    <td>94.57</td>
    <td>95.42</td>
    <td>94.99</td>
    <td>94.72</td>
  </tr>
  <tr>
    <td>baseline</td>
    <td>93.36</td>
    <td>84.02</td>
    <td>88.44</td>
    <td>88.47</td>
  </tr>
</table>

Table 6: The results of Monolingual

<table>
  <tr>
    <th>Team</th>
    <th>Prec</th>
    <th>Recall</th>
    <th>F1-score</th>
    <th>Acc</th>
  </tr>
  <tr>
    <td>M-RangeDetector</td>
    <td><b>99.10</b></td>
    <td>95.05</td>
    <td><b>97.03</b></td>
    <td><b>96.98</b></td>
  </tr>
  <tr>
    <td>USTC-BUPT</td>
    <td>94.93</td>
    <td>97.53</td>
    <td>96.21</td>
    <td>95.99</td>
  </tr>
  <tr>
    <td>FI Group</td>
    <td>94.28</td>
    <td>98.00</td>
    <td>96.10</td>
    <td>95.85</td>
  </tr>
  <tr>
    <td>KInIT</td>
    <td>92.95</td>
    <td>97.86</td>
    <td>95.34</td>
    <td>95.00</td>
  </tr>
  <tr>
    <td>priyansk</td>
    <td>90.70</td>
    <td>98.14</td>
    <td>94.28</td>
    <td>93.77</td>
  </tr>
  <tr>
    <td>L3i++</td>
    <td>92.47</td>
    <td>94.00</td>
    <td>93.23</td>
    <td>92.87</td>
  </tr>
  <tr>
    <td>QUST</td>
    <td>90.45</td>
    <td>90.98</td>
    <td>90.71</td>
    <td>90.27</td>
  </tr>
  <tr>
    <td>baseline</td>
    <td>73.45</td>
    <td>99.30</td>
    <td>84.44</td>
    <td>80.89</td>
  </tr>
</table>

Table 7: The results of Multilingual.

<table>
  <tr>
    <th>model</th>
    <th>Macro-F1</th>
    <th>Acc</th>
  </tr>
  <tr>
    <td>Binoculars</td>
    <td>76.51</td>
    <td>76.76</td>
  </tr>
  <tr>
    <td>RoBERTa</td>
    <td>70.34</td>
    <td>72.56</td>
  </tr>
  <tr>
    <td>T5-Sentinel</td>
    <td>69.20</td>
    <td>71.12</td>
  </tr>
  <tr>
    <td><b>M-RangeDetector</b></td>
    <td><b>86.10</b></td>
    <td><b>86.34</b></td>
  </tr>
</table>

Table 8: Performance comparison of our approach and baseline methods on the GenAI workshop dataset.

<table>
  <tr>
    <th>Rank</th>
    <th>Team</th>
    <th>Macro-F1</th>
    <th>Acc</th>
  </tr>
  <tr>
    <td>1</td>
    <td>Advacheck</td>
    <td>83.07</td>
    <td>83.11</td>
  </tr>
  <tr>
    <td>2</td>
    <td>Unibuc-NLP</td>
    <td>83.01</td>
    <td>83.33</td>
  </tr>
  <tr>
    <td>3</td>
    <td>Fraunhofer SIT</td>
    <td>82.80</td>
    <td>82.89</td>
  </tr>
  <tr>
    <td>-</td>
    <td><b>Baseline</b></td>
    <td>73.42</td>
    <td>74.89</td>
  </tr>
</table>

Table 9: Partial leaderboard results for the English subtask in the GenAI workshop.
<table>
  <tr>
    <th>Method</th>
    <th>Precision</th>
    <th>Accuracy</th>
    <th>Recall</th>
    <th>F1</th>
  </tr>
  <tr>
    <td>Random Forest</td>
    <td>58.93</td>
    <td>61.47</td>
    <td>60.53</td>
    <td>58.47</td>
  </tr>
  <tr>
    <td>SVM (3-grams)</td>
    <td>71.24</td>
    <td>72.99</td>
    <td>72.23</td>
    <td>71.49</td>
  </tr>
  <tr>
    <td>WriteprintsRFC</td>
    <td>45.78</td>
    <td>49.43</td>
    <td>48.51</td>
    <td>46.51</td>
  </tr>
  <tr>
    <td>Syntax-CNN</td>
    <td>65.20</td>
    <td>66.13</td>
    <td>65.44</td>
    <td>64.80</td>
  </tr>
  <tr>
    <td>N-gram CNN</td>
    <td>69.09</td>
    <td>69.14</td>
    <td>68.32</td>
    <td>66.65</td>
  </tr>
  <tr>
    <td>N-gram LSTM</td>
    <td>66.94</td>
    <td>68.98</td>
    <td>68.24</td>
    <td>66.46</td>
  </tr>
  <tr>
    <td>OpenAI Detector</td>
    <td>78.10</td>
    <td>78.73</td>
    <td>78.12</td>
    <td>77.41</td>
  </tr>
  <tr>
    <td>BertAA</td>
    <td>77.96</td>
    <td>78.12</td>
    <td>77.50</td>
    <td>77.58</td>
  </tr>
  <tr>
    <td>BERT-Multinomial</td>
    <td>80.31</td>
    <td>80.78</td>
    <td>80.21</td>
    <td>79.96</td>
  </tr>
  <tr>
    <td>RoBERTa-Multinomial</td>
    <td>82.14</td>
    <td>81.73</td>
    <td>81.26</td>
    <td>81.07</td>
  </tr>
  <tr>
    <td>DeTeCtive</td>
    <td>84.04</td>
    <td>82.75</td>
    <td>82.59</td>
    <td>83.05</td>
  </tr>
  <tr>
    <td>Ours</td>
    <td><b>85.39</b></td>
    <td><b>86.08</b></td>
    <td><b>85.77</b></td>
    <td><b>85.41</b></td>
  </tr>
</table>

Table 10: The results of Authorship Attribution task on TuringBench dataset.

![Confusion Matrix for Authorship Attribution on the TuringBench Dataset.](page_324_670_708_312.png)

Figure 5: Confusion Matrix for Authorship Attribution on the TuringBench Dataset.

D Additional Experiments Results

D.1 GenAI Workshop Dataset

As shown in Table 8, our method achieves a Macro-F1 score of 86.10% and an Accuracy of 86.34% on the GenAI workshop dataset, outperforming other baseline methods (Best) by 10%. Additionally, as shown in Table 9, our model outperforms all teams in the GenAI workshop, ranking 1st on the English Subtask leaderboard and surpassing the top-ranked model by 3%. These results underscore the robustness of our method in detecting text from newer LLMs, maintaining strong and consistent performance even on the latest datasets.

D.2 The Authorship Attribution Task

The author attribution task requires not only determining whether a text is written by human or generated by LLMs, but also further identifying which generator generated the text. This constitutes a multi-class classification problem. To further evaluate the effectiveness of our method in the author attribution task, we conducted comprehensive experiments on the TuringBench dataset. The results, shown in Figure 10, demonstrate that our approach outperforms other methods, SOTA performance across multiple metrics, including Precision (84.56%), Accuracy (84.18%), Recall (83.67%), and F1 score (83.23%). However, the confusion matrix results in Figure 5 indicate that our method can distinguish whether a text is machine-generated. Nonetheless, there are still significant misclassifications within models from the same family (e.g., GPT). This is because while human and machine writing strategies differ greatly, writing strategies among machines can be quite similar, making it challenging for our method to accurately identify specific LLMs. In future work, we will further investigate the potential of our approach in author attribution tasks.

Reviewer paper summaries: The paper introduces M-RangeDetector, a new approach for detecting machine-generated text by focusing on how writing strategies differ between humans and LLMs. Existing detectors often struggle to generalize to unseen text domains, because they mainly rely on surface-level features. This method solves it by capturing the potential difference that humans tend to write using a mix of context, while LLMs rely only on the preceding context. Authors use frozen Proxy LLM to get the output hidden states and train on them specific Multi-Range Attention module that uses four types of attention masks (global, band, dilated, and random) to better mimic the way humans write. This strategy helps the detector generalize more effectively across domains, languages, and even unseen LLMs, and it consistently outperforms existing methods on several benchmarks.
Summary of strengths: - **(S1)** Novelty: the paper introduces a creative and well-motivated approach to model human-like writing strategies, addressing a core limitation in current detection methods. This method could indeed be used to push further the development of AI-written text detection.
- **(S2)** Comprehensive empirical evaluation: extensive experiments on multiple benchmark datasets (Ghostbuster, M4, OUTFOX, TuringBench) demonstrate good performance, including robustness to paraphrase attacks and cross-lingual transfer.
- **(S3)** Ablation and Analysis: the paper provides necessary ablation studies and attention score analysis, which clearly shows that Multi-Range Attention module contributes significantly to the final score 
- **(S4)** Organization and Clarity: the paper is well-structured and clearly written, which makes it very accessible for the reader. Almost all details are provided
Summary of weaknesses: - **(W1)** Lack of Clarity on Alpha Hyperparameters: the paper introduces a weighted loss function with hyperparameters alpha, but authors, as I believe, do not specify the values used. A discussion and analysis on the effect of these parameters would significantly improve reproducibility and offer insight into the method's stability.
- **(W2)** Insufficient evidence supporting the writing strategy hypothesis: the central claim of the paper is that M-Range Attention captures human-like writing strategies better than existing methods and so it helps to classify AI-written text more robustly. However, there is little concrete inspection or case study evidence to support this. It would be valuable to include qualitative examples where the multi-range features clearly aid detection, as well as examples where they fail, and show that this is indeed the writing strategy. Other possible way to inspect it is to inspect how the pseudo-classifiers work or ablate the avg pooling from the raw hidden states from proxy LLM.
- **(W3)** Outdated model coverage in datasets: while the authors evaluate on widely used datasets, these benchmarks rely on relatively older model generations. Given the fast development of LLM development, incorporating more recent and challenging datasets such as RAID [1] or the GenAI workshop [2] dataset would better demonstrate the method's robustness and relevance in current detection scenarios.

[1] Liam Dugan, Alyssa Hwang, Filip Trhlík, Andrew Zhu, Josh Magnus Ludan, Hainiu Xu, Daphne Ippolito, and Chris Callison-Burch. 2024. RAID: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectors. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12463–12492, Bangkok, Thailand. Association for Computational Linguistics.

[2] Yuxia Wang, Artem Shelmanov, Jonibek Mansurov, Akim Tsvigun, Vladislav Mikhailov, Rui Xing, Zhuohan Xie, Jiahui Geng, Giovanni Puccetti, Ekaterina Artemova, Jinyan Su, Minh Ngoc Ta, Mervat Abassy, Kareem Ashraf Elozeiri, Saad El Dine Ahmed El Etter, Maiya Goloburda, Tarek Mahmoud, Raj Vardhan Tomar, Nurkhan Laiyk, Osama Mohammed Afzal, Ryuto Koike, Masahiro Kaneko, Alham Fikri Aji, Nizar Habash, Iryna Gurevych, and Preslav Nakov. 2025. GenAI Content Detection Task 1: English and Multilingual Machine-Generated Text Detection: AI vs. Human. In Proceedings of the 1stWorkshop on GenAI Content Detection (GenAIDetect), pages 244–261, Abu Dhabi, UAE. International Conference on Computational Linguistics
Comments, suggestions and typos: I appreciate the novelty of the approach and overall find this work interesting and promising. I believe, addressing the outlined weaknesses (especially in terms of evaluation with more up-to-date models and alpha parameters) would significantly strengthen the paper and its impact.
Overall assessment scores: 4
Metareview score: 3.5
Metareview: The paper proposes a Multi-Range Attention mechanism for detecting machine-generated text by modeling different “writing strategies.” Reviewers generally agree that:

Strengths:
Novel approach that captures how humans may use varied context (global, local, etc.) differently than LLMs;
Strong empirical results on multiple datasets (Ghostbuster, M4, etc.), including multilingual and paraphrase-attack scenarios;
Ablation studies demonstrate the effectiveness of the proposed multi-range attention.

Weaknesses:
Some reviewers felt the discussion of prior work and in-domain features was too narrow;
More thorough justification and clarity were requested for attention-score experiments and hyperparameter choices (especially 𝛼 α in the loss);

Comparisons to a few newer baselines (e.g., DNA-GPT) were missing or limited.

The authors provided additional experiments (e.g., partial AUC, new datasets like GenAI) and clarifications that addressed many concerns. Yet, minor reservations remain regarding clarity and completeness of baseline coverage. Overall, I believe the method is promising with a borderline but positive recommendation.

Final Recommendation: 3.5 (Borderline Accept).
Summary of reasons to publish: The paper tackles the challenge of detecting machine-generated text by introducing a multi-range attention mechanism that models how humans vary their focus on context. Its strong experimental results on multiple datasets (including multilingual and paraphrase-attack scenarios) indicate robust generalization. Ablations convincingly show the contribution of each masking strategy, and the overall method yields improvements over established detectors. The contribution is novel, and the authors’ additional experiments (e.g., partial AUC, GenAI dataset) solidify the claim that the proposed approach is effective across domains and models.