Who Writes What: Unveiling the Impact of Author Roles on AI-generated Text Detection
Authors: Jiatao Li, Xiaojun Wan
Accepted at: acl25

Who Writes What: Unveiling the Impact of Author Roles on AI-generated Text Detection

Jiatao Li1,2, Xiaojun Wan1
1Wangxuan Institute of Computer Technology, Peking University
2Information Management Department, Peking University
leejames@stu.pku.edu.cn, wanxiaojun@pku.edu.cn

Abstract
The rise of Large Language Models (LLMs) necessitates accurate AI-generated text detection. However, current approaches largely overlook the influence of author characteristics. We investigate how sociolinguistic attributes—gender, CEFR proficiency, academic field, and language environment—impact state-of-the-art AI text detectors. Using the ICNALE corpus of human-authored texts and parallel AI-generated texts from diverse LLMs, we conduct a rigorous evaluation employing multi-factor ANOVA and weighted least squares (WLS). Our results reveal significant biases: CEFR proficiency and language environment consistently affected detector accuracy, while gender and academic field showed detector-dependent effects. These findings highlight the crucial need for socially aware AI text detection to avoid unfairly penalizing specific demographic groups. We offer novel empirical evidence, a robust statistical framework, and actionable insights for developing more equitable and reliable detection systems in real-world, out-of-domain contexts. This work paves the way for future research on bias mitigation, inclusive evaluation benchmarks, and socially responsible LLM detectors.

1 Introduction
Large Language Models (LLMs) have transformed the way we produce and consume written communication, driving remarkable advancements in text generation across a wide array of domains. From social media content moderation to automated report writing in academia, the breadth and sophistication of these models have brought renewed focus to a pressing challenge: detecting AI-generated text. Although recent efforts have expanded detection benchmarks to include multiple LLMs, multilingual settings, and adversarial perturbations (Tao et al., 2024; Dugan et al., 2024; Li et al., 2024; Wang et al., 2024), the human dimension of text production has received far less scrutiny. Traditional detection pipelines predominantly center on model-level and data-centric strategies—e.g., sampling, prompting, and adversarial augmentation—while overlooking social and linguistic factors tied to the authors themselves.

Yet decades of sociolinguistic and applied linguistics research underscore that writing style varies systematically by gender, language proficiency, disciplinary norms, and cultural context (Hyland, 2000; Lantolf, 2000; Long, 1996). Such differences manifest in vocabulary choices, syntactic complexity, and rhetorical conventions, which can inadvertently bias AI text detectors if not carefully addressed. For example, non-native speakers or lower-proficiency writers might be wrongly flagged as AI-generated due to errors or unidiomatic expressions that differ from native-level training data. Conversely, sophisticated second-language learners might produce polished texts that resemble those generated by larger LLMs, thereby complicating detection efforts. In the absence of benchmarks that account for such diversity, existing detectors risk disproportionately penalizing particular demographic or educational groups.

Motivated by this gap, we pose the question: “Who is writing?” Specifically, we investigate whether and how author-level attributes—including gender, Common European Framework of Reference (CEFR) proficiency, academic field, and language environment (e.g., native vs. EFL/ESL)—systematically influence AI (or AI-generated) text detection outcomes. To this end, we craft a comprehensive end-to-end analysis. First, we source human-authored texts with rich metadata from the ICNALE corpus, a well-established learner corpus that spans multiple Asian regions and diverse proficiency levels (Ishikawa, 2013). Next, we generate parallel AI-written texts using a range of modern LLMs (e.g., Qwen, LLaMA, Mistral). We then evaluate cutting-edge off-the-shelf detectors un-
der out-of-domain conditions—a scenario closely mirroring real-world deployments where detectors must generalize beyond their training distributions. Figure 1 offers a concise overview of our end-to-end methodology, culminating in an out-of-domain evaluation that mirrors real-world use cases.

Our findings, drawn from rigorous t-tests and ANOVA analyses, show that CEFR proficiency level and language environment exert consistent and sizable effects on detection accuracy across nearly all detectors. By contrast, gender- and academic field-related biases tend to be more detector-dependent, with some models demonstrating marked variability and others remaining largely agnostic. These results suggest a strong need to incorporate demographic and linguistic attributes into AI text detection benchmarks and training protocols. Without doing so, state-of-the-art detectors are liable to unfairly penalize certain writer populations or misclassify legitimate texts.

In summary, our work highlights the human-centered challenges that arise when AI text detection collides with real-world diversity. By shifting the focus toward the people behind the text, we aim to catalyze a broader conversation on socially responsible model development, enhanced debiasing strategies, and inclusive evaluation frameworks. We hope that this renewed emphasis will pave the way for fairer, more robust, and ultimately more trustworthy detection systems.

Our contributions are as follows ¹:

• We introduce a 67K-text dataset that combines human-authored ICNALE essays with parallel AI outputs from 12 modern LLMs, each labeled with detailed persona metadata. This resource enables rigorous, human-centered analysis in AI text detection.

• We propose a multi-factor WLS framework with Type II ANOVA, controlling for multiple attributes to isolate each factor’s influence on detection errors and pinpoint where bias emerges. This approach can also be extended to future bias analysis tasks that consider additional demographic or linguistic variables.

• We demonstrate that human-level attributes—particularly language proficiency and environment—can overshadow model-based differences, causing significant biases in AI text detection.

2 Related Work

Author Characteristics and Linguistic Theory
Linguistic research demonstrates how author characteristics influence writing style. Functional linguistics and discourse analysis show that registers and genres vary in vocabulary, syntax, and pragmatics (Biber et al., 1998), creating distinctions between AI- and human-generated text. Sociolinguistics and Second Language Acquisition (SLA) research reveals textual differences (e.g., lexical diversity, grammatical complexity) between native and non-native speakers (Hyland, 2000; Lantolf, 2000; Long, 1996). Hyland (2000) specifically highlights how disciplinary norms in academic writing create stylistic variations. "World Englishes" theory (Kachru et al., 1985; Jenkins, 2003) further emphasizes the diverse forms of English usage

¹Our code and data are released at https://github.com/leejamesss/AuthorAwareDetection to facilitate related research.

![Research workflow: ICNALE data and LLM-generated text with author attributes are used for out-of-domain evaluation of AI text detectors and subsequent bias analysis using multi-factor ANOVA and weighted OLS.](page_120_120_1247_312.png)

Figure 1: Research workflow: ICNALE data and LLM-generated text with author attributes are used for out-of-domain evaluation of AI text detectors and subsequent bias analysis using multi-factor ANOVA and weighted OLS.
across cultures. This linguistic diversity complicates AI text detection, potentially introducing bias if detectors are trained on limited data and fail to generalize across varied social and cultural backgrounds.

AI Text Detection Datasets and Benchmarks
As LLMs continue to advance in various NLP tasks, the need to distinguish AI-generated text from human-authored text has become a hot research topic. Early work explored supervised learning or feature engineering, but as models scale, attention has shifted to multi-scenario, multi-model, and multilingual detection benchmarks.

Examples include CUDRT, which focuses on polished and post-processed LLM text (Tao et al., 2024); RAID, a comprehensive benchmark with multiple models, domains, languages, and adversarial samples (Dugan et al., 2024); MAGE, which systematically combines models and domains for in- and out-of-domain evaluations (Li et al., 2024); M4GT-Bench, for multilingual, black-box detection (Wang et al., 2024); and HC3, which compares ChatGPT and human expert outputs (Guo et al., 2023a).

Despite this progress in creating diverse detection benchmarks, the human dimension—specifically, the demographic and identity attributes of authors—has received limited attention. The influence of factors like gender, native language, academic discipline, and language proficiency on detection accuracy and potential bias remains largely unexplored. This study addresses this gap by analyzing how these key author attributes affect AI text detection.

3 Dataset Creation

3.1 Overview
We construct our dataset from 2,569 learners in the International Corpus Network of Asian Learners of English (ICNALE) (Ishikawa, 2013), each providing short essays on two prompts: banning smoking in restaurants and the importance of part-time jobs. This yields 5,138 human-authored texts. ICNALE also includes rich demographic and identity data (e.g., sex, academic genre, proficiency), as shown in Table 2.

To capture a broader range of writing styles, we pair each human-authored text with parallel responses from 12 modern LLMs. Each model thus produces another 5,138 essays, adding up to 61,656 AI-generated texts. Overall, our corpus contains 66,794 samples. We acquired the ICNALE data under its official license, following all guidelines. The consistent task prompts and detailed metadata enable controlled investigations of how AI models handle varied writer profiles.

Table 1 compares our dataset to other publicly available resources of AI-generated text. While many incorporate multiple models, none (to our knowledge) matches our depth of persona annotations. Our corpus thus serves as a valuable benchmark for research on model fairness and sociolinguistic bias. Detailed ICNALE task prompts appear in Appendix H.3.

<table>
  <tr>
    <th>Name</th>
    <th>Size</th>
    <th>MultiModel</th>
    <th>Persona</th>
  </tr>
  <tr>
    <td>TuringBench (Uchenda et al., 2021)</td>
    <td>200k</td>
    <td>✓</td>
    <td>✗</td>
  </tr>
  <tr>
    <td>RuATD (Shamardina et al., 2022)</td>
    <td>215k</td>
    <td>✓</td>
    <td>✗</td>
  </tr>
  <tr>
    <td>HC3 (Guo et al., 2023b)</td>
    <td>26.9k</td>
    <td>✗</td>
    <td>✗</td>
  </tr>
  <tr>
    <td>MGTBench (He et al., 2023)</td>
    <td>2817</td>
    <td>✓</td>
    <td>✗</td>
  </tr>
  <tr>
    <td>MULTItude (Macko et al., 2023)</td>
    <td>74.1k</td>
    <td>✓</td>
    <td>✗</td>
  </tr>
  <tr>
    <td>AuText2023 (Sarvazyan et al., 2023)</td>
    <td>160k</td>
    <td>✗</td>
    <td>✗</td>
  </tr>
  <tr>
    <td>M4 (Wang et al., 2023b)</td>
    <td>122k</td>
    <td>✓</td>
    <td>✗</td>
  </tr>
  <tr>
    <td>CCD (Wang et al., 2023a)</td>
    <td>467k</td>
    <td>✗</td>
    <td>✗</td>
  </tr>
  <tr>
    <td>IMDGSP (Mosca et al., 2023)</td>
    <td>29k</td>
    <td>✓</td>
    <td>✗</td>
  </tr>
  <tr>
    <td>HC-Var (Xu et al., 2023)</td>
    <td>145k</td>
    <td>✗</td>
    <td>✗</td>
  </tr>
  <tr>
    <td>HC3 Plus (Su et al., 2024)</td>
    <td>210k</td>
    <td>✗</td>
    <td>✗</td>
  </tr>
  <tr>
    <td>MAGE (Li et al., 2024)</td>
    <td>447k</td>
    <td>✓</td>
    <td>✗</td>
  </tr>
  <tr>
    <td>RAID (Dugan et al., 2024)</td>
    <td>6.2M</td>
    <td>✓</td>
    <td>✗</td>
  </tr>
  <tr>
    <th>Ours</th>
    <th>67k</th>
    <th>✓</th>
    <th>✓</th>
  </tr>
</table>

Table 1: Comparison of publicly available AI-generated text datasets. MultiModel indicates whether each benchmark includes texts from multiple LLMs (✓) or only one (✗). Persona indicates whether the dataset provides demographic or identity metadata.

3.2 Author Attribute Construction

In selecting author attributes, we consider both data feasibility and theoretical grounding. Practically, we choose attributes with minimal missing values, relatively balanced distributions, and manageable category sizes. Theoretically, we prioritize attributes that are well-studied in sociolinguistics, second-language acquisition (SLA), and academic discourse, so they can reveal deeper insights into potential biases. Accordingly, we focus on four key dimensions:

Gender (Sex). As indicated by Cameron (2005); Gal (2012), gender is shaped by societal and cultural constructions, potentially giving rise to “gendered” linguistic features. Table 2 shows that 1,430 learners self-identified as female (\( F \)) and 1,139 as male (\( M \)), for a total of 2,569.
<table>
  <tr>
    <th>Attribute</th>
    <th>Value</th>
    <th>Count</th>
  </tr>
  <tr>
    <td rowspan="5">CEFR</td>
    <td>B1_1</td>
    <td>914</td>
  </tr>
  <tr>
    <td>B1_2</td>
    <td>881</td>
  </tr>
  <tr>
    <td>A2_0</td>
    <td>470</td>
  </tr>
  <tr>
    <td>B2_0</td>
    <td>231</td>
  </tr>
  <tr>
    <td>XX_0</td>
    <td>73</td>
  </tr>
  <tr>
    <td rowspan="4">Acad. Genre</td>
    <td>Sciences & Tech.</td>
    <td>1,034</td>
  </tr>
  <tr>
    <td>Social Sciences</td>
    <td>762</td>
  </tr>
  <tr>
    <td>Humanities</td>
    <td>674</td>
  </tr>
  <tr>
    <td>Life Sciences</td>
    <td>99</td>
  </tr>
  <tr>
    <td rowspan="3">Lang. Env.</td>
    <td>EFL</td>
    <td>1,886</td>
  </tr>
  <tr>
    <td>ESL</td>
    <td>610</td>
  </tr>
  <tr>
    <td>NS</td>
    <td>73</td>
  </tr>
  <tr>
    <td rowspan="2">Sex</td>
    <td>F</td>
    <td>1,430</td>
  </tr>
  <tr>
    <td>M</td>
    <td>1,139</td>
  </tr>
</table>

Table 2: Distribution of author attributes across 2,569 learners.

Academic Field (Acad. Genre). Rooted in genre theory (Swales, 2014) and Bourdieu’s concept of academic capital (Bourdieu, 1991), different disciplines (e.g., Humanities, Social Sciences) may exhibit distinct rhetorical styles (Hyland, 2000). Our dataset includes four such academic fields, with Sciences & Technology (1,034) being the largest group (Table 2).

CEFR Proficiency Level (CEFR). The Common European Framework of Reference for Languages (of Europe. Council for Cultural Cooperation. Education Committee. Modern Languages Division, 2001) provides a graded scale for language proficiency (A2, B1, B2, etc.), widely used in SLA research (Krashen, 2006; Long, 1996; Lantolf, 2000). As shown in Table 2, the most common levels in our dataset are B1_1 (914) and B1_2 (881), with smaller counts at A2_0 (470), B2_0 (231), and XX_0 (73; native speakers).

Language Environment (NS/EFL/ESL). Following Kachru’s notion of “World Englishes” (Kachru et al., 1985) and Jenkins’ work on English as a lingua franca (Jenkins, 2003), we distinguish between native speakers (NS, 73 learners), EFL (English as a Foreign Language, 1,886 learners), and ESL (English as a Second Language, 610 learners). Such differences in exposure and context can shape writing styles that AI text detectors may treat unevenly.

These four attributes are both available at scale (with minimal missing data) and theoretically grounded in prior linguistic research. By focusing on them, we can systematically examine how AI detectors handle diverse writer profiles and detect potential biases. Table 2 shows the distribution of each category in our dataset.

Definition of Subgroup. For our analysis, we define a subgroup based on the values of a categorical author attribute. Let \( A \) represent a categorical attribute (e.g., CEFR level, Language Environment, Academic Genre, Sex). This attribute \( A \) can take on a set of \( k \) possible values, denoted as \( \{A_1, \ldots, A_k\} \). Each \( A_i \) (where \( i \in \{1, 2, \ldots, k\} \)) represents a specific value of the attribute \( A \) and defines a distinct subgroup. For example, if \( A \) is CEFR level, then \( A_1 \) might be A2_0, \( A_2 \) might be B1_1, and so on. All texts associated with a particular attribute value \( A_i \) constitute that subgroup.

3.3 Generators

We employ a diverse set of Large Language Models (LLMs) to generate texts, spanning different architectures, training sets, and parameter sizes (0.5B–72B). Our lineup includes the Qwen2.5 family (Yang et al., 2024; Team, 2024) from Alibaba DAMO Academy, recognized for strong instruction-following performance and extended-context capabilities; the LLaMA3.1 and LLaMA3.2 models (Dubey et al., 2024), noted for their open-source availability and efficient pre-training; and the compact yet high-performing Mistral (Jiang et al., 2023).

Table 3 lists the specific model versions used. Each model was prompted with carefully designed instructions to emulate learners with diverse gender, academic fields, CEFR proficiency levels, and language environments (see Appendix H). By covering a wide range of parameter scales and training paradigms, our approach seeks to capture heterogeneous AI-generated outputs and enable us to evaluate detection robustness across various author attributes.

4 Detector

4.1 Detector Selection

Building upon the methodology of Dugan et al. (2024) (RAID), we evaluate detectors from two categories: classifier-based, metric-based. Classifier-based detectors typically involve fine-tuning a pretrained language model such as RoBERTa (Liu et al., 2019), while metric-based detectors compute a value based on generative model probabilities.

Specifically, following RAID’s unified pipeline
<table>
  <tr>
    <th>LLM Name</th>
    <th>Series</th>
    <th>Params</th>
  </tr>
  <tr>
    <td>Qwen2.5-0.5B-Instruct</td>
    <td>Qwen2.5</td>
    <td>0.5B</td>
  </tr>
  <tr>
    <td>Qwen2.5-1.5B-Instruct</td>
    <td>Qwen2.5</td>
    <td>1.5B</td>
  </tr>
  <tr>
    <td>Qwen2.5-3B-Instruct</td>
    <td>Qwen2.5</td>
    <td>3B</td>
  </tr>
  <tr>
    <td>Qwen2.5-7B-Instruct</td>
    <td>Qwen2.5</td>
    <td>7B</td>
  </tr>
  <tr>
    <td>Qwen2.5-14B-Instruct</td>
    <td>Qwen2.5</td>
    <td>14B</td>
  </tr>
  <tr>
    <td>Qwen2.5-32B-Instruct</td>
    <td>Qwen2.5</td>
    <td>32B</td>
  </tr>
  <tr>
    <td>Qwen2.5-72B-Instruct</td>
    <td>Qwen2.5</td>
    <td>72B</td>
  </tr>
  <tr>
    <td>Llama3.1-8b-instruct</td>
    <td>Llama3.1</td>
    <td>8B</td>
  </tr>
  <tr>
    <td>Llama3.1-70b-instruct</td>
    <td>Llama3.1</td>
    <td>70B</td>
  </tr>
  <tr>
    <td>Llama3.2-1b-instruct</td>
    <td>Llama3.2</td>
    <td>1B</td>
  </tr>
  <tr>
    <td>Llama3.2-3b-instruct</td>
    <td>Llama3.2</td>
    <td>3B</td>
  </tr>
  <tr>
    <td>Mistral-Small-Instruct-2409</td>
    <td>Mistral</td>
    <td>22B</td>
  </tr>
</table>

Table 3: List of employed LLMs

and detector set, we tested the detectors summarized in Table 4.

<table>
  <tr>
    <th>Category</th>
    <th>Detectors</th>
  </tr>
  <tr>
    <th rowspan="4">Classifier-Based</th>
    <td>RoBERTa-Base (GPT2) (<a href="https://arxiv.org/abs/1906.08237">Solaiman et al., 2019</a>)</td>
  </tr>
  <tr>
    <td>RoBERTa-Large (GPT2) (<a href="https://arxiv.org/abs/1906.08237">Liu et al., 2019</a>)</td>
  </tr>
  <tr>
    <td>RoBERTa-Base (ChatGPT) (<a href="https://arxiv.org/abs/2303.07422">Guo et al., 2023b</a>)</td>
  </tr>
  <tr>
    <td>RADAR (<a href="https://arxiv.org/abs/2303.07422">Hu et al., 2023</a>)</td>
  </tr>
  <tr>
    <th rowspan="5">Metric-Based</th>
    <td>GLTR (<a href="https://arxiv.org/abs/1906.08237">Gehrmann et al., 2019</a>)</td>
  </tr>
  <tr>
    <td>Binoculars (<a href="https://arxiv.org/abs/2401.07226">Hans et al., 2024</a>)</td>
  </tr>
  <tr>
    <td>Fast DetectGPT (<a href="https://arxiv.org/abs/2303.07422">Bao et al., 2023</a>)</td>
  </tr>
  <tr>
    <td>DetectGPT (<a href="https://arxiv.org/abs/2303.07422">Mitchell et al., 2023</a>)</td>
  </tr>
  <tr>
    <td>LLMDet (<a href="https://arxiv.org/abs/2303.07422">Wu et al., 2023</a>)</td>
  </tr>
</table>

Table 4: Overview of Detectors Evaluated

In contrast to Li et al. (2024), we adopt RAID’s principle of analyzing off-the-shelf models without further fine-tuning them on our target dataset (Dugan et al., 2024). This setup directly assesses the generalization capability of these pre-trained detectors. Meanwhile, for the metric-based detectors, we utilize the default generative models within each repository to replicate realistic usage scenarios.2

4.2 Detector Evaluation

In keeping with RAID’s evaluation paradigm, each detector produces a scalar score for a given sequence, and we transform this score into a binary AI/human judgment by setting a threshold \( \tau \). We then calibrate \( \tau \) so that the false positive rate (FPR) against human-authored text remains at 5%. Accuracy at a fixed 5% FPR thus measures how effectively each detector identifies machine-generated text without unfairly penalizing human writers. This aligns with recent trends in robustness evaluations (Hans et al., 2024; Krishna et al., 2023; Soto et al., 2024).3

<table>
  <tr>
    <th></th>
    <th>\( \tau=0.25 \)</th>
    <th>\( \tau=0.5 \)</th>
    <th>\( \tau=0.75 \)</th>
    <th>\( \tau=0.95 \)</th>
  </tr>
  <tr>
    <td>R-B GPT2</td>
    <td>8.71%</td>
    <td>6.59%</td>
    <td>5.18%</td>
    <td>3.38%</td>
  </tr>
  <tr>
    <td>R-L GPT2</td>
    <td>6.14%</td>
    <td>2.91%</td>
    <td>1.46%</td>
    <td>0.25%</td>
  </tr>
  <tr>
    <td>R-B CGPT</td>
    <td>21.6%</td>
    <td>15.8%</td>
    <td>15.1%</td>
    <td>10.4%</td>
  </tr>
  <tr>
    <td>RADAR</td>
    <td>7.48%</td>
    <td>3.48%</td>
    <td>2.17%</td>
    <td>1.23%</td>
  </tr>
  <tr>
    <td>GLTR</td>
    <td>100%</td>
    <td>99.3%</td>
    <td>21.0%</td>
    <td>0.05%</td>
  </tr>
  <tr>
    <td>F-DetectGPT</td>
    <td>47.3%</td>
    <td>23.2%</td>
    <td>13.1%</td>
    <td>1.70%</td>
  </tr>
  <tr>
    <td>LLMDet</td>
    <td>97.9%</td>
    <td>96.0%</td>
    <td>92.0%</td>
    <td>75.3%</td>
  </tr>
  <tr>
    <td>Binoculars</td>
    <td>0.07%</td>
    <td>0.00%</td>
    <td>0.00%</td>
    <td>0.00%</td>
  </tr>
</table>

Table 5: False Positive Rates for detectors on RAID at naive choices of threshold (\( \tau \)). We see that, for open-source detectors, thresholding naively results in unacceptably high false positive rates.

3Dugan et al. (2024) discusses detailed rationale for choosing FPR-based calibration over traditional precision/recall/F1 metrics.

5 Bias Analysis

This section illustrates our approach for identifying and quantifying bias in AI text detection with respect to different author attributes (e.g., gender, CEFR level, academic field, and language environment). Our primary goal is to examine whether certain subgroups experience systematically higher or lower detection accuracy, even after controlling for other factors. Below, we introduce the core principles, define bias mathematically, and discuss why our multi-factor method is both appropriate and advantageous over simpler alternatives. Detailed algorithmic steps appear in Appendix C.

5.1 Task Definition

Input-Output Principle. Our pipeline takes as input texts labeled as human-authored or LLM-generated (aggregating outputs from all LLMs). Each text includes metadata describing the author’s (or LLM emulated author’s) demographic and linguistic attributes. Each detector provides a binary classification (human or AI). We then group
all texts (human and LLM-generated) by these attributes (e.g., CEFR level) to compute and compare detection accuracy across subgroups.

Definition of Bias. We define bias in terms of statistically significant differences in detection accuracy across subgroups. Let \( A \) represent a categorical author attribute (e.g., CEFR level) with values \( \{A_1, \ldots, A_k\} \), and let Accuracy\((A = a)\) be the detection accuracy for texts where \( A = a \). A detector is considered biased with respect to \( A \) if our multi-factor analysis reveals a significant difference in accuracy between any two attribute values \( a_i \) and \( a_j \).

We quantify this bias using a two-stage process: 1) ANOVA \( p \)-values: We first use Type II ANOVA to determine whether attribute \( A \) has a statistically significant overall effect on detection accuracy, while controlling for other attributes. A \( p \)-value below 0.05 indicates that the attribute significantly impacts detector performance.

2) Post-hoc Comparisons using LSMeans: If (and only if) the ANOVA \( p \)-value for attribute \( A \) is significant, we proceed to post-hoc pairwise comparisons. These comparisons use Least-Squares Means (LSMeans), which are adjusted means for each level of \( A \), holding other factors constant. We employ pairwise Wald tests with Holm correction on the LSMeans to reveal the magnitude and direction of significant differences between specific subgroups. This pinpoints which subgroups are disproportionately affected by the detector.

5.2 Multi-Factor Weighted Least Squares and ANOVA.

We employ a multi-factor regression framework to isolate each attribute’s effect on detection accuracy, using ordinary least squares (OLS) adapted to unbalanced data through weighted least squares (WLS). We then perform Type II ANOVA to determine each factor’s unique contribution once other attributes are held constant (Scheffé, 1999). This approach is well-suited for the uneven subgroup sizes common in sociolinguistic and fairness research (Hardt et al., 2016), as WLS estimates how accuracy varies with each factor while Type II ANOVA tests whether including a particular attribute (e.g., language environment) significantly reduces unexplained variance. By including all attributes jointly, we disentangle their individual effects and correct for subgroup size imbalances.

In contrast to single-factor tests—which examine each attribute in isolation—our multi-factor WLS+ANOVA approach accounts for overlapping or correlated attributes. For instance, CEFR level and language environment may be intertwined if certain regions tend to have higher proficiency, and ignoring these dependencies risks spurious conclusions or masking real biases (Angwin et al., 2016). Although rule-based or single-factor methods can provide initial insights (e.g., focusing on one attribute at a time), they cannot robustly address the interactions among multiple variables (Solon, 2017). Our multi-factor framework systematically partitions the variance in detection accuracy explained by each attribute, giving a more reliable measure of bias and enabling fairer AI text detection systems.

High-Level Implementation. Algorithm 1 summarizes our multi-factor WLS and Type II ANOVA procedure. Broadly, we (1) aggregate accuracy and assign weights for each unique combination of attribute values; (2) fit a WLS model, treating each attribute as a factor; and (3) apply Type II ANOVA by iteratively removing each attribute to assess its unique contribution to the model.

Outputs and Usage. The WLS and ANOVA procedure yields (i) ANOVA tables with each attribute’s \( F \)-statistic, \( p \)-value, and significance, and (ii) LSMeans, providing model-adjusted accuracy for each attribute combination. By accounting for correlated attributes and weighting groups by size, we detect and interpret biases without discarding data or ignoring interactions. Appendix A provides full implementation details, including pseudocode (Algorithm 2) and our Python statsmodels setup.

References to Additional Analyses. While our primary approach is the multi-factor WLS, we also conduct complementary tests for confirmation or exploratory checks. First, we perform single-factor analyses, such as Welch’s t-test or one-way (weighted) ANOVA, when focusing on a single categorical attribute in isolation. Second, we explore data subsetting, in which we match certain attributes or filter the data to reduce confounding (often shrinking the sample size). Finally, we implement down-sampling to balance group sizes, potentially discarding data from larger subgroups (see Appendix C). Although these approaches can yield valuable insights, they are less comprehensive in controlling for multiple attributes simultaneously, reinforcing the importance of multi-factor
Algorithm 1 Multi-Factor WLS + Type II ANOVA
Input: Dataset of n texts, each with a binary classification \( y_i \) (0: human, 1: AI) and a set of attribute values (e.g., CEFR=B1, Sex=F, ...).
Output: ANOVA table (F-statistics, p-values) and LSMeans.
1: Data Preparation: Group texts by unique combinations of attribute values. Calculate the mean accuracy \( a_j \) and weight \( w_j \) (text count) for each group \( j \). Treat each attribute as a factor.
2: Fit WLS Model: Estimate parameters \( \beta \) using weighted least squares, weighting each group’s contribution by \( w_j \).
3: Perform Type II ANOVA: For each attribute:
   • Construct a reduced model by removing the attribute.
   • Refit the WLS model to the reduced model, obtaining RSS_{reduced}.
   • Compute the \( F \)-statistic and \( p \)-value by comparing RSS_{reduced} to the full model’s RSS_{full}.
4: Calculate LSMeans: Compute LSMeans for each attribute level using the full WLS model.
5: ANOVA table and LSMeans.

methodologies as the central tool in our bias assessment.

5.3 Hypothesis Testing Results
To quantify how each demographic or contextual factor influences detector performance, we conduct a two-stage bias analysis. First, we use multi-factor Weighted Least Squares (WLS) regression and Type II ANOVA to determine which author attributes (factors) significantly influence detector accuracy. Second, for those attributes found to be significant in the ANOVA, we conduct post-hoc pairwise comparisons using Least-Squares Means (LSMeans) to identify which specific levels of the attribute exhibit significant differences in accuracy.

ANOVA Results. Table 6 presents the ANOVA results, with \( p \)-values for each factor (CEFR, Sex, Academic Genre, Language Environment) and detector. A \( p \)-value below 0.05 signifies a statistically significant effect of the factor on accuracy, controlling for other factors. **CEFR proficiency level** (cefr) is highly significant for all detectors, indicating a strong, consistent bias related to language proficiency. **Language environment** (language_env) is significant for most detectors, suggesting the context of English learning (EFL, ESL, or NS) matters. **Sex (Sex)** shows no significant effect, implying no evidence of gender-based bias. **Academic genre** (academic_genre) exhibits detector-dependent effects, with about half the detectors showing significant differences across fields.

<table>
  <tr>
    <th>Detector</th>
    <th>CEFR</th>
    <th>Sex</th>
    <th>Academic Genre</th>
    <th>Language Env.</th>
  </tr>
  <tr>
    <td>binoculars</td>
    <td><b>Yes</b></td>
    <td>No</td>
    <td>No</td>
    <td><b>Yes</b></td>
  </tr>
  <tr>
    <td>chatgpt-roberta</td>
    <td><b>Yes</b></td>
    <td>No</td>
    <td>No</td>
    <td>No</td>
  </tr>
  <tr>
    <td>detectgpt</td>
    <td><b>Yes</b></td>
    <td>No</td>
    <td><b>Yes</b></td>
    <td><b>Yes</b></td>
  </tr>
  <tr>
    <td>fastdetectgpt</td>
    <td><b>Yes</b></td>
    <td>No</td>
    <td>No</td>
    <td><b>Yes</b></td>
  </tr>
  <tr>
    <td>fastdetectllm</td>
    <td><b>Yes</b></td>
    <td>No</td>
    <td><b>Yes</b></td>
    <td><b>Yes</b></td>
  </tr>
  <tr>
    <td>gltr</td>
    <td><b>Yes</b></td>
    <td>No</td>
    <td>No</td>
    <td><b>Yes</b></td>
  </tr>
  <tr>
    <td>gpt2-base</td>
    <td><b>Yes</b></td>
    <td>No</td>
    <td><b>Yes</b></td>
    <td><b>Yes</b></td>
  </tr>
  <tr>
    <td>gpt2-large</td>
    <td><b>Yes</b></td>
    <td>No</td>
    <td><b>Yes</b></td>
    <td><b>Yes</b></td>
  </tr>
  <tr>
    <td>llmdet</td>
    <td><b>Yes</b></td>
    <td>No</td>
    <td><b>Yes</b></td>
    <td><b>Yes</b></td>
  </tr>
  <tr>
    <td>radar</td>
    <td><b>Yes</b></td>
    <td>No</td>
    <td>No</td>
    <td><b>Yes</b></td>
  </tr>
</table>

Table 6: ANOVA \( p \)-values for each factor across detectors. Columns show significance (Sig.?). “Yes” indicates \( p < 0.05 \) (significant), “No” otherwise. Green-shaded cells indicate significant results (\( p < 0.05 \)).

Post-Hoc Comparisons and LSMeans. For factors with significant overall effects in the ANOVA (Table 6), we conduct post-hoc pairwise comparisons using the Wald test with Holm correction. These tests, performed on the LSMeans, identify which specific levels of a factor differ significantly in detection accuracy. The LSMeans (Table 7) represent the model-adjusted mean accuracy for each subgroup, controlling for all other factors. In other words, they estimate the average detection accuracy for a particular subgroup (e.g., CEFR level B1), assuming all other attributes (Sex, Academic Genre, Language Environment) are held constant at their average values across the dataset. This allows us to isolate the effect of the factor of interest. A “--” in Table 7 indicates a non-significant ANOVA result for that factor and detector, precluding meaningful post-hoc tests. Table 10 presents the significant comparisons. To interpret the direction of significant differences (i.e., which subgroup has higher accuracy), the LSMeans values in Table 7 must be compared. Table 10 reveals the following key patterns:

**CEFR Level**: ‘binoculars’ shows the most extensive bias; all non-native CEFR levels differ significantly from native speakers (XX_0), with
<table>
  <tr>
    <th>Factor</th>
    <th>Level</th>
    <th>binoc.</th>
    <th>chatgpt-r</th>
    <th>detectgpt</th>
    <th>fdgpt</th>
    <th>fdllm</th>
    <th>gltr</th>
    <th>gpt2-base</th>
    <th>gpt2-large</th>
    <th>llmdet</th>
    <th>radar</th>
  </tr>
  <tr>
    <td rowspan="5">CEFR</td>
    <td>A2_0</td>
    <td>0.9482</td>
    <td>0.7480</td>
    <td>0.7944</td>
    <td>0.8963</td>
    <td>0.4873</td>
    <td>0.8366</td>
    <td>0.6493</td>
    <td>0.5402</td>
    <td>0.5063</td>
    <td>0.6886</td>
  </tr>
  <tr>
    <td>B1_1</td>
    <td>0.9443</td>
    <td>0.7408</td>
    <td>0.8072</td>
    <td>0.8887</td>
    <td>0.4842</td>
    <td>0.8305</td>
    <td>0.6399</td>
    <td>0.5676</td>
    <td>0.4961</td>
    <td>0.7267</td>
  </tr>
  <tr>
    <td>B1_2</td>
    <td>0.9475</td>
    <td>0.7377</td>
    <td>0.8228</td>
    <td>0.9023</td>
    <td>0.4786</td>
    <td>0.8206</td>
    <td>0.5881</td>
    <td>0.5410</td>
    <td>0.4812</td>
    <td>0.7210</td>
  </tr>
  <tr>
    <td>B2_0</td>
    <td>0.9507</td>
    <td>0.7045</td>
    <td>0.8289</td>
    <td>0.9143</td>
    <td>0.4828</td>
    <td>0.7967</td>
    <td>0.5623</td>
    <td>0.5462</td>
    <td>0.4581</td>
    <td>0.7020</td>
  </tr>
  <tr>
    <td>XX_0</td>
    <td>0.8981</td>
    <td>0.7410</td>
    <td>0.7722</td>
    <td>0.8748</td>
    <td>0.4975</td>
    <td>0.7180</td>
    <td>0.5482</td>
    <td>0.5162</td>
    <td>0.5037</td>
    <td>0.6868</td>
  </tr>
  <tr>
    <td rowspan="3">LangEnv</td>
    <td>EFL</td>
    <td>0.9482</td>
    <td>--</td>
    <td>0.7944</td>
    <td>0.8963</td>
    <td>0.4873</td>
    <td>0.8366</td>
    <td>0.6493</td>
    <td>0.5402</td>
    <td>0.5063</td>
    <td>0.6886</td>
  </tr>
  <tr>
    <td>ESL</td>
    <td>0.9337</td>
    <td>--</td>
    <td>0.7867</td>
    <td>0.8941</td>
    <td>0.4837</td>
    <td>0.8346</td>
    <td>0.6504</td>
    <td>0.5521</td>
    <td>0.5250</td>
    <td>0.6671</td>
  </tr>
  <tr>
    <td>NS</td>
    <td>0.8981</td>
    <td>--</td>
    <td>0.7722</td>
    <td>0.8748</td>
    <td>0.4975</td>
    <td>0.7180</td>
    <td>0.5482</td>
    <td>0.5162</td>
    <td>0.5037</td>
    <td>0.6868</td>
  </tr>
  <tr>
    <td rowspan="4">AcadGenre</td>
    <td>Humanities</td>
    <td>--</td>
    <td>--</td>
    <td>0.7944</td>
    <td>--</td>
    <td>0.4873</td>
    <td>--</td>
    <td>0.6493</td>
    <td>0.5402</td>
    <td>0.5063</td>
    <td>--</td>
  </tr>
  <tr>
    <td>Social Sciences</td>
    <td>--</td>
    <td>--</td>
    <td>0.7884</td>
    <td>--</td>
    <td>0.4856</td>
    <td>--</td>
    <td>0.6409</td>
    <td>0.5238</td>
    <td>0.5097</td>
    <td>--</td>
  </tr>
  <tr>
    <td>Sciences & Tech</td>
    <td>--</td>
    <td>--</td>
    <td>0.7759</td>
    <td>--</td>
    <td>0.4813</td>
    <td>--</td>
    <td>0.6866</td>
    <td>0.5705</td>
    <td>0.5109</td>
    <td>--</td>
  </tr>
  <tr>
    <td>Life Sciences</td>
    <td>--</td>
    <td>--</td>
    <td>0.7547</td>
    <td>--</td>
    <td>0.4732</td>
    <td>--</td>
    <td>0.6741</td>
    <td>0.5484</td>
    <td>0.5265</td>
    <td>--</td>
  </tr>
</table>

Table 7: LSMeans for CEFR Level, Language Environment (EFL, ESL, NS), and Academic Genre (Humanities, Social Sciences, Sciences & Technology, Life Sciences), across detectors. A “--” indicates that the overall ANOVA for that factor and detector was not statistically significant (see Table 6), so LSMeans are not reported.

<table>
  <tr>
    <th>Factor</th>
    <th>Detector</th>
    <th>Comparison</th>
    <th>Sig.?</th>
  </tr>
  <tr>
    <td rowspan="4">CEFR</td>
    <td rowspan="3">binoc.</td>
    <td>B2_0 vs XX_0</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td>A2_0 vs XX_0</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td>B1_2 vs XX_0</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td rowspan="2">chatgpt-r</td>
    <td>A2_0 vs B2_0</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td>B1_1 vs B2_0</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td rowspan="3">detectgpt</td>
    <td>B2_0 vs A2_0</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td>B2_0 vs XX_0</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td rowspan="3">Lang. Env.</td>
    <td>EFL vs ESL</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td>EFL vs NS</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td>ESL vs NS</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td rowspan="3">detectgpt</td>
    <td>EFL vs NS</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td rowspan="3">detectgpt</td>
    <td>Hum. vs S.&T.</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td>Hum. vs Life S.</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td>Soc. S. vs Life S.</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td rowspan="2">Acad. Genre</td>
    <td rowspan="2">gpt2-base</td>
    <td>S.&T. vs Hum.</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td>S.&T. vs Soc. S.</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td rowspan="2">gpt2-large</td>
    <td>S.&T. vs Hum.</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td>S.&T. vs Soc. S.</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td rowspan="3">llmdet</td>
    <td>Life S. vs S.&T.</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td>Life S. vs Soc. S.</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td>Life S. vs Hum.</td>
    <td>Yes</td>
  </tr>
</table>

Table 8: Post-hoc pairwise comparisons (Wald test, Holm correction) for CEFR Level, Language Environment, and Academic Genre. Only showing significant pairs for detectors where the overall ANOVA for the respective factor was significant (see Table 6).

higher accuracy for non-native levels (see Table 7). ‘chatgpt-roberta’ distinguishes A2 from B2, and B1.1 from B2. ‘detectgpt’ distinguishes B2 from both A2 and XX_0.

Language Environment: For ‘binoculars’, all pairwise comparisons (EFL vs. ESL, EFL vs. NS, ESL vs. NS) are significant, with EFL and ESL generally showing higher accuracy than NS (see Table 7). ‘detectgpt’ shows a significant difference only between EFL and NS.

Academic Genre: Significant differences are found for ‘detectgpt’, ‘gpt2-base’, ‘gpt2-large’, and ‘llmdet’, but specific differing pairs vary by detector. For instance, ‘detectgpt’ distinguishes Humanities from both Science & Technology and Life Sciences, and Social Sciences from Life Sciences.

6 Conclusion

In this study, we examined how social and linguistic attributes—gender, CEFR proficiency, academic field, and language environment—affect AI text detection in realistic, out-of-domain settings. Our analyses revealed that all tested detectors are highly sensitive to CEFR level and language environment, while biases tied to gender and academic field manifest more inconsistently across models. These findings highlight the pivotal role of author diversity in shaping detection performance and underscore the need for socially aware benchmarks, debiasing strategies, and more inclusive training data. By recognizing the nuanced ways in which “who is writing” influences text characteristics, future research can foster more equitable and reliable AI detection systems that effectively serve diverse linguistic and cultural communities.

Limitations

While our study offers valuable insights into how author-level attributes influence AI text detection, our evaluation is primarily based on open-source detectors and the ICNALE corpus, which predominantly comprises texts from Asian English learners. Future work should extend this analysis to additional systems and more diverse datasets to further
validate the generalizability of our findings.

Ethical Consideration

We follow ICNALE’s terms of use and do not redistribute its original data. Instead, we offer a public tool for analyzing locally obtained ICNALE files, ensuring researchers can replicate our workflow independently. Synthetic texts were generated under proper licenses; we share only generation methods and derived results to respect intellectual property and privacy.

Acknowledgments

This work was supported by Beijing Science and Technology Program (Z231100007423011) and Key Laboratory of Science, Technology and Standard in Press Industry (Key Laboratory of Intelligent Press Media Technology). We appreciate the anonymous reviewers for their helpful comments. Xiaojun Wan is the corresponding author.

References

Ebtesam Almazrouei, Hamza Alobeidli, Abdulaziz Alshamsi, Alessandro Cappelli, Ruxandra Cojocaru, Mérourane Debbah, Étienne Goffinet, Daniel Hesslow, Julien Launay, Quentin Malartic, Daniele Mazzotta, Badreddine Noune, Baptiste Pannier, and Guilherme Penedo. 2023. The falcon series of open language models. Preprint, arXiv:2311.16867.

Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. 2016. Machine Bias.

Guangsheng Bao, Yanbin Zhao, Zhiyang Teng, Linyi Yang, and Yue Zhang. 2023. Fast-detectgpt: Efficient zero-shot detection of machine-generated text via conditional probability curvature. Preprint, arXiv:2310.05130.

D. Biber, S. Conrad, and R. Reppen. 1998. Corpus Linguistics: Investigating Language Structure and Use. Cambridge Approaches to Linguistics. Cambridge University Press.

Pierre Bourdieu. 1991. Language and symbolic power. Polity.

Deborah Cameron. 2005. Language, gender, and sexuality: Current issues and new directions. Applied linguistics, 26(4):482–502.

Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurelien Rodriguez, Austen Gregerson, Ava Spataru, Baptiste Roziere, Bethany Biron, Binh Tang, Bobbie Chern, Charlotte Caucheteux, Chaya Nayak, Chloe Bi, Chris Marra, Chris McConnell, Christian Keller, Christophe Touret, Chunyang Wu, Corinne Wong, Cristian Canton Ferrer, et al. 2024. The llama 3 herd of models. Preprint, arXiv:2407.21783.

Liam Dugan, Alyssa Hwang, Filip Trhlík, Josh Magnus Ludan, Andrew Zhu, Hainiu Xu, Daphne Ippolito, and Chris Callison-Burch. 2024. Raid: A shared benchmark for robust evaluation of machine-generated text detectors. Preprint, arXiv:2405.07940.

R.A. Fisher. 1925. Statistical Methods for Research Workers. Biological monographs and manuals. Oliver and Boyd.

Susan Gal. 2012. Language, gender, and power: An anthropological review. In Gender articulated, pages 169–182. Routledge.

Sebastian Gehrmann, Hendrik Strobelt, and Alexander Rush. 2019. GLTR: Statistical detection and visualization of generated text. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 111–116, Florence, Italy. Association for Computational Linguistics.

Biyang Guo, Xin Zhang, Ziyuan Wang, Minqi Jiang, Jinran Nie, Yuxuan Ding, Jianwei Yue, and Yupeng Wu. 2023a. How close is chatgpt to human experts? comparison corpus, evaluation, and detection. Preprint, arXiv:2301.07597.

Biyang Guo, Xin Zhang, Ziyuan Wang, Minqi Jiang, Jinran Nie, Yuxuan Ding, Jianwei Yue, and Yupeng Wu. 2023b. How close is chatgpt to human experts? comparison corpus, evaluation, and detection. Preprint, arXiv:2301.07597.

Abhimanyu Hans, Avi Schwarzschild, Valeriia Cherepanova, Hamid Kazemi, Aniruddha Saha, Micah Goldblum, Jonas Geiping, and Tom Goldstein. 2024. Spotting llms with binoculars: Zero-shot detection of machine-generated text. Preprint, arXiv:2401.12070.

Moritz Hardt, Eric Price, and Nathan Srebro. 2016. Equality of opportunity in supervised learning. Preprint, arXiv:1610.02413.

Xinlei He, Xinyue Shen, Zeyuan Chen, Michael Backes, and Yang Zhang. 2023. Mtgbench: Benchmarking machine-generated text detection. Preprint, arXiv:2303.14822.

Xiaomeng Hu, Pin-Yu Chen, and Tsung-Yi Ho. 2023. Radar: Robust ai-text detection via adversarial learning. Advances in Neural Information Processing Systems.

Ken Hyland. 2000. Disciplinary discourses: Social interactions in academic writing.
Shin’ichiro Ishikawa. 2013. The icnale and sophisticated contrastive interlanguage analysis of asian learners of english. Learner corpus studies in Asia and the world, 1:91–118.

Jennifer Jenkins. 2003. World Englishes: A resource book for students. Psychology Press.

Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud-Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. 2023. Mistral 7b. Preprint, arXiv:2310.06825.

Braj B Kachru, R Quirk, and HG Widdowson. 1985. Standards, codification and sociolinguistic realism: World Englishes. Critical Concepts in Linguistics, pages 241–270.

Stephen D Krashen. 2006. The input hypothesis: issues and implications. Crane Resource Centre.

Kalpesh Krishna, Yixiao Song, Marzena Karpinska, John Wieting, and Mohit Iyyer. 2023. Paraphrasing evades detectors of ai-generated text, but retrieval is an effective defense. Preprint, arXiv:2303.13408.

James P Lantolf. 2000. Sociocultural theory and second language learning. New York Oxford University Press google schola, 2:29–63.

Yafu Li, Qintong Li, Leyang Cui, Wei Bi, Zhilin Wang, Longyue Wang, Linyi Yang, Shuming Shi, and Yue Zhang. 2024. Mage: Machine-generated text detection in the wild. Preprint, arXiv:2305.13242.

Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. Preprint, arXiv:1907.11692.

MH Long. 1996. The role of the linguistic environment in second language acquisition. Handbook of research on language acquisition, 2.

Dominik Macko, Robert Moro, Adaku Uchendu, Jason Lucas, Michiharu Yamashita, Matúš Pikuliak, Ivan Srba, Thai Le, Dongwon Lee, Jakub Simko, and Maria Bielikova. 2023. MULTITuDE: Large-scale multilingual machine-generated text detection benchmark. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 9960–9987. Singapore. Association for Computational Linguistics.

Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D. Manning, and Chelsea Finn. 2023. Detectgpt: zero-shot machine-generated text detection using probability curvature. In Proceedings of the 40th International Conference on Machine Learning, ICML’23. JMLR.org.

Edoardo Mosca, Mohamed Hesham Ibrahim Abdalla, Paolo Basso, Margherita Musumeci, and Georg Groh. 2023. Distinguishing fact from fiction: A benchmark dataset for identifying machine-generated scientific papers in the LLM era. In Proceedings of the 3rd Workshop on Trustworthy Natural Language Processing (TrustNLP 2023), pages 190–207, Toronto, Canada. Association for Computational Linguistics.

Council of Europe. Council for Cultural Co-operation. Education Committee. Modern Languages Division. 2001. Common European framework of reference for languages: Learning, teaching, assessment. Cambridge University Press.

Areg Mikael Sarvazyan, José Ángel González, Marc Franco-Salvador, Francisco Rangel, Berta Chulvi, and Paolo Rosso. 2023. Overview of autextification at iberlef 2023: Detection and attribution of machine-generated text in multiple domains. Preprint, arXiv:2309.11285.

H. Scheffé. 1999. The Analysis of Variance. Wiley Classics Library. Wiley.

Tatiana Shamardina, Vladislav Mikhailov, Daniil Chernianskii, Alena Fenogenova, Marat Saidov, Anastasiya Valeeva, Tatiana Shavrina, Ivan Smurov, Elena Tutubalina, and Ekaterina Artemova. 2022. Findings of the the ruatd shared task 2022 on artificial text detection in russian. In Computational Linguistics and Intellectual Technologies. RSUH.

Irene Solaiman, Miles Brundage, Jack Clark, Amanda Askell, Ariel Herbert-Voss, Jeff Wu, Alec Radford, Gretchen Krueger, Jong Wook Kim, Sarah Kreps, Miles McCain, Alex Newhouse, Jason Blazaklis, Kris McGuffie, and Jasmine Wang. 2019. Release strategies and the social impacts of language models. arXiv preprint.

B Solon. 2017. Fairness in machine learning nips tutorial. (No Title).

Rafael Rivera Soto, Kailin Koch, Aleem Khan, Barry Chen, Marcus Bishop, and Nicholas Andrews. 2024. Few-shot detection of machine-generated text using style representations. Preprint, arXiv:2401.06712.

STUDENT. 1908. The probable error of a mean. Biometrika, 6(1):1–25.

Zhenpeng Su, Xing Wu, Wei Zhou, Guangyuan Ma, and Songlin Hu. 2024. He3 plus: A semantic-invariant human chatgpt comparison corpus. Preprint, arXiv:2309.02731.

John M Swales. 2014. Genre analysis: English in academic and research settings. cambridge: Cambridge university press, selected 45–47, 52–60. In The Discourse Studies Reader: Main currents in theory and analysis, pages 306–316. John Benjamins Publishing Company.
Zhen Tao, Yanfang Chen, Dinghao Xi, Zhiyu Li, and Wei Xu. 2024. Towards reliable detection of Ilm-generated texts: A comprehensive evaluation framework with cudrt. Preprint, arXiv:2406.09056.

Qwen Team. 2024. Qwen2.5: A party of foundation models.

Adaku Uchendu, Zeyu Ma, Thai Le, Rui Zhang, and Dongwon Lee. 2021. Turingbench: A benchmark environment for turing test in the age of neural text generation. Preprint, arXiv:2109.13296.

Jian Wang, Shangjing Liu, Xiaofei Xie, and Yi Li. 2023a. Evaluating aigc detectors on code content. Preprint, arXiv:2304.05193.

Yuxia Wang, Jonibek Mansurov, Petar Ivanov, Jinyan Su, Artem Shelmanov, Akim Tsvigun, Osama Mohanned Afzal, Tarek Mahmoud, Giovanni Puccetti, Thomas Arnold, Alham Fikri Aji, Nizar Habash, Iryna Gurevych, and Preslav Nakov. 2024. M4gt-bench: Evaluation benchmark for black-box machine-generated text detection. Preprint, arXiv:2402.11175.

Yuxia Wang, Jonibek Mansurov, Petar Ivanov, Jinyan Su, Artem Shelmanov, Akim Tsvigun, Chenxi Whitehouse, Osama Mohammed Afzal, Tarek Mahmoud, Alham Fikri Aji, and Preslav Nakov. 2023b. M4: Multi-generator, multi-domain, and multilingual black-box machine-generated text detection. Preprint, arXiv:2305.14902.

Kangxi Wu, Liang Pang, Huawei Shen, Xueqi Cheng, and Tat-Seng Chua. 2023. LLMdet: A third party large language models generated text detection tool. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 2113–2133, Singapore. Association for Computational Linguistics.

Han Xu, Jie Ren, Pengfei He, Shenglai Zeng, Yingqian Cui, Amy Liu, Hui Liu, and Jiliang Tang. 2023. On the generalization of training-based chatgpt detection methods. Preprint, arXiv:2310.01307.

An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. 2024. Qwen2 technical report. arXiv preprint arXiv:2407.10671.

A Details of multi-factor WLS

To rigorously assess potential bias in each detector’s predictions across various author attributes, we adopt a multi-factor (multivariate) analysis with a significance threshold of \( \alpha = 0.05 \). This reveals whether a focal attribute (e.g., gender) retains an effect on detection outcomes once we control for other variables that might jointly influence performance (e.g., CEFR level, language environment).

Weighted Least Square for Detector Accuracy.
In our implementation, each row in the dataset often represents an aggregated outcome (e.g., mean accuracy) across several underlying samples. Let \( w_i \) be the total sample size for the \( i \)-th aggregated row, and let \( \text{accuracy}_i \in [0, 1] \) be the observed detection accuracy for that group. We fit a WLS model
\[
\min_{\beta} \sum_{i=1}^n w_i (\text{accuracy}_i - \beta_0 - \sum_{k=1}^p \beta_k x_{ik})^2.
\]
In Python statsmodels, this is accomplished by specifying weights=\( w_i \) in a model such as
\[
\text{accuracy} \sim C(\text{gender}) + C(\text{CEFR}) + \ldots
\]
where each predictor is treated as a categorical variable \( C(\cdot) \). If we let \( \hat{y}_i \) be the model’s prediction, then the weighted sum of squares is
\[
\text{WSSE} = \sum_{i=1}^n w_i (\text{accuracy}_i - \hat{y}_i)^2.
\]
This penalizes errors in proportion to the sample size \( w_i \).

Multi-Factor Type II ANOVA. After estimating \( \beta \), we evaluate each attribute’s unique contribution via Type II ANOVA. Concretely, for each predictor (e.g., gender), we compare:
\[
\begin{align*}
&\text{RSS}_{\text{reduced}} \quad \text{(model without gender)} \\
&\qquad \text{vs.} \\
&\text{RSS}_{\text{full}} \quad \text{(model with gender)},
\end{align*}
\]
where RSS is the weighted residual sum of squares. The partial \( F \)-test for that attribute is
\[
F = \frac{(\text{RSS}_{\text{reduced}} - \text{RSS}_{\text{full}})/\Delta p}{\text{RSS}_{\text{full}}/(n - p_{\text{full}})},
\]
where \( \Delta p \) is the difference in number of parameters between the two models, and \( p_{\text{full}} \) is the total parameter count. If the resulting \( p \)-value falls below \( \alpha \), we conclude that gender explains additional variance not captured by the other attributes.

Partial \( R^2 \). As an effect-size measure, we may compute a partial \( R^2 \) for each predictor:
\[
R^2_{\text{partial}} = \frac{\text{RSS}_{\text{reduced}} - \text{RSS}_{\text{full}}}{\text{RSS}_{\text{reduced}}} = 1 - \frac{\text{RSS}_{\text{full}}}{\text{RSS}_{\text{reduced}}}.
\]
This indicates what fraction of the previously unexplained variation is accounted for by reintroducing the focal attribute.
Pseudo-Code Sketch for Multi-Factor Analysis
Algorithm 2 provides a pseudocode flow that mirrors our Python statsmodels procedure for fitting a WLS model with multiple categorical predictors and performing Type II ANOVA to assess each predictor’s significance.

By controlling for multiple attributes at once, our multi-factor framework helps isolate each attribute’s direct influence on detection accuracy, thereby mitigating spurious correlations and confounding. This process is implemented in Python using statsmodels (for WLS and Type II ANOVA) alongside supporting libraries for data preparation and hypothesis testing.

Partitioning the Variance When we treat accuracy as a continuous outcome, we can write the total sum of squares (TSS) as

\[
\text{TSS} = \sum_{i=1}^n w_i \left( \text{accuracy}_i - \text{accuracy} \right)^2,
\]

where accuracy is the weighted grand mean. Fitting the model in Equation A yields predictions \( \hat{y}_i \), letting us define

\[
\text{WSR} = \sum_{i=1}^n w_i (\hat{y}_i - \text{accuracy})^2.
\]

Here, *WSR is the “weighted sum of regression”*. and

\[
\text{WSSE} = \sum_{i=1}^n w_i \left( \text{accuracy}_i - \hat{y}_i \right)^2.
\]

Hence, TSS = WSR + WSSE. In a multi-factor model, the partial \( F \)-tests correspond to whether WSSE drops sufficiently when an attribute is included.

Implementation. We implement all these procedures in Python. We leverage statsmodels for WLS fits and Type II ANOVA, scipy.stats for complementary t-tests and distribution checks, and custom matching/down-sampling routines to handle partial confounds. Part A details how variance in accuracy is decomposed into regression (WSR) and residual (WSSE) sums of squares, while Tables 9 present the resulting \( p \)-values and significance decisions at \( \alpha = 0.05 \). Through this process, we can identify which author factors (e.g., gender, CEFR, environment) exhibit genuine biases within the AI text detection pipeline, and which factors do not remain significant once confounds are accounted for.

Algorithm 2 Multi-Factor WLS + Type II ANOVA
Input: Dataset of \( n \) texts, each with:

• A binary classification \( y_i \in \{0, 1\} \) (0: human, 1: AI).

• An attribute vector \( x_i \) (e.g., CEFR=B1, Sex=F, ...).

Significance level \( \alpha \) (e.g., 0.05).
Output: ANOVA table (F-statistics, \( p \)-values) and LSMeans.
1: Data Preparation:
• Group texts by unique attribute combinations (\( x_i \)).
• For each group \( j \), compute the mean accuracy \( a_j \) and weight \( w_j \) (number of texts in group).
• Treat each attribute as a factor.
2: Fit WLS Model: Estimate parameters \( \beta \) by minimizing the weighted sum of squared errors:

\[
\min_{\beta} \sum_{j=1}^g w_j (a_j - x_j^T \beta)^2,
\]

where \( g \) is the number of groups, and \( x_j \) is a representative attribute vector for group \( j \).
3: Perform Type II ANOVA: For each attribute:
• Construct a reduced model by removing the attribute.
• Refit the WLS model to the reduced model, obtaining RSS_reduced.
• Compute the \( F \)-statistic:

\[
F = \frac{(\text{RSS}_{\text{reduced}} - \text{RSS}_{\text{full}})/\Delta p}{\text{RSS}_{\text{full}}/(n - p_{\text{full}})},
\]

where \( \Delta p \) is the difference in the number of parameters between the full and reduced models, and \( p_{\text{full}} \) is the number of parameters in the full model.
• Calculate the \( p \)-value from the \( F \)-distribution.
4: Calculate LSMeans: Compute LSMeans for each attribute level using the *full* WLS model.
5: ANOVA table and LSMeans.
B Detector Details

RoBERTa (GPT-2) (Solaiman et al., 2019) This detector is a RoBERTa model (Liu et al., 2019) fine-tuned on outputs from GPT-2. The training dataset consists of GPT-2 generations from various decoding strategies (greedy, top-k=50, and random sampling) in an open-domain setting. This detector has been a long-standing baseline in the field. We use both the base and large versions, obtained from OpenAI4.

RoBERTa (ChatGPT) (Guo et al., 2023b) This RoBERTa-base (Liu et al., 2019) detector is fine-tuned on the HC3 dataset, which contains approximately 27,000 question-answer pairs, with answers generated by both humans and ChatGPT. The questions span diverse domains, including Reddit, medicine, finance, and law. We access the detector via HuggingFace Datasets: Hello-SimpleAI/chatgpt-detector-roberta.

RADAR (Hu et al., 2023) This detector is a fine-tuned Vicuna 7B model (itself a fine-tune of LLaMA 7B), trained in a generative adversarial setting. The training involved a paraphraser model designed to fool the detector, and the detector was trained to distinguish between paraphraser outputs, human-written text from the WebText dataset, and generations from the original language model. We access RADAR via HuggingFace: TrustSafeAI/RADAR-Vicuna-7B.

GLTR (Gehrmann et al., 2019) Originally designed as an interactive tool to aid human detection of generated text, GLTR has become a standard baseline in detector robustness evaluations. It analyzes the likelihood of text under a language model, binning tokens based on their predicted probabilities. These bins then serve as features for detection. We use the default GLTR settings5: a cutoff rank of 10 and GPT-2 small as the language model.

DetectGPT (Mitchell et al., 2023) This zero-shot detector leverages the observation that LM-generated text often resides in regions of negative curvature within the model’s log probability function. DetectGPT compares the log probability of an input text, computed by the target LM, to the average log probability of slightly perturbed versions of the text (generated using a separate masked language model like T5). A significant drop in log probability for the perturbed text indicates a higher likelihood of machine generation.

FastDetectGPT (Bao et al., 2023) This detector is an optimized version of DetectGPT (Mitchell et al., 2023), achieving a 340x speedup without sacrificing accuracy. Following the original implementation, we use GPT-Neo-2.7B as the scoring model and GPT-J-6B as the reference model for generating perturbations. Neither of these models was used to generate the continuations in our dataset.

FastDetectLLM Referenced as fastdetectllm in the RAID benchmark code (Dugan et al., 2024), this zero-shot detector is conceptually related to FastDetectGPT (Mitchell et al., 2023). FastDetectLLM directly uses the average log-rank of input tokens, predicted by a scoring language model (default: GPT-Neo-2.7B), as its detection metric. Lower average log-ranks suggest a higher likelihood of machine generation. This approach bypasses FastDetectGPT’s perturbation and sampling steps, significantly improving speed.

Binoculars (Hans et al., 2024) This detector uses the ratio of perplexity to cross-entropy between two similar language models as its detection metric. We use the official code and default models (Falcon 7B and Falcon 7B Instruct (Almazrouei et al., 2023)), which, like FastDetectGPT, were not used for text generation in our dataset.

LLMDet (Wu et al., 2023) This detector uses the "proxy-perplexity" from 10 small language models as features. Proxy-perplexity approximates true perplexity by sampling n-grams, avoiding full model execution. None of the models used for proxy-perplexity calculation were involved in generating text for our dataset.

C Additional Statistical Tests

To further explore whether potential bias arises under different conditions, we implement three complementary strategies: (1) a single-factor analysis that does not control for confounding variables, (2) a matched subset analysis that fixes control features, and (3) a down-sampled matched analysis that enforces balanced sample sizes.6 Below, we

4https://openapublic.azureedge.net/gpt-2/detector-models/v1/detector-large.pt
5https://github.com/HendrikStrobelt/detecting-fake-text
6We note that, due to the complete coverage of all attribute combinations in our dataset, the single-factor and matched-
describe each data manipulation strategy, followed by a description of the statistical tests applied to the resulting datasets.

C.1 Data Subsetting and Sampling Strategies

(1) Single-Factor (No Confound Control). In the simplest approach, we do not explicitly control for other attributes. We group the data by the focal attribute (e.g., gender = Female vs. Male) and compare detection outcomes. This is a baseline approach that is straightforward but may be susceptible to confounding.

(2) Matched Subset (Fixing Control Features). In a more controlled approach, we fix all other (control) attributes so that only the main feature varies among samples. Suppose our main feature has values \( \{f_1, f_2, \ldots, f_n\} \). For each unique combination of the control features (e.g., CEFR = B2, environment = ESL), we identify those samples that share exactly those control-feature values but differ exclusively in the main feature. We keep only the combinations that contain all categories \( \{f_1, \ldots, f_n\} \). Concatenating these subsets yields a matched dataset where any difference in detector outcomes is more plausibly attributed to the main feature rather than being confounded by other factors. Algorithm 3 shows how we construct the matched subset.

Mathematically, let

\[
C = \{c_1, \ldots, c_r\} \quad \text{and} \quad \Omega = \prod_{j=1}^r \text{Levels}(c_j).
\]

For each combination \( \omega \in \Omega \), we form the subset

\[
S_\omega = \{ x \in D \mid x[c_j] = \omega_j, \forall c_j \in C \}.
\]

If \( S_\omega \) spans all main feature categories, we retain it. We then unify these retained subsets to form

\[
\text{Matched} = \bigcup_{\omega \in \Omega : \{f_1, \ldots, f_n\} \subseteq \text{MainFeature}(S_\omega)} S_\omega.
\]

(3) Down-Sampled Matched (One-to-One Matching). Even if the matched dataset includes all main-feature categories, different groups may still have unequal sample sizes. In this one-to-one variant, we require that for every record in the smallest category (within each combination of control features), there is exactly one matched record in every other category. Concretely, for subset approaches yield identical results in this specific case; however, we describe both for conceptual clarity.

Algorithm 3 Matched Subset (Fix Control Features)

Input: Dataset \( D \), main feature \( F \) with categories \( \{f_1, \ldots, f_n\} \), control features \( C = \{c_1, \ldots, c_r\} \)

Output: A subset \( M \) where only \( F \) varies in each control-feature combo

1: \( M \leftarrow \emptyset \)
2: \( \Omega \leftarrow \text{CartProd}(\text{Levels}(c_1), \ldots, \text{Levels}(c_r)) \)
3: for each \( \omega \in \Omega \) do
4:   \( S_\omega \leftarrow \{ x \in D \mid \forall j, x[c_j] = \omega_j \} \)
5:   \( \mathcal{F}_\omega \leftarrow \{\text{main feature value for } x \in S_\omega \} \)
6:   if \( \{f_1, \ldots, f_n\} \subseteq \mathcal{F}_\omega \) then
7:     \( M \leftarrow M \cup S_\omega \)
8:   end if
9: end for
10: return \( M \)

each record in the smallest category, we randomly select one record from each larger category that shares the same control-feature values. This ensures balanced sample sizes and enforces a strict one-to-one matching across categories, though it may discard additional data from larger categories. Algorithm 4 shows how to downsample to create matched data.

Formally, let

\[
\Omega = \text{CartProd}(\text{Levels}(c_1), \ldots, \text{Levels}(c_r)),
\]

be the Cartesian product of all levels of control features \( C = \{c_1, \ldots, c_r\} \). For each \( \omega \in \Omega \), define

\[
S_\omega = \{ x \in D \mid \forall c_j \in C,\ x[c_j] = \omega_j \},
\]

the subset of \( D \) whose rows match the control-feature values in \( \omega \). Let \( f_{k^*} \) be the category of the main feature \( F \) with the fewest samples in \( S_\omega \). For each \( x \in S_\omega \) with \( x[F] = f_{k^*} \), define a matched set

\[
S_x = \{x\} \cup \bigcup_{\substack{f_i \in \{f_1, \ldots, f_n\} \\ f_i \neq f_{k^*}}} \Big\{ y \in S_\omega \mid y[F] = f_i, \\
\text{y is sampled exactly once}\Big\}.
\]

where we pick exactly one such \( y \) randomly from the available rows each time (removing it to prevent re-use). The overall matched subset for \( \omega \) is then

\[
\tilde{S}_\omega = \bigcup_{x : x[F] = f_{k^*}} S_x,
\]
and the final down-sampled matched dataset is

\[
M = \bigcup_{\omega \in \Omega} \tilde{S}_\omega.
\]

This matching enforces that each record in the smallest category is paired one-to-one with exactly one record from each of the other categories.

Algorithm 4 Down-Sampled Matched Data Construction (One-to-One Matching)
Input: Dataset \( D \), main feature \( F \) with categories \( \{f_1, \ldots, f_n\} \), control features \( C = \{c_1, \ldots, c_r\} \)
Output: A balanced, matched dataset \( M \) where each sample in the smallest category is paired one-to-one with exactly one sample in every other category, for the same control-feature values
1: \( M \leftarrow \emptyset \)
2: \( \Omega \leftarrow \mathrm{CartProd}(\mathrm{Levels}(c_1), \ldots, \mathrm{Levels}(c_r)) \)

3: for each \( \omega \in \Omega \) do
4:   \( S_\omega \leftarrow \{ x \in D \mid \forall j, x[c_j] = \omega_j \} \)
5:   Let \( \{f_{k^*}\} \) be the category in \( S_\omega \) with the fewest samples (the “smallest” category)
6:   for each \( x \in S_\omega \) such that \( x[F] = f_{k^*} \) do
7:     \( S_x \leftarrow \{x\} \) [Start a matched set with the sample from the smallest category]
8:     for each \( f_i \in \{f_1, \ldots, f_n\} \setminus \{f_{k^*}\} \) do
9:       From \( S_\omega \) with \( F = f_i \), randomly pick exactly one sample \( y \) (if any remain)
10:      \( S_x \leftarrow S_x \cup \{y\} \)
11:     Remove \( y \) from \( S_\omega \) to prevent re-use
12:   end for
13:  \( M \leftarrow M \cup S_x \)
14: end for
15: end for
16: return \( M \)

C.2 Statistical Tests
After applying one of the data strategies above (single-factor, matched subset, or down-sampled matched), we perform hypothesis tests to assess the significance of the focal attribute. We use weighted statistical tests to account for the varying number of texts within each group.

Binary Attributes (Two Groups). If the focal attribute has exactly two categories (e.g., gender: Female vs. Male), we employ Welch’s two-sample t-test (STUDENT, 1908), modified to incorporate weights. The weighted t-test, with unequal variances, is calculated as follows.

Let \( w_i \) be the weight, and let \( X_i \) be its detection accuracy of each observation. Let two groups be \( G_1 \) and \( G_2 \)

The weighted means:

\[
\bar{X}_k = \frac{\sum_{i \in G_k} w_i X_i}{\sum_{i \in G_k} w_i}, \quad k \in \{1, 2\}.
\]

Weighted variances:

\[
\hat{\sigma}_k^2 = \frac{\sum_{i \in G_k} w_i (X_i - \bar{X}_k)^2}{\sum_{i \in G_k} w_i}, \quad k \in \{1, 2\}.
\]

Total weights per group:

\[
W_k = \sum_{i \in G_k} w_i, \quad k \in \{1, 2\}.
\]

Weighted Welch’s t-statistic:

\[
t = \frac{\bar{X}_1 - \bar{X}_2}{\sqrt{\frac{\hat{\sigma}_1^2}{W_1} + \frac{\hat{\sigma}_2^2}{W_2}}}.
\]

Approximate degrees of freedom:

\[
\nu \approx \frac{\left( \frac{\hat{\sigma}_1^2}{W_1} + \frac{\hat{\sigma}_2^2}{W_2} \right)^2}{\left( \frac{\hat{\sigma}_1^2}{W_1} \right)^2 / (n_1-1) + \left( \frac{\hat{\sigma}_2^2}{W_2} \right)^2 / (n_2-1)},
\]

where \( n_1, n_2 \) denote the (unweighted) sample counts in each group.

We compare the resulting p-value to \( \alpha \) (e.g., 0.05) to assess significance.

Multi-Class Attributes (More Than Two Groups). When the focal attribute has more than two categories, we use a weighted one-way ANOVA (Fisher, 1925), implemented via a Weighted Least Squares (WLS) regression. The model is of the form:

\[
\text{accuracy}_i \sim C(\text{focal\_attribute})
\]

with weights \( w_i \). We fit this model using ‘statsmodels’ and then perform a Type II ANOVA. The relevant \( F \)-statistic is:

\[
F = \frac{\text{MS}_{\text{Between}}}{\text{MS}_{\text{Within}}}
\]

A significant p-value indicates that at least one category differs from the others.

D Detailed Results
E Additional Results
F Detailed Subset Results
<table>
  <tr>
    <th rowspan="2">Detector</th>
    <th colspan="3">CEFR</th>
    <th colspan="3">Sex</th>
    <th colspan="3">Academic Genre</th>
    <th colspan="3">Language Env.</th>
  </tr>
  <tr>
    <th>p-value</th>
    <th>Sig.?</th>
    <th></th>
    <th>p-value</th>
    <th>Sig.?</th>
    <th></th>
    <th>p-value</th>
    <th>Sig.?</th>
    <th></th>
    <th>p-value</th>
    <th>Sig.?</th>
    <th></th>
  </tr>
  <tr>
    <td>binoculars</td>
    <td>1.3745 \times 10^{-15}</td>
    <td>Yes</td>
    <td>0.80451</td>
    <td>No</td>
    <td>0.81889</td>
    <td>No</td>
    <td>1.6875 \times 10^{-17}</td>
    <td>Yes</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>chatgpt-roberta</td>
    <td>0.036675</td>
    <td>Yes</td>
    <td>0.61647</td>
    <td>No</td>
    <td>0.41117</td>
    <td>No</td>
    <td>0.79088</td>
    <td>No</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>detectgpt</td>
    <td>3.4682 \times 10^{-7}</td>
    <td>Yes</td>
    <td>0.17331</td>
    <td>No</td>
    <td>0.001146</td>
    <td>Yes</td>
    <td>0.0067062</td>
    <td>Yes</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>fastdetectgpt</td>
    <td>1.7303 \times 10^{-5}</td>
    <td>Yes</td>
    <td>0.95033</td>
    <td>No</td>
    <td>0.22113</td>
    <td>No</td>
    <td>0.0023632</td>
    <td>Yes</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>fastdetectllm</td>
    <td>2.3643 \times 10^{-13}</td>
    <td>Yes</td>
    <td>0.14879</td>
    <td>No</td>
    <td>2.5190 \times 10^{-6}</td>
    <td>Yes</td>
    <td>1.0241 \times 10^{-7}</td>
    <td>Yes</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>gltr</td>
    <td>5.0894 \times 10^{-17}</td>
    <td>Yes</td>
    <td>0.37228</td>
    <td>No</td>
    <td>0.79752</td>
    <td>No</td>
    <td>4.2131 \times 10^{-17}</td>
    <td>Yes</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>gpt2-base</td>
    <td>3.5869 \times 10^{-37}</td>
    <td>Yes</td>
    <td>0.47449</td>
    <td>No</td>
    <td>2.6296 \times 10^{-10}</td>
    <td>Yes</td>
    <td>1.4756 \times 10^{-25}</td>
    <td>Yes</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>gpt2-large</td>
    <td>1.7213 \times 10^{-4}</td>
    <td>Yes</td>
    <td>0.70402</td>
    <td>No</td>
    <td>2.8182 \times 10^{-7}</td>
    <td>Yes</td>
    <td>0.030228</td>
    <td>Yes</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>llmdet</td>
    <td>1.1682 \times 10^{-38}</td>
    <td>Yes</td>
    <td>0.07170</td>
    <td>No</td>
    <td>2.7679 \times 10^{-4}</td>
    <td>Yes</td>
    <td>8.4468 \times 10^{-15}</td>
    <td>Yes</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>radar</td>
    <td>2.0507 \times 10^{-5}</td>
    <td>Yes</td>
    <td>0.77951</td>
    <td>No</td>
    <td>0.12957</td>
    <td>No</td>
    <td>0.010882</td>
    <td>Yes</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
</table>

Table 9: WLS results: ANOVA p-values for each factor across detectors. “Yes” indicates \( p < 0.05 \) (significant), “No” otherwise.

<table>
  <tr>
    <th rowspan="2">Factor</th>
    <th rowspan="2">Detector</th>
    <th rowspan="2">Comparison</th>
    <th colspan="2">WaldStat</th>
    <th colspan="2">raw_p</th>
    <th colspan="2">p_corr</th>
    <th colspan="2">Sig.?</th>
  </tr>
  <tr>
    <th>raw_p</th>
    <th>raw_p</th>
    <th>raw_p</th>
    <th>raw_p</th>
    <th>raw_p</th>
    <th>raw_p</th>
    <th>raw_p</th>
    <th>raw_p</th>
    <th>raw_p</th>
    <th>raw_p</th>
  </tr>
  <tr>
    <td rowspan="4">CEFR</td>
    <td rowspan="4">binoculars</td>
    <td>B2_0 vs XX_0</td>
    <td>36.1784</td>
    <td>2.7460e-09</td>
    <td>1.9222e-08</td>
    <td>Yes</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>A2_0 vs XX_0</td>
    <td>72.4739</td>
    <td>8.4083e-17</td>
    <td>8.4083e-16</td>
    <td>Yes</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>B1_2 vs XX_0</td>
    <td>49.9436</td>
    <td>3.4660e-12</td>
    <td>3.1194e-11</td>
    <td>Yes</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>B1_1 vs XX_0</td>
    <td>48.0481</td>
    <td>8.6122e-12</td>
    <td>6.8897e-11</td>
    <td>Yes</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td rowspan="2">chatgpt-roberta</td>
    <td>A2_0 vs B2_0</td>
    <td>9.4951</td>
    <td>2.1314e-03</td>
    <td>2.1314e-02</td>
    <td>Yes</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>B1_1 vs B2_0</td>
    <td>8.0170</td>
    <td>4.7512e-03</td>
    <td>4.2761e-02</td>
    <td>Yes</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td rowspan="2">detectgpt</td>
    <td>B2_0 vs A2_0</td>
    <td>11.6703</td>
    <td>6.6733e-04</td>
    <td>4.0040e-03</td>
    <td>Yes</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>B2_0 vs XX_0</td>
    <td>25.2118</td>
    <td>6.3457e-07</td>
    <td>5.7111e-06</td>
    <td>Yes</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td rowspan="4">Language Env.</td>
    <td rowspan="4">binoculars</td>
    <td>EFL vs ESL</td>
    <td>8.8010</td>
    <td>3.1010e-03</td>
    <td>3.1010e-03</td>
    <td>Yes</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>EFL vs NS</td>
    <td>72.4739</td>
    <td>8.4083e-17</td>
    <td>2.5225e-16</td>
    <td>Yes</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>ESL vs NS</td>
    <td>21.8296</td>
    <td>3.4990e-06</td>
    <td>6.9980e-06</td>
    <td>Yes</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>detectgpt</td>
    <td>EFL vs NS</td>
    <td>8.5938</td>
    <td>3.4700e-03</td>
    <td>1.0410e-02</td>
    <td>Yes</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td rowspan="3">detectgpt</td>
    <td>Humanities vs Sci&Tech</td>
    <td>8.8179</td>
    <td>3.0727e-03</td>
    <td>1.5363e-02</td>
    <td>Yes</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>Humanities vs Life Sci</td>
    <td>9.3496</td>
    <td>2.3052e-03</td>
    <td>1.3831e-02</td>
    <td>Yes</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>Social Sci vs Life Sci</td>
    <td>6.7344</td>
    <td>9.6317e-03</td>
    <td>3.8527e-02</td>
    <td>Yes</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td rowspan="2">Academic Genre</td>
    <td rowspan="2">gpt2-base</td>
    <td>Sci&Tech vs Humanities</td>
    <td>24.8324</td>
    <td>7.6795e-07</td>
    <td>3.8398e-06</td>
    <td>Yes</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>Sci&Tech vs Social Sci</td>
    <td>41.8589</td>
    <td>1.7148e-10</td>
    <td>1.0289e-09</td>
    <td>Yes</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td rowspan="2">gpt2-large</td>
    <td>Sci&Tech vs Humanities</td>
    <td>12.1460</td>
    <td>5.1890e-04</td>
    <td>2.5945e-03</td>
    <td>Yes</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>Sci&Tech vs Social Sci</td>
    <td>32.5064</td>
    <td>1.6753e-08</td>
    <td>1.0052e-07</td>
    <td>Yes</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td rowspan="3">llmdet</td>
    <td>Life Sci vs Sci&Tech</td>
    <td>11.0328</td>
    <td>9.3605e-04</td>
    <td>3.7442e-03</td>
    <td>Yes</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>Life Sci vs Social Sci</td>
    <td>12.6490</td>
    <td>3.9804e-04</td>
    <td>1.9902e-03</td>
    <td>Yes</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>Life Sci vs Humanities</td>
    <td>18.2247</td>
    <td>2.2005e-05</td>
    <td>1.3203e-04</td>
    <td>Yes</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
</table>

Table 10: WLS results: Post-hoc pairwise comparisons (Wald test, Holm correction) for CEFR Level, Language Environment, and Academic Genre. Only showing significant pairs for detectors where the overall ANOVA for the corresponding factor was significant (see Table 9).
<table>
  <tr>
    <th>Factor</th>
    <th>Detector</th>
    <th>Statistic</th>
    <th>p-value</th>
    <th>Sig?</th>
    <th>Ranking (LS-means)</th>
  </tr>
  <tr>
    <td rowspan="8">CEFR</td>
    <td>binoculars</td>
    <td>F = 19.0332</td>
    <td>5.78e-15</td>
    <td>Yes</td>
    <td>A2_0 &gt; B2_0 &gt; B1_1 &gt; B1_2 &gt; XX_0</td>
  </tr>
  <tr>
    <td>chatgpt-roberta</td>
    <td>F = 2.9443</td>
    <td>1.97e-02</td>
    <td>Yes</td>
    <td>A2_0 &gt; B1_1 &gt; B1_2 &gt; XX_0 &gt; B2_0</td>
  </tr>
  <tr>
    <td>detectgpt</td>
    <td>F = 9.3006</td>
    <td>2.39e-07</td>
    <td>Yes</td>
    <td>B2_0 &gt; B1_2 &gt; B1_1 &gt; A2_0 &gt; XX_0</td>
  </tr>
  <tr>
    <td>fastdetectgpt</td>
    <td>F = 7.4965</td>
    <td>5.56e-06</td>
    <td>Yes</td>
    <td>B2_0 &gt; B1_2 &gt; A2_0 &gt; B1_1 &gt; XX_0</td>
  </tr>
  <tr>
    <td>fastdetectllm</td>
    <td>F = 21.9696</td>
    <td>3.19e-17</td>
    <td>Yes</td>
    <td>XX_0 &gt; A2_0 &gt; B1_1 &gt; B2_0 &gt; B1_2</td>
  </tr>
  <tr>
    <td>gltr</td>
    <td>F = 21.8672</td>
    <td>3.82e-17</td>
    <td>Yes</td>
    <td>A2_0 &gt; B1_1 &gt; B1_2 &gt; B2_0 &gt; XX_0</td>
  </tr>
  <tr>
    <td>gpt2-base</td>
    <td>F = 50.8631</td>
    <td>3.60e-38</td>
    <td>Yes</td>
    <td>A2_0 &gt; B1_1 &gt; B1_2 &gt; B2_0 &gt; XX_0</td>
  </tr>
  <tr>
    <td>gpt2-large</td>
    <td>F = 6.0261</td>
    <td>8.85e-05</td>
    <td>Yes</td>
    <td>B1_1 &gt; B2_0 &gt; B1_2 &gt; A2_0 &gt; XX_0</td>
  </tr>
  <tr>
    <td>llmdet</td>
    <td>F = 35.2707</td>
    <td>3.74e-27</td>
    <td>Yes</td>
    <td>A2_0 &gt; XX_0 &gt; B1_1 &gt; B1_2 &gt; B2_0</td>
  </tr>
  <tr>
    <td>radar</td>
    <td>F = 6.6219</td>
    <td>3.03e-05</td>
    <td>Yes</td>
    <td>B1_1 &gt; B1_2 &gt; B2_0 &gt; A2_0 &gt; XX_0</td>
  </tr>
  <tr>
    <td rowspan="8">Sex</td>
    <td>binoculars</td>
    <td>t = -4.2320</td>
    <td>2.32e-05</td>
    <td>Yes</td>
    <td>F: 0.9412, M: 0.9425 (M &gt; F)</td>
  </tr>
  <tr>
    <td>chatgpt-roberta</td>
    <td>t = -10.3064</td>
    <td>6.76e-25</td>
    <td>Yes</td>
    <td>F: 0.7433, M: 0.7489 (M &gt; F)</td>
  </tr>
  <tr>
    <td>detectgpt</td>
    <td>t = 28.7863</td>
    <td>1.39e-181</td>
    <td>Yes</td>
    <td>F: 0.8000, M: 0.7885 (F &gt; M)</td>
  </tr>
  <tr>
    <td>fastdetectgpt</td>
    <td>t = 7.9967</td>
    <td>1.28e-15</td>
    <td>Yes</td>
    <td>F: 0.8986, M: 0.8961 (F &gt; M)</td>
  </tr>
  <tr>
    <td>fastdetectllm</td>
    <td>t = -12.2473</td>
    <td>1.83e-34</td>
    <td>Yes</td>
    <td>F: 0.4793, M: 0.4806 (M &gt; F)</td>
  </tr>
  <tr>
    <td>gltr</td>
    <td>t = 7.2602</td>
    <td>3.89e-13</td>
    <td>Yes</td>
    <td>F: 0.8190, M: 0.8138 (F &gt; M)</td>
  </tr>
  <tr>
    <td>gpt2-base</td>
    <td>t = -16.7136</td>
    <td>1.24e-62</td>
    <td>Yes</td>
    <td>F: 0.6193, M: 0.6282 (M &gt; F)</td>
  </tr>
  <tr>
    <td>gpt2-large</td>
    <td>t = -11.7789</td>
    <td>5.23e-32</td>
    <td>Yes</td>
    <td>F: 0.5558, M: 0.5623 (M &gt; F)</td>
  </tr>
  <tr>
    <td>llmdet</td>
    <td>t = -32.8895</td>
    <td>3.45e-236</td>
    <td>Yes</td>
    <td>F: 0.4968, M: 0.5020 (M &gt; F)</td>
  </tr>
  <tr>
    <td>radar</td>
    <td>t = -0.9671</td>
    <td>3.34e-01</td>
    <td>No</td>
    <td>F vs. M not significant</td>
  </tr>
  <tr>
    <td rowspan="8">Academic Genre</td>
    <td>binoculars</td>
    <td>F = 0.5213</td>
    <td>6.68e-01</td>
    <td>No</td>
    <td>–</td>
  </tr>
  <tr>
    <td>chatgpt-roberta</td>
    <td>F = 1.1469</td>
    <td>3.29e-01</td>
    <td>No</td>
    <td>–</td>
  </tr>
  <tr>
    <td>detectgpt</td>
    <td>F = 6.4159</td>
    <td>2.69e-04</td>
    <td>Yes</td>
    <td>Humanities &gt; Social Sci &gt; Sci&Tech &gt; Life Sci</td>
  </tr>
  <tr>
    <td>fastdetectgpt</td>
    <td>F = 1.8218</td>
    <td>1.41e-01</td>
    <td>No</td>
    <td>–</td>
  </tr>
  <tr>
    <td>fastdetectllm</td>
    <td>F = 8.1352</td>
    <td>2.43e-05</td>
    <td>Yes</td>
    <td>Humanities &gt; Social Sci &gt; Sci&Tech &gt; Life Sci</td>
  </tr>
  <tr>
    <td>gltr</td>
    <td>F = 0.2925</td>
    <td>8.31e-01</td>
    <td>No</td>
    <td>–</td>
  </tr>
  <tr>
    <td>gpt2-base</td>
    <td>F = 14.7583</td>
    <td>2.32e-09</td>
    <td>Yes</td>
    <td>Sci&Tech &gt; Life Sci &gt; Humanities &gt; Social Sci</td>
  </tr>
  <tr>
    <td>gpt2-large</td>
    <td>F = 12.1231</td>
    <td>9.15e-08</td>
    <td>Yes</td>
    <td>Sci&Tech &gt; Life Sci &gt; Humanities &gt; Social Sci</td>
  </tr>
  <tr>
    <td>llmdet</td>
    <td>F = 7.7058</td>
    <td>4.44e-05</td>
    <td>Yes</td>
    <td>Life Sci &gt; Sci&Tech &gt; Social Sci &gt; Humanities</td>
  </tr>
  <tr>
    <td>radar</td>
    <td>F = 2.4177</td>
    <td>6.51e-02</td>
    <td>No</td>
    <td>–</td>
  </tr>
  <tr>
    <td rowspan="8">Language Env.</td>
    <td>binoculars</td>
    <td>F = 42.2609</td>
    <td>3.56e-18</td>
    <td>Yes</td>
    <td>EFL &gt; ESL &gt; NS</td>
  </tr>
  <tr>
    <td>chatgpt-roberta</td>
    <td>F = 0.3534</td>
    <td>7.02e-01</td>
    <td>No</td>
    <td>–</td>
  </tr>
  <tr>
    <td>detectgpt</td>
    <td>F = 9.1837</td>
    <td>1.14e-04</td>
    <td>Yes</td>
    <td>ESL &gt; EFL &gt; NS</td>
  </tr>
  <tr>
    <td>fastdetectgpt</td>
    <td>F = 7.2639</td>
    <td>7.24e-04</td>
    <td>Yes</td>
    <td>ESL &gt; EFL &gt; NS</td>
  </tr>
  <tr>
    <td>fastdetectllm</td>
    <td>F = 33.2769</td>
    <td>1.31e-14</td>
    <td>Yes</td>
    <td>NS &gt; EFL &gt; ESL</td>
  </tr>
  <tr>
    <td>gltr</td>
    <td>F = 40.7305</td>
    <td>1.42e-17</td>
    <td>Yes</td>
    <td>EFL &gt; ESL &gt; NS</td>
  </tr>
  <tr>
    <td>gpt2-base</td>
    <td>F = 49.4957</td>
    <td>5.40e-21</td>
    <td>Yes</td>
    <td>EFL &gt; ESL &gt; NS</td>
  </tr>
  <tr>
    <td>gpt2-large</td>
    <td>F = 4.6056</td>
    <td>1.03e-02</td>
    <td>Yes</td>
    <td>ESL &gt; EFL &gt; NS</td>
  </tr>
  <tr>
    <td>llmdet</td>
    <td>F = 6.2017</td>
    <td>2.12e-03</td>
    <td>Yes</td>
    <td>NS &gt; ESL &gt; EFL</td>
  </tr>
  <tr>
    <td>radar</td>
    <td>F = 7.1878</td>
    <td>8.05e-04</td>
    <td>Yes</td>
    <td>EFL &gt; ESL &gt; NS</td>
  </tr>
</table>

Table 11: Single Factor and Matched Subset Results: Weighted ANOVA results for CEFR, Academic Genre, and Language Environment; Weighted T-test results for Sex. "Statistic" indicates the test statistic (F for ANOVA, t for T-test). "Sig?" indicates omnibus significance. "Ranking" shows LS-means. A “--” indicates that the overall ANOVA for that factor and detector was not statistically significant, so LSMeans are not reported.

<table>
  <tr>
    <th>Detector</th>
    <th>N</th>
    <th>F</th>
    <th>p</th>
    <th>Sig?</th>
    <th>LS Mean (Hum)</th>
    <th>LS Mean (SS)</th>
    <th>LS Mean (ST)</th>
    <th>LS Mean (LSci)</th>
  </tr>
  <tr>
    <td>binoculars</td>
    <td>624</td>
    <td>0.4983</td>
    <td>0.684</td>
    <td>No</td>
    <td>–</td>
    <td>–</td>
    <td>–</td>
    <td>–</td>
  </tr>
  <tr>
    <td>chatgpt-roberta</td>
    <td>624</td>
    <td>1.0527</td>
    <td>0.369</td>
    <td>No</td>
    <td>–</td>
    <td>–</td>
    <td>–</td>
    <td>–</td>
  </tr>
  <tr>
    <td>detectgpt</td>
    <td>624</td>
    <td>4.5679</td>
    <td>0.0036</td>
    <td>Yes</td>
    <td>0.8066</td>
    <td>0.7979</td>
    <td>0.7857</td>
    <td>0.7679</td>
  </tr>
  <tr>
    <td>fastdetectgpt</td>
    <td>1248</td>
    <td>1.2188</td>
    <td>0.302</td>
    <td>No</td>
    <td>–</td>
    <td>–</td>
    <td>–</td>
    <td>–</td>
  </tr>
  <tr>
    <td>fastdetectllm</td>
    <td>624</td>
    <td>6.3876</td>
    <td>2.88e-04</td>
    <td>Yes</td>
    <td>0.4834</td>
    <td>0.4813</td>
    <td>0.4778</td>
    <td>0.4697</td>
  </tr>
  <tr>
    <td>gltr</td>
    <td>624</td>
    <td>0.2762</td>
    <td>0.843</td>
    <td>No</td>
    <td>–</td>
    <td>–</td>
    <td>–</td>
    <td>–</td>
  </tr>
  <tr>
    <td>gpt2-base</td>
    <td>624</td>
    <td>11.3682</td>
    <td>2.89e-07</td>
    <td>Yes</td>
    <td>0.6074</td>
    <td>0.6011</td>
    <td>0.6474</td>
    <td>0.6216</td>
  </tr>
  <tr>
    <td>gpt2-large</td>
    <td>624</td>
    <td>10.3231</td>
    <td>1.23e-06</td>
    <td>Yes</td>
    <td>0.5505</td>
    <td>0.5310</td>
    <td>0.5810</td>
    <td>0.5568</td>
  </tr>
  <tr>
    <td>llmdet</td>
    <td>624</td>
    <td>5.9057</td>
    <td>5.06e-04</td>
    <td>Yes</td>
    <td>0.4918</td>
    <td>0.4995</td>
    <td>0.5009</td>
    <td>0.5109</td>
  </tr>
  <tr>
    <td>radar</td>
    <td>624</td>
    <td>1.9868</td>
    <td>0.115</td>
    <td>No</td>
    <td>–</td>
    <td>–</td>
    <td>–</td>
    <td>–</td>
  </tr>
</table>

Table 12: Down-Sample Matched Results: Weighted ANOVA on “Academic Genre” for each detector. "Sig?" refers to overall significance of the ANOVA (p &lt; 0.05). Only detectors with Yes in the “Sig?” column have meaningful LS Means and potential post-hoc comparisons. A “--” indicates that the overall ANOVA for that factor and detector was not statistically significant, so LSMeans are not reported.
<table>
  <tr>
    <th>Detector</th>
    <th>N</th>
    <th>t</th>
    <th>p</th>
    <th>df</th>
    <th>Sig?</th>
    <th>LS Mean (M)</th>
    <th>LS Mean (F)</th>
    <th>Post-hoc p</th>
  </tr>
  <tr>
    <td>binoculars</td>
    <td>792</td>
    <td>-4.7853</td>
    <td>1.71e-06</td>
    <td>116529.44</td>
    <td>Yes</td>
    <td>0.9425</td>
    <td>0.9410</td>
    <td>0.701</td>
  </tr>
  <tr>
    <td>chatgpt-roberta</td>
    <td>792</td>
    <td>-10.3512</td>
    <td>4.24e-25</td>
    <td>114663.56</td>
    <td>Yes</td>
    <td>0.7489</td>
    <td>0.7433</td>
    <td>0.405</td>
  </tr>
  <tr>
    <td>detectgpt</td>
    <td>792</td>
    <td>29.4391</td>
    <td>8.78e-190</td>
    <td>115554.43</td>
    <td>Yes</td>
    <td>0.7885</td>
    <td>0.8002</td>
    <td>0.018*</td>
  </tr>
  <tr>
    <td>fastdetectgpt</td>
    <td>1584</td>
    <td>7.6643</td>
    <td>1.81e-14</td>
    <td>221280.83</td>
    <td>Yes</td>
    <td>0.8961</td>
    <td>0.8985</td>
    <td>0.534</td>
  </tr>
  <tr>
    <td>fastdetectllm</td>
    <td>792</td>
    <td>-13.2158</td>
    <td>7.60e-40</td>
    <td>116208.70</td>
    <td>Yes</td>
    <td>0.4806</td>
    <td>0.4792</td>
    <td>0.289</td>
  </tr>
  <tr>
    <td>gltr</td>
    <td>792</td>
    <td>7.5210</td>
    <td>5.48e-14</td>
    <td>113707.20</td>
    <td>Yes</td>
    <td>0.8138</td>
    <td>0.8192</td>
    <td>0.544</td>
  </tr>
  <tr>
    <td>gpt2-base</td>
    <td>792</td>
    <td>-16.5020</td>
    <td>4.19e-61</td>
    <td>112626.50</td>
    <td>Yes</td>
    <td>0.6282</td>
    <td>0.6194</td>
    <td>0.182</td>
  </tr>
  <tr>
    <td>gpt2-large</td>
    <td>792</td>
    <td>-11.9331</td>
    <td>8.32e-33</td>
    <td>115088.05</td>
    <td>Yes</td>
    <td>0.5623</td>
    <td>0.5557</td>
    <td>0.337</td>
  </tr>
  <tr>
    <td>llmdet</td>
    <td>792</td>
    <td>-31.0913</td>
    <td>2.18e-211</td>
    <td>120438.22</td>
    <td>Yes</td>
    <td>0.5020</td>
    <td>0.4971</td>
    <td>0.014*</td>
  </tr>
  <tr>
    <td>radar</td>
    <td>792</td>
    <td>-1.0418</td>
    <td>0.298</td>
    <td>118146.32</td>
    <td>No</td>
    <td>–</td>
    <td>–</td>
    <td>–</td>
  </tr>
</table>

Table 13: Down-Sample Matched Results: Weighted T-tests on “Sex” (M vs F) for each detector.“Sig?” refers to overall significance of the T-test (\( p < 0.05 \)). An asterisk (*) in “Post-hoc p” indicates a significant difference (Holm-adjusted) specifically for M vs F. A “–” indicates that the overall T-test for that factor and detector was not statistically significant, so LSMeans are not reported.

<table>
  <tr>
    <th>Factor</th>
    <th>Detector</th>
    <th>Comparison</th>
    <th>Sig.?</th>
  </tr>
  <tr>
    <td rowspan="4">Acad. Genre</td>
    <td>detectgpt</td>
    <td>Hum. vs S.&T.</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td></td>
    <td>Hum. vs Life S.</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td>fastdetectllm</td>
    <td>Hum. vs S.&T.</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td></td>
    <td>Soc. S. vs Life S.</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td></td>
    <td>gpt2-base</td>
    <td>S.&T. vs Hum.</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td></td>
    <td></td>
    <td>S.&T. vs Soc. S.</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td></td>
    <td>gpt2-large</td>
    <td>S.&T. vs Hum.</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td></td>
    <td></td>
    <td>S.&T. vs Soc. S.</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td></td>
    <td>Ilmdet</td>
    <td>Life S. vs Hum.</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td></td>
    <td></td>
    <td>S.&T. vs Hum.</td>
    <td>Yes</td>
  </tr>
  <tr>
    <td></td>
    <td></td>
    <td>Soc. S. vs Hum.</td>
    <td>Yes</td>
  </tr>
</table>

Table 14: Down-Sample Matched Results: Significant post-hoc pairs (Holm-adjusted) among academic genres. “Hum.”, “Soc. S.”, “S.&T.”, “Life S.” abbreviate Humanities, Social Sciences, Sciences & Technology, and Life Sciences, respectively. Only detectors that showed a significant overall ANOVA in Table 12 are included.

<table>
  <tr>
    <th>Generator Model</th>
    <th>binoculars</th>
    <th>fastdetectgpt</th>
    <th>detectgpt</th>
    <th>gltr</th>
    <th>chatgpt-roberta</th>
    <th>radar</th>
    <th>gpt2-base</th>
    <th>gpt2-large</th>
    <th>llmdet</th>
    <th>fastdetectllm</th>
  </tr>
  <tr>
    <td>Qwen2.5-7B-Instruct</td>
    <td>0.959</td>
    <td>0.967</td>
    <td>0.897</td>
    <td>0.939</td>
    <td>0.808</td>
    <td>0.822</td>
    <td>0.653</td>
    <td>0.531</td>
    <td>0.496</td>
    <td>0.479</td>
  </tr>
  <tr>
    <td>Qwen2.5-32B-Instruct</td>
    <td>0.958</td>
    <td>0.906</td>
    <td>0.789</td>
    <td>0.789</td>
    <td>0.691</td>
    <td>0.518</td>
    <td>0.483</td>
    <td>0.487</td>
    <td>0.479</td>
    <td>0.479</td>
  </tr>
  <tr>
    <td>Qwen2.5-14B-Instruct</td>
    <td>0.958</td>
    <td>0.896</td>
    <td>0.751</td>
    <td>0.836</td>
    <td>0.874</td>
    <td>0.802</td>
    <td>0.579</td>
    <td>0.496</td>
    <td>0.493</td>
    <td>0.479</td>
  </tr>
  <tr>
    <td>Qwen2.5-7B-Instruct</td>
    <td>0.958</td>
    <td>0.964</td>
    <td>0.840</td>
    <td>0.884</td>
    <td>0.822</td>
    <td>0.700</td>
    <td>0.525</td>
    <td>0.485</td>
    <td>0.487</td>
    <td>0.479</td>
  </tr>
  <tr>
    <td>Qwen2.5-3B-Instruct</td>
    <td>0.946</td>
    <td>0.735</td>
    <td>0.693</td>
    <td>0.603</td>
    <td>0.774</td>
    <td>0.562</td>
    <td>0.506</td>
    <td>0.485</td>
    <td>0.497</td>
    <td>0.479</td>
  </tr>
  <tr>
    <td>Qwen2.5-1.5B-Instruct</td>
    <td>0.925</td>
    <td>0.876</td>
    <td>0.786</td>
    <td>0.661</td>
    <td>0.733</td>
    <td>0.647</td>
    <td>0.571</td>
    <td>0.534</td>
    <td>0.500</td>
    <td>0.480</td>
  </tr>
  <tr>
    <td>Qwen2.5-0.5B-Instruct</td>
    <td>0.825</td>
    <td>0.873</td>
    <td>0.762</td>
    <td>0.621</td>
    <td>0.651</td>
    <td>0.622</td>
    <td>0.654</td>
    <td>0.779</td>
    <td>0.499</td>
    <td>0.480</td>
  </tr>
  <tr>
    <td>llama2-7b-instruct</td>
    <td>0.950</td>
    <td>0.905</td>
    <td>0.802</td>
    <td>0.871</td>
    <td>0.759</td>
    <td>0.721</td>
    <td>0.637</td>
    <td>0.540</td>
    <td>0.483</td>
    <td>0.483</td>
  </tr>
  <tr>
    <td>llama3.1-18b-instruct</td>
    <td>0.955</td>
    <td>0.935</td>
    <td>0.812</td>
    <td>0.902</td>
    <td>0.890</td>
    <td>0.825</td>
    <td>0.672</td>
    <td>0.572</td>
    <td>0.505</td>
    <td>0.479</td>
  </tr>
  <tr>
    <td>llama3.2-3b-instruct</td>
    <td>0.924</td>
    <td>0.907</td>
    <td>0.776</td>
    <td>0.831</td>
    <td>0.649</td>
    <td>0.728</td>
    <td>0.635</td>
    <td>0.559</td>
    <td>0.519</td>
    <td>0.483</td>
  </tr>
  <tr>
    <td>llama3.2-1b-instruct</td>
    <td>0.824</td>
    <td>0.795</td>
    <td>0.715</td>
    <td>0.687</td>
    <td>0.706</td>
    <td>0.631</td>
    <td>0.671</td>
    <td>0.668</td>
    <td>0.512</td>
    <td>0.527</td>
  </tr>
  <tr>
    <td>Mistral-Small-Instruct-2409</td>
    <td>0.959</td>
    <td>0.947</td>
    <td>0.767</td>
    <td>0.773</td>
    <td>0.685</td>
    <td>0.630</td>
    <td>0.524</td>
    <td>0.481</td>
    <td>0.497</td>
    <td>0.479</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>0.927</td>
    <td>0.894</td>
    <td>0.784</td>
    <td>0.783</td>
    <td>0.745</td>
    <td>0.690</td>
    <td>0.595</td>
    <td>0.551</td>
    <td>0.502</td>
    <td>0.484</td>
  </tr>
</table>

Table 15: Overall Detector performance across various generator models without subsetting
<table>
  <tr>
    <th>Generator Model</th>
    <th>binoculars</th>
    <th>fastdetectgpt</th>
    <th>detetectgpt</th>
    <th>glt</th>
    <th>chatgpt-roberta</th>
    <th>radar</th>
    <th>gpt2-base</th>
    <th>gpt2-large</th>
    <th>llmdet</th>
    <th>fastdetectilm</th>
  </tr>
  <tr><td colspan="11">Qwen2.5-72B-Instruct</td></tr>
  <tr><td>Qwen2.5-32b-Instruct</td><td>0.990</td><td>0.915</td><td>0.930</td><td>0.801</td><td>0.858</td><td>0.759</td><td>0.620</td><td>0.480</td><td>0.497</td><td>0.483</td></tr>
  <tr><td>Qwen2.5-14B-Instruct</td><td>0.989</td><td>0.918</td><td>0.933</td><td>0.781</td><td>0.947</td><td>0.757</td><td>0.551</td><td>0.470</td><td>0.491</td><td>0.486</td></tr>
  <tr><td>Qwen2.5-7B-Instruct</td><td>0.990</td><td>0.965</td><td>0.945</td><td>0.837</td><td>0.707</td><td>0.619</td><td>0.514</td><td>0.483</td><td>0.476</td><td>0.473</td></tr>
  <tr><td>Qwen2.5-3B-Instruct</td><td>0.976</td><td>0.799</td><td>0.722</td><td>0.717</td><td>0.717</td><td>0.701</td><td>0.485</td><td>0.468</td><td>0.466</td><td>0.464</td></tr>
  <tr><td>Qwen2.5-1.5B-Instruct</td><td>0.959</td><td>0.894</td><td>0.750</td><td>0.836</td><td>0.794</td><td>0.677</td><td>0.594</td><td>0.524</td><td>0.511</td><td>0.489</td></tr>
  <tr><td>Mistral-Small-Instruct-2409</td><td>0.870</td><td>0.878</td><td>0.683</td><td>0.802</td><td>0.671</td><td>0.626</td><td>0.673</td><td>0.722</td><td>0.513</td><td>0.489</td></tr>
  <tr><td>llama3.1.70B-instruct</td><td>0.944</td><td>0.856</td><td>0.860</td><td>0.743</td><td>0.651</td><td>0.610</td><td>0.674</td><td>0.552</td><td>0.575</td><td>0.481</td></tr>
  <tr><td>llama3.1-8B-instruct</td><td>0.985</td><td>0.955</td><td>0.954</td><td>0.836</td><td>0.674</td><td>0.716</td><td>0.784</td><td>0.665</td><td>0.526</td><td>0.489</td></tr>
  <tr><td>llama3.2-3B-instruct</td><td>0.944</td><td>0.889</td><td>0.870</td><td>0.755</td><td>0.644</td><td>0.692</td><td>0.679</td><td>0.572</td><td>0.583</td><td>0.492</td></tr>
  <tr><td>llama3.2-1B-instruct</td><td>0.839</td><td>0.765</td><td>0.720</td><td>0.707</td><td>0.692</td><td>0.660</td><td>0.702</td><td>0.696</td><td>0.532</td><td>0.543</td></tr>
  <tr><td>Average</td><td>0.956</td><td>0.891</td><td>0.845</td><td>0.789</td><td>0.757</td><td>0.694</td><td>0.664</td><td>0.565</td><td>0.520</td><td>0.493</td></tr>
</table>

Table 16: Detector performance across various generator models under subgroup CEFR = A2-0

<table>
  <tr>
    <th>Generator Model</th>
    <th>binoculars</th>
    <th>fastdetectgpt</th>
    <th>detetectgpt</th>
    <th>glt</th>
    <th>chatgpt-roberta</th>
    <th>radar</th>
    <th>gpt2-base</th>
    <th>gpt2-large</th>
    <th>llmdet</th>
    <th>fastdetectilm</th>
  </tr>
  <tr><td colspan="11">Qwen2.5-72B-Instruct</td></tr>
  <tr><td>Qwen2.5-32b-Instruct</td><td>0.981</td><td>0.976</td><td>0.987</td><td>0.932</td><td>0.843</td><td>0.909</td><td>0.774</td><td>0.572</td><td>0.502</td><td>0.475</td></tr>
  <tr><td>Qwen2.5-14B-Instruct</td><td>0.981</td><td>0.942</td><td>0.891</td><td>0.817</td><td>0.818</td><td>0.789</td><td>0.588</td><td>0.488</td><td>0.483</td><td>0.475</td></tr>
  <tr><td>Qwen2.5-7B-Instruct</td><td>0.981</td><td>0.908</td><td>0.925</td><td>0.786</td><td>0.926</td><td>0.849</td><td>0.658</td><td>0.514</td><td>0.505</td><td>0.475</td></tr>
  <tr><td>Qwen2.5-3B-Instruct</td><td>0.981</td><td>0.973</td><td>0.956</td><td>0.867</td><td>0.856</td><td>0.805</td><td>0.605</td><td>0.500</td><td>0.490</td><td>0.475</td></tr>
  <tr><td>Qwen2.5-1.5B-Instruct</td><td>0.952</td><td>0.879</td><td>0.737</td><td>0.603</td><td>0.782</td><td>0.682</td><td>0.619</td><td>0.558</td><td>0.505</td><td>0.476</td></tr>
  <tr><td>Mistral-Small-Instruct-2409</td><td>0.850</td><td>0.888</td><td>0.670</td><td>0.811</td><td>0.670</td><td>0.643</td><td>0.674</td><td>0.788</td><td>0.493</td><td>0.475</td></tr>
  <tr><td>llama3.1-70B-instruct</td><td>0.922</td><td>0.862</td><td>0.872</td><td>0.751</td><td>0.682</td><td>0.668</td><td>0.692</td><td>0.585</td><td>0.560</td><td>0.482</td></tr>
  <tr><td>llama3.1-8B-Instruct</td><td>0.974</td><td>0.961</td><td>0.951</td><td>0.827</td><td>0.665</td><td>0.734</td><td>0.781</td><td>0.639</td><td>0.523</td><td>0.475</td></tr>
  <tr><td>llama3.2-3B-Instruct</td><td>0.916</td><td>0.868</td><td>0.809</td><td>0.713</td><td>0.589</td><td>0.687</td><td>0.684</td><td>0.595</td><td>0.542</td><td>0.483</td></tr>
  <tr><td>llama3.2-1B-Instruct</td><td>0.832</td><td>0.781</td><td>0.710</td><td>0.722</td><td>0.682</td><td>0.661</td><td>0.692</td><td>0.697</td><td>0.524</td><td>0.513</td></tr>
  <tr><td>Average</td><td>0.942</td><td>0.891</td><td>0.837</td><td>0.798</td><td>0.751</td><td>0.724</td><td>0.655</td><td>0.576</td><td>0.511</td><td>0.479</td></tr>
</table>

Table 17: Detector performance across various generator models under subgroup CEFR = B1-1

<table>
  <tr>
    <th>Generator Model</th>
    <th>binoculars</th>
    <th>fastdetectgpt</th>
    <th>detetectgpt</th>
    <th>glt</th>
    <th>chatgpt-roberta</th>
    <th>radar</th>
    <th>gpt2-base</th>
    <th>gpt2-large</th>
    <th>llmdet</th>
    <th>fastdetectilm</th>
  </tr>
  <tr><td colspan="11">Qwen2.5-72B-Instruct</td></tr>
  <tr><td>Qwen2.5-32b-Instruct</td><td>0.974</td><td>0.981</td><td>0.982</td><td>0.913</td><td>0.812</td><td>0.870</td><td>0.658</td><td>0.521</td><td>0.496</td><td>0.466</td></tr>
  <tr><td>Qwen2.5-14B-Instruct</td><td>0.974</td><td>0.929</td><td>0.843</td><td>0.811</td><td>0.800</td><td>0.669</td><td>0.514</td><td>0.488</td><td>0.484</td><td>0.466</td></tr>
  <tr><td>Qwen2.5-7B-Instruct</td><td>0.974</td><td>0.895</td><td>0.857</td><td>0.763</td><td>0.895</td><td>0.801</td><td>0.574</td><td>0.497</td><td>0.496</td><td>0.466</td></tr>
  <tr><td>Qwen2.5-3B-Instruct</td><td>0.974</td><td>0.979</td><td>0.921</td><td>0.854</td><td>0.827</td><td>0.729</td><td>0.526</td><td>0.491</td><td>0.488</td><td>0.466</td></tr>
  <tr><td>Qwen2.5-1.5B-Instruct</td><td>0.943</td><td>0.725</td><td>0.604</td><td>0.711</td><td>0.770</td><td>0.547</td><td>0.506</td><td>0.485</td><td>0.485</td><td>0.466</td></tr>
  <tr><td>Mistral-Small-Instruct-2409</td><td>0.838</td><td>0.889</td><td>0.649</td><td>0.774</td><td>0.647</td><td>0.642</td><td>0.652</td><td>0.772</td><td>0.493</td><td>0.468</td></tr>
  <tr><td>llama3.1-70B-instruct</td><td>0.949</td><td>0.925</td><td>0.841</td><td>0.788</td><td>0.795</td><td>0.750</td><td>0.663</td><td>0.548</td><td>0.534</td><td>0.471</td></tr>
  <tr><td>llama3.1-8B-Instruct</td><td>0.971</td><td>0.972</td><td>0.954</td><td>0.852</td><td>0.897</td><td>0.761</td><td>0.679</td><td>0.565</td><td>0.546</td><td>0.466</td></tr>
  <tr><td>llama3.2-3B-Instruct</td><td>0.936</td><td>0.905</td><td>0.850</td><td>0.767</td><td>0.877</td><td>0.662</td><td>0.652</td><td>0.572</td><td>0.519</td><td>0.473</td></tr>
  <tr><td>llama3.2-1B-Instruct</td><td>0.852</td><td>0.822</td><td>0.715</td><td>0.743</td><td>0.724</td><td>0.623</td><td>0.678</td><td>0.671</td><td>0.509</td><td>0.516</td></tr>
  <tr><td>Average</td><td>0.944</td><td>0.964</td><td>0.841</td><td>0.788</td><td>0.685</td><td>0.657</td><td>0.523</td><td>0.488</td><td>0.492</td><td>0.466</td></tr>
</table>

Table 18: Detector performance across various generator models under subgroup CEFR = B1-2

<table>
  <tr>
    <th>Generator Model</th>
    <th>binoculars</th>
    <th>fastdetectgpt</th>
    <th>detetectgpt</th>
    <th>glt</th>
    <th>chatgpt-roberta</th>
    <th>radar</th>
    <th>gpt2-base</th>
    <th>gpt2-large</th>
    <th>llmdet</th>
    <th>fastdetectilm</th>
  </tr>
  <tr><td colspan="11">Qwen2.5-72B-Instruct</td></tr>
  <tr><td>Qwen2.5-32b-Instruct</td><td>0.961</td><td>0.980</td><td>0.950</td><td>0.902</td><td>0.705</td><td>0.781</td><td>0.538</td><td>0.507</td><td>0.467</td><td>0.474</td></tr>
  <tr><td>Qwen2.5-14B-Instruct</td><td>0.961</td><td>0.899</td><td>0.716</td><td>0.794</td><td>0.693</td><td>0.632</td><td>0.490</td><td>0.500</td><td>0.467</td><td>0.474</td></tr>
  <tr><td>Qwen2.5-7B-Instruct</td><td>0.961</td><td>0.900</td><td>0.807</td><td>0.744</td><td>0.818</td><td>0.794</td><td>0.531</td><td>0.500</td><td>0.468</td><td>0.474</td></tr>
  <tr><td>Qwen2.5-3B-Instruct</td><td>0.961</td><td>0.970</td><td>0.873</td><td>0.816</td><td>0.780</td><td>0.635</td><td>0.513</td><td>0.501</td><td>0.470</td><td>0.474</td></tr>
  <tr><td>Qwen2.5-1.5B-Instruct</td><td>0.936</td><td>0.897</td><td>0.653</td><td>0.775</td><td>0.684</td><td>0.629</td><td>0.559</td><td>0.544</td><td>0.476</td><td>0.474</td></tr>
  <tr><td>Mistral-Small-Instruct-2409</td><td>0.854</td><td>0.889</td><td>0.647</td><td>0.752</td><td>0.642</td><td>0.623</td><td>0.682</td><td>0.806</td><td>0.481</td><td>0.474</td></tr>
  <tr><td>llama3.1-70B-instruct</td><td>0.958</td><td>0.966</td><td>0.925</td><td>0.868</td><td>0.795</td><td>0.777</td><td>0.606</td><td>0.525</td><td>0.481</td><td>0.476</td></tr>
  <tr><td>llama3.1-8B-Instruct</td><td>0.960</td><td>0.971</td><td>0.903</td><td>0.860</td><td>0.697</td><td>0.743</td><td>0.607</td><td>0.523</td><td>0.469</td><td>0.474</td></tr>
  <tr><td>llama3.2-3B-Instruct</td><td>0.952</td><td>0.960</td><td>0.895</td><td>0.849</td><td>0.649</td><td>0.762</td><td>0.634</td><td>0.553</td><td>0.481</td><td>0.474</td></tr>
  <tr><td>llama3.2-1B-Instruct</td><td>0.848</td><td>0.837</td><td>0.736</td><td>0.729</td><td>0.705</td><td>0.634</td><td>0.692</td><td>0.679</td><td>0.486</td><td>0.523</td></tr>
  <tr><td>Average</td><td>0.941</td><td>0.956</td><td>0.776</td><td>0.771</td><td>0.663</td><td>0.628</td><td>0.505</td><td>0.500</td><td>0.470</td><td>0.474</td></tr>
</table>

Table 19: Detector performance across various generator models under subgroup CEFR = B2-0
<table>
  <tr>
    <th>Generator Model</th>
    <th>binoculars</th>
    <th>fastdetectgpt</th>
    <th>gltr</th>
    <th>detectgpt</th>
    <th>chatgpt-roberta</th>
    <th>radar</th>
    <th>gpt2-base</th>
    <th>gpt2-large</th>
    <th>llmnet</th>
    <th>fastdetecttlim</th>
  </tr>
  <tr><td>Qwen2.5-72B-Instruct</td><td>0.875</td><td>0.923</td><td>0.756</td><td>0.837</td><td>0.809</td><td>0.685</td><td>0.453</td><td>0.460</td><td>0.503</td><td>0.500</td></tr>
  <tr><td>Qwen2.5-32B-Instruct</td><td>0.875</td><td>0.825</td><td>0.527</td><td>0.711</td><td>0.773</td><td>0.598</td><td>0.395</td><td>0.457</td><td>0.509</td><td>0.500</td></tr>
  <tr><td>Qwen2.5-14B-Instruct</td><td>0.875</td><td>0.857</td><td>0.632</td><td>0.675</td><td>0.773</td><td>0.741</td><td>0.421</td><td>0.464</td><td>0.500</td><td>0.500</td></tr>
  <tr><td>Qwen2.5-7B-Instruct</td><td>0.875</td><td>0.922</td><td>0.694</td><td>0.799</td><td>0.809</td><td>0.545</td><td>0.410</td><td>0.457</td><td>0.500</td><td>0.500</td></tr>
  <tr><td>Qwen2.5-3B-Instruct</td><td>0.873</td><td>0.693</td><td>0.394</td><td>0.645</td><td>0.766</td><td>0.537</td><td>0.399</td><td>0.457</td><td>0.506</td><td>0.500</td></tr>
  <tr><td>Qwen2.5-1.5B-Instruct</td><td>0.824</td><td>0.815</td><td>0.451</td><td>0.730</td><td>0.689</td><td>0.603</td><td>0.500</td><td>0.521</td><td>0.521</td><td>0.503</td></tr>
  <tr><td>Qwen2.0-SB-Instruct</td><td>0.699</td><td>0.806</td><td>0.436</td><td>0.664</td><td>0.625</td><td>0.594</td><td>0.587</td><td>0.761</td><td>0.522</td><td>0.500</td></tr>
  <tr><td>Llama3.1-70b-Instruct</td><td>0.873</td><td>0.951</td><td>0.755</td><td>0.810</td><td>0.877</td><td>0.804</td><td>0.522</td><td>0.476</td><td>0.503</td><td>0.502</td></tr>
  <tr><td>Llama3.1-8h-Instruct</td><td>0.872</td><td>0.917</td><td>0.716</td><td>0.782</td><td>0.721</td><td>0.664</td><td>0.485</td><td>0.462</td><td>0.503</td><td>0.500</td></tr>
  <tr><td>Llama3.2-3h-Instruct</td><td>0.870</td><td>0.913</td><td>0.726</td><td>0.797</td><td>0.716</td><td>0.759</td><td>0.488</td><td>0.491</td><td>0.509</td><td>0.500</td></tr>
  <tr><td>Llama3.2-1h-Instruct</td><td>0.739</td><td>0.763</td><td>0.531</td><td>0.667</td><td>0.728</td><td>0.573</td><td>0.581</td><td>0.586</td><td>0.510</td><td>0.548</td></tr>
  <tr><td>Mistral-Small-Instruct-2409</td><td>0.875</td><td>0.914</td><td>0.578</td><td>0.699</td><td>0.693</td><td>0.590</td><td>0.394</td><td>0.457</td><td>0.506</td><td>0.500</td></tr>
  <tr><td>Average</td><td>0.844</td><td>0.855</td><td>0.600</td><td>0.735</td><td>0.748</td><td>0.642</td><td>0.470</td><td>0.504</td><td>0.507</td><td>0.504</td></tr>
</table>

Table 20: Detector performance across various generator models under subgroup CEFR = XX-0

<table>
  <tr>
    <th>Generator Model</th>
    <th>binoculars</th>
    <th>fastdetectgpt</th>
    <th>gltr</th>
    <th>detectgpt</th>
    <th>chatgpt-roberta</th>
    <th>radar</th>
    <th>gpt2-base</th>
    <th>gpt2-large</th>
    <th>llmnet</th>
    <th>fastdetecttlim</th>
  </tr>
  <tr><td>Qwen2.5-72B-Instruct</td><td>0.983</td><td>0.981</td><td>0.978</td><td>0.917</td><td>0.823</td><td>0.862</td><td>0.716</td><td>0.554</td><td>0.489</td><td>0.474</td></tr>
  <tr><td>Qwen2.5-32B-Instruct</td><td>0.983</td><td>0.932</td><td>0.866</td><td>0.808</td><td>0.817</td><td>0.725</td><td>0.547</td><td>0.478</td><td>0.477</td><td>0.474</td></tr>
  <tr><td>Qwen2.5-14B-Instruct</td><td>0.983</td><td>0.973</td><td>0.863</td><td>0.766</td><td>0.908</td><td>0.563</td><td>0.418</td><td>0.491</td><td>0.457</td><td>0.477</td></tr>
  <tr><td>Qwen2.5-7B-Instruct</td><td>0.979</td><td>0.973</td><td>0.829</td><td>0.860</td><td>0.829</td><td>0.879</td><td>0.540</td><td>0.478</td><td>0.478</td><td>0.474</td></tr>
  <tr><td>Qwen2.5-3B-Instruct</td><td>0.967</td><td>0.943</td><td>0.643</td><td>0.710</td><td>0.791</td><td>0.574</td><td>0.520</td><td>0.480</td><td>0.490</td><td>0.474</td></tr>
  <tr><td>Qwen2.5-1.5B-Instruct</td><td>0.956</td><td>0.909</td><td>0.717</td><td>0.803</td><td>0.749</td><td>0.667</td><td>0.587</td><td>0.531</td><td>0.489</td><td>0.475</td></tr>
  <tr><td>Qwen2.0-SB-Instructors</td><td>0.861</td><td>0.895</td><td>0.670</td><td>0.803</td><td>0.670</td><td>0.649</td><td>0.669</td><td>0.773</td><td>0.484</td><td>0.474</td></tr>
  <tr><td>Llama3.1-70b-instructor</td><td>0.943</td><td>0.897</td><td>0.885</td><td>0.787</td><td>0.716</td><td>0.696</td><td>0.547</td><td>0.540</td><td>0.480</td><td>0.469</td></tr>
  <tr><td>Llama3.1.8k-instruction</td><td>0.979</td><td>0.969</td><td>0.941</td><td>0.845</td><td>0.682</td><td>0.737</td><td>0.726</td><td>0.603</td><td>0.503</td><td>0.474</td></tr>
  <tr><td>Llama3.2.3k-instruction</td><td>0.936</td><td>0.899</td><td>0.842</td><td>0.761</td><td>0.627</td><td>0.714</td><td>0.660</td><td>0.565</td><td>0.521</td><td>0.481</td></tr>
  <tr><td>Llama3.2.1h-instruction</td><td>0.846</td><td>0.796</td><td>0.705</td><td>0.724</td><td>0.695</td><td>0.658</td><td>0.678</td><td>0.681</td><td>0.506</td><td>0.516</td></tr>
  <tr><td>Mistral-Small-Instruct-2409</td><td>0.983</td><td>0.957</td><td>0.816</td><td>0.794</td><td>0.675</td><td>0.644</td><td>0.552</td><td>0.477</td><td>0.491</td><td>0.474</td></tr>
  <tr><td>Average</td><td>0.950</td><td>0.905</td><td>0.823</td><td>0.797</td><td>0.748</td><td>0.709</td><td>0.623</td><td>0.556</td><td>0.496</td><td>0.479</td></tr>
</table>

Table 21: Detector performance across various generator models under subgroup Language Environment = EFL

<table>
  <tr>
    <th>Generator Model</th>
    <th>binoculars</th>
    <th>fastdetectgpt</th>
    <th>gltr</th>
    <th>detectgpt</th>
    <th>chatgpt-roberta</th>
    <th>radar</th>
    <th>gpt2-base</th>
    <th>gpt2-large</th>
    <th>llmnet</th>
    <th>fastdetecttlim</th>
  </tr>
  <tr><td>Qwen2.5-72B-Instruct</td><td>0.967</td><td>0.967</td><td>0.977</td><td>0.904</td><td>0.787</td><td>0.836</td><td>0.665</td><td>0.534</td><td>0.501</td><td>0.477</td></tr>
  <tr><td>Qwen2.5-32B-Instruct</td><td>0.966</td><td>0.907</td><td>0.813</td><td>0.816</td><td>0.757</td><td>0.691</td><td>0.535</td><td>0.505</td><td>0.490</td><td>0.477</td></tr>
  <tr><td>Qwen2.5-14B-Instruct</td><td>0.967</td><td>0.890</td><td>0.874</td><td>0.777</td><td>0.816</td><td>0.805</td><td>0.605</td><td>0.554</td><td>0.498</td><td>0.477</td></tr>
  <tr><td>Qwen2.5-7B-Instruct</td><td>0.963</td><td>0.906</td><td>0.915</td><td>0.833</td><td>0.823</td><td>0.686</td><td>0.555</td><td>0.507</td><td>0.494</td><td>0.477</td></tr>
  <tr><td>Qwen2.5-3B-Instruct</td><td>0.957</td><td>0.739</td><td>0.654</td><td>0.698</td><td>0.750</td><td>0.567</td><td>0.544</td><td>0.509</td><td>0.501</td><td>0.477</td></tr>
  <tr><td>Qwen2.5-1.5B-Instruct</td><td>0.935</td><td>0.869</td><td>0.690</td><td>0.793</td><td>0.732</td><td>0.643</td><td>0.585</td><td>0.549</td><td>0.507</td><td>0.477</td></tr>
  <tr><td>Qwen2.0-SB-Instructions</td><td>0.839</td><td>0.873</td><td>0.648</td><td>0.758</td><td>0.640</td><td>0.600</td><td>0.670</td><td>0.804</td><td>0.510</td><td>0.478</td></tr>
  <tr><td>Llama3.1-8h-instruction</td><td>0.942</td><td>0.908</td><td>0.911</td><td>0.824</td><td>0.764</td><td>0.716</td><td>0.682</td><td>0.636</td><td>0.535</td><td>0.479</td></tr>
  <tr><td>Llama3.1.8k-instruction</td><td>0.963</td><td>0.959</td><td>0.840</td><td>0.785</td><td>0.690</td><td>0.746</td><td>0.591</td><td>0.584</td><td>0.470</td><td>0.477</td></tr>
  <tr><td>Llama3.1-18h-instruction</td><td>0.936</td><td>0.927</td><td>0.808</td><td>0.764</td><td>0.681</td><td>0.673</td><td>0.576</td><td>0.551</td><td>0.479</td><td>0.473</td></tr>
  <tr><td>Llama3.2.1h-instruction</td><td>0.854</td><td>0.814</td><td>0.740</td><td>0.732</td><td>0.712</td><td>0.623</td><td>0.710</td><td>0.692</td><td>0.521</td><td>0.533</td></tr>
  <tr><td>Mistral-Small-Instruct-2409</td><td>0.967</td><td>0.948</td><td>0.740</td><td>0.732</td><td>0.712</td><td>0.623</td><td>0.710</td><td>0.692</td><td>0.521</td><td>0.619</td></tr>
  <tr><td>Average</td><td>0.937</td><td>0.896</td><td>0.821</td><td>0.795</td><td>0.741</td><td>0.690</td><td>0.620</td><td>0.570</td><td>0.507</td><td>0.489</td></tr>
</table>

Table 22: Detector performance across various generator models under subgroup Language Environment = ESL

<table>
  <tr>
    <th>Generator Model</th>
    <th>binoculars</th>
    <th>fastdetectgpt</th>
    <th>chatgpt-roberta</th>
    <th>detectgpt</th>
    <th>radar</th>
    <th>gltr</th>
    <th>gpt2-large</th>
    <th>gpt2-base</th>
    <th>llmnet</th>
    <th>fastdetecttlim</th>
  </tr>
  <tr><td>Qwen2.5-72B-Instruct</td><td>0.875</td><td>0.923</td><td>0.839</td><td>0.837</td><td>0.685</td><td>0.756</td><td>0.460</td><td>0.453</td><td>0.503</td><td>0.500</td></tr>
  <tr><td>Qwen2.5-32B-Instruct</td><td>0.875</td><td>0.823</td><td>0.773</td><td>0.711</td><td>0.598</td><td>0.527</td><td>0.457</td><td>0.395</td><td>0.509</td><td>0.500</td></tr>
  <tr><td>Qwen2.5-14B-Instruct</td><td>0.875</td><td>0.857</td><td>0.773</td><td>0.675</td><td>0.741</td><td>0.632</td><td>0.464</td><td>0.421</td><td>0.500</td><td>0.500</td></tr>
  <tr><td>Qwen2.5-7B-Instruct</td><td>0.875</td><td>0.922</td><td>0.809</td><td>0.799</td><td>0.545</td><td>0.694</td><td>0.457</td><td>0.410</td><td>0.500</td><td>0.500</td></tr>
  <tr><td>Qwen2.5-3B-Instruct</td><td>0.875</td><td>0.693</td><td>0.766</td><td>0.645</td><td>0.537</td><td>0.394</td><td>0.457</td><td>0.399</td><td>0.506</td><td>0.500</td></tr>
  <tr><td>Qwen2.5-1.5B-Instruct</td><td>0.824</td><td>0.815</td><td>0.698</td><td>0.730</td><td>0.603</td><td>0.451</td><td>0.521</td><td>0.500</td><td>0.521</td><td>0.503</td></tr>
  <tr><td>Qwen2.5-0.5B-Instruct</td><td>0.699</td><td>0.806</td><td>0.625</td><td>0.664</td><td>0.594</td><td>0.436</td><td>0.761</td><td>0.587</td><td>0.522</td><td>0.500</td></tr>
  <tr><td>Llama3.1-70b-Instruct</td><td>0.873</td><td>0.915</td><td>0.877</td><td>0.810</td><td>0.804</td><td>0.755</td><td>0.476</td><td>0.522</td><td>0.503</td><td>0.502</td></tr>
  <tr><td>Llama3.1.8h-Instruct</td><td>0.872</td><td>0.971</td><td>0.721</td><td>0.782</td><td>0.664</td><td>0.462</td><td>0.485</td><td>0.503</td><td>0.500</td><td>0.500</td></tr>
  <tr><td>Llama3.2.3h-Instruct</td><td>0.870</td><td>0.913</td><td>0.716</td><td>0.797</td><td>0.759</td><td>0.726</td><td>0.491</td><td>0.489</td><td>0.509</td><td>0.500</td></tr>
  <tr><td>Llama3.2.1h-Instruct</td><td>0.739</td><td>0.763</td><td>0.728</td><td>0.667</td><td>0.573</td><td>0.531</td><td>0.586</td><td>0.581</td><td>0.510</td><td>0.548</td></tr>
  <tr><td>Mistral-Small-Instruct-2409</td><td>0.875</td><td>0.914</td><td>0.693</td><td>0.697</td><td>0.599</td><td>0.576</td><td>0.457</td><td>0.394</td><td>0.500</td><td>0.500</td></tr>
  <tr><td>Average</td><td>0.844</td><td>0.855</td><td>0.748</td><td>0.735</td><td>0.642</td><td>0.600</td><td>0.504</td><td>0.470</td><td>0.507</td><td>0.504</td></tr>
</table>

Table 23: Detector performance across various generator models under subgroup Language Environment = NS
<table>
  <tr>
    <th>Generator Model</th>
    <th>binoculars</th>
    <th>fastdetectgpt</th>
    <th>gltr</th>
    <th>detectgpt</th>
    <th>chatgpt-roberta</th>
    <th>radar</th>
    <th>gpt2-base</th>
    <th>gpt2-large</th>
    <th>llmdet</th>
    <th>fastdetectlilm</th>
  </tr>
  <tr><td>Qwen2.5-7B2-Instruct</td><td>0.951</td><td>0.963</td><td>0.937</td><td>0.903</td><td>0.820</td><td>0.814</td><td>0.649</td><td>0.538</td><td>0.495</td><td>0.479</td></tr>
  <tr><td>Qwen2.5-32B-Instruct</td><td>0.950</td><td>0.892</td><td>0.788</td><td>0.793</td><td>0.783</td><td>0.681</td><td>0.513</td><td>0.484</td><td>0.484</td><td>0.479</td></tr>
  <tr><td>Qwen2.5-14B-Instruct</td><td>0.951</td><td>0.895</td><td>0.839</td><td>0.757</td><td>0.877</td><td>0.805</td><td>0.574</td><td>0.500</td><td>0.492</td><td>0.479</td></tr>
  <tr><td>Qwen2.5-7B-Instruct</td><td>0.951</td><td>0.959</td><td>0.880</td><td>0.841</td><td>0.807</td><td>0.705</td><td>0.521</td><td>0.485</td><td>0.484</td><td>0.479</td></tr>
  <tr><td>Qwen2.5-3B-Instruct</td><td>0.938</td><td>0.733</td><td>0.603</td><td>0.690</td><td>0.775</td><td>0.558</td><td>0.510</td><td>0.490</td><td>0.494</td><td>0.479</td></tr>
  <tr><td>Qwen2.5-1.5B-Instruct</td><td>0.919</td><td>0.871</td><td>0.676</td><td>0.785</td><td>0.736</td><td>0.645</td><td>0.579</td><td>0.544</td><td>0.498</td><td>0.480</td></tr>
  <tr><td>Qwen2.5-0.5B-Instruct</td><td>0.817</td><td>0.867</td><td>0.626</td><td>0.770</td><td>0.650</td><td>0.620</td><td>0.652</td><td>0.779</td><td>0.499</td><td>0.480</td></tr>
  <tr><td>Llama3.1-70B-instruct</td><td>0.926</td><td>0.903</td><td>0.882</td><td>0.813</td><td>0.764</td><td>0.722</td><td>0.639</td><td>0.547</td><td>0.523</td><td>0.483</td></tr>
  <tr><td>Llama3.1-8B-instruct</td><td>0.946</td><td>0.951</td><td>0.900</td><td>0.836</td><td>0.679</td><td>0.714</td><td>0.671</td><td>0.571</td><td>0.500</td><td>0.479</td></tr>
  <tr><td>Llama3.2-3B-instruct</td><td>0.921</td><td>0.903</td><td>0.832</td><td>0.786</td><td>0.642</td><td>0.727</td><td>0.628</td><td>0.562</td><td>0.517</td><td>0.483</td></tr>
  <tr><td>Llama3.2-1B-instruct</td><td>0.834</td><td>0.805</td><td>0.699</td><td>0.716</td><td>0.708</td><td>0.630</td><td>0.670</td><td>0.671</td><td>0.510</td><td>0.521</td></tr>
  <tr><td>Mistral-Small-Instruct-2409</td><td>0.951</td><td>0.941</td><td>0.776</td><td>0.787</td><td>0.675</td><td>0.619</td><td>0.518</td><td>0.482</td><td>0.494</td><td>0.479</td></tr>
  <tr><td>Average</td><td>0.921</td><td>0.890</td><td>0.787</td><td>0.790</td><td>0.743</td><td>0.687</td><td>0.594</td><td>0.554</td><td>0.499</td><td>0.483</td></tr>
</table>

Table 24: Detector performance across various generator models under subgroup Sex = Female

<table>
  <tr>
    <th>Generator Model</th>
    <th>binoculars</th>
    <th>fastdetectgpt</th>
    <th>gltr</th>
    <th>detectgpt</th>
    <th>chatgpt-roberta</th>
    <th>radar</th>
    <th>gpt2-base</th>
    <th>gpt2-large</th>
    <th>llmdep</th>
    <th>fastdetectlilm</th>
  </tr>
  <tr><td>Qwen2.5-7B2-Instruct</td><td>0.968</td><td>0.972</td><td>0.942</td><td>0.891</td><td>0.793</td><td>0.834</td><td>0.657</td><td>0.522</td><td>0.496</td><td>0.479</td></tr>
  <tr><td>Qwen2.5-32B-Instruct</td><td>0.968</td><td>0.921</td><td>0.791</td><td>0.788</td><td>0.797</td><td>0.704</td><td>0.523</td><td>0.482</td><td>0.492</td><td>0.479</td></tr>
  <tr><td>Qwen2.5-14B-Instruct</td><td>0.967</td><td>0.899</td><td>0.843</td><td>0.747</td><td>0.868</td><td>0.798</td><td>0.584</td><td>0.495</td><td>0.496</td><td>0.479</td></tr>
  <tr><td>Qwen2.5-7B-Instruct</td><td>0.968</td><td>0.909</td><td>0.890</td><td>0.837</td><td>0.839</td><td>0.694</td><td>0.529</td><td>0.485</td><td>0.491</td><td>0.479</td></tr>
  <tr><td>Qwen2.5-3B-Instruct</td><td>0.936</td><td>0.736</td><td>0.606</td><td>0.769</td><td>0.770</td><td>0.570</td><td>0.503</td><td>0.480</td><td>0.479</td><td>0.479</td></tr>
  <tr><td>Qwen2.5-1.5B-Instruct</td><td>0.933</td><td>0.881</td><td>0.644</td><td>0.788</td><td>0.729</td><td>0.651</td><td>0.560</td><td>0.524</td><td>0.504</td><td>0.480</td></tr>
  <tr><td>Qwen2.5-0.5B-Instruct</td><td>0.834</td><td>0.880</td><td>0.617</td><td>0.754</td><td>0.653</td><td>0.629</td><td>0.658</td><td>0.777</td><td>0.501</td><td>0.479</td></tr>
  <tr><td>Llama3.1-70B-Instruct</td><td>0.936</td><td>0.909</td><td>0.858</td><td>0.788</td><td>0.750</td><td>0.721</td><td>0.637</td><td>0.532</td><td>0.541</td><td>0.483</td></tr>
  <tr><td>Llama3.1-8B-Instruct</td><td>0.965</td><td>0.964</td><td>0.906</td><td>0.828</td><td>0.703</td><td>0.741</td><td>0.674</td><td>0.575</td><td>0.512</td><td>0.479</td></tr>
  <tr><td>Llama3.2-3B-Instruct</td><td>0.929</td><td>0.911</td><td>0.832</td><td>0.766</td><td>0.654</td><td>0.730</td><td>0.641</td><td>0.555</td><td>0.520</td><td>0.483</td></tr>
  <tr><td>Llama3.2-1B-Instruct</td><td>0.811</td><td>0.783</td><td>0.671</td><td>0.715</td><td>0.702</td><td>0.633</td><td>0.671</td><td>0.666</td><td>0.514</td><td>0.535</td></tr>
  <tr><td>Mistral-Small-Instruct-2409</td><td>0.968</td><td>0.954</td><td>0.771</td><td>0.742</td><td>0.699</td><td>0.644</td><td>0.533</td><td>0.480</td><td>0.502</td><td>0.479</td></tr>
  <tr><td>Average</td><td>0.934</td><td>0.898</td><td>0.780</td><td>0.779</td><td>0.746</td><td>0.696</td><td>0.597</td><td>0.548</td><td>0.506</td><td>0.485</td></tr>
</table>

Table 25: Detector performance across various generator models under subgroup Sex = Male

<table>
  <tr>
    <th>Generator Model</th>
    <th>binoculars</th>
    <th>fastdetectgpt</th>
    <th>gltr</th>
    <th>detectgpt</th>
    <th>chatgpt-roberta</th>
    <th>radar</th>
    <th>gpt2-base</th>
    <th>gpt2-large</th>
    <th>llmdep</th>
    <th>fastdetectlilm</th>
  </tr>
  <tr><td>Qwen2.5-7B2-Instruct</td><td>0.962</td><td>0.960</td><td>0.948</td><td>0.921</td><td>0.794</td><td>0.823</td><td>0.619</td><td>0.510</td><td>0.491</td><td>0.482</td></tr>
  <tr><td>Qwen2.5-32B-Instruct</td><td>0.962</td><td>0.889</td><td>0.771</td><td>0.799</td><td>0.759</td><td>0.669</td><td>0.489</td><td>0.483</td><td>0.485</td><td>0.482</td></tr>
  <tr><td>Qwen2.5-14B-Instruct</td><td>0.961</td><td>0.910</td><td>0.859</td><td>0.789</td><td>0.873</td><td>0.811</td><td>0.565</td><td>0.496</td><td>0.490</td><td>0.482</td></tr>
  <tr><td>Qwen2.5-7B-Instruct</td><td>0.962</td><td>0.957</td><td>0.890</td><td>0.845</td><td>0.806</td><td>0.705</td><td>0.525</td><td>0.487</td><td>0.484</td><td>0.482</td></tr>
  <tr><td>Qwen2.5-3B-Instruct</td><td>0.953</td><td>0.729</td><td>0.605</td><td>0.716</td><td>0.770</td><td>0.571</td><td>0.496</td><td>0.486</td><td>0.490</td><td>0.482</td></tr>
  <tr><td>Qwen2.5-1.5B-Instruct</td><td>0.927</td><td>0.854</td><td>0.640</td><td>0.777</td><td>0.703</td><td>0.629</td><td>0.556</td><td>0.531</td><td>0.498</td><td>0.482</td></tr>
  <tr><td>Qwen2.5-0.5B-Instruct</td><td>0.811</td><td>0.851</td><td>0.627</td><td>0.770</td><td>0.637</td><td>0.612</td><td>0.660</td><td>0.796</td><td>0.503</td><td>0.482</td></tr>
  <tr><td>Llama3.1-70B-instruct</td><td>0.934</td><td>0.890</td><td>0.885</td><td>0.799</td><td>0.755</td><td>0.723</td><td>0.613</td><td>0.528</td><td>0.528</td><td>0.487</td></tr>
  <tr><td>Llama3.1-8B-instruct</td><td>0.957</td><td>0.866</td><td>0.894</td><td>0.852</td><td>0.679</td><td>0.732</td><td>0.643</td><td>0.464</td><td>0.468</td><td>0.482</td></tr>
  <tr><td>Llama3.2-3B-instruct</td><td>0.938</td><td>0.906</td><td>0.841</td><td>0.810</td><td>0.651</td><td>0.743</td><td>0.624</td><td>0.552</td><td>0.508</td><td>0.486</td></tr>
  <tr><td>Llama3.2-1B-instruct</td><td>0.838</td><td>0.790</td><td>0.694</td><td>0.732</td><td>0.695</td><td>0.638</td><td>0.659</td><td>0.665</td><td>0.511</td><td>0.529</td></tr>
  <tr><td>Mistral-Small-Instruct-2409</td><td>0.962</td><td>0.935</td><td>0.757</td><td>0.791</td><td>0.677</td><td>0.626</td><td>0.497</td><td>0.484</td><td>0.499</td><td>0.482</td></tr>
  <tr><td>Average</td><td>0.931</td><td>0.885</td><td>0.784</td><td>0.800</td><td>0.733</td><td>0.690</td><td>0.579</td><td>0.547</td><td>0.499</td><td>0.486</td></tr>
</table>

Table 26: Detector performance across various generator models under subgroup Academic Genre = Humanities

<table>
  <tr>
    <th>Generator Model</th>
    <th>binoculars</th>
    <th>fastdetectgpt</th>
    <th>gltr</th>
    <th>detectgpt</th>
    <th>chatgpt-roberta</th>
    <th>radar</th>
    <th>gpt2-base</th>
    <th>gpt2-large</th>
    <th>llmdep</th>
    <th>fastdetectlilm</th>
  </tr>
  <tr><td>Qwen2.5-7B2-Instruct</td><td>0.951</td><td>0.976</td><td>0.931</td><td>0.868</td><td>0.852</td><td>0.777</td><td>0.683</td><td>0.562</td><td>0.496</td><td>0.476</td></tr>
  <tr><td>Qwen2.5-32B-Instruct</td><td>0.951</td><td>0.911</td><td>0.797</td><td>0.751</td><td>0.824</td><td>0.690</td><td>0.553</td><td>0.490</td><td>0.493</td><td>0.476</td></tr>
  <tr><td>Qwen2.5-14B-Instruct</td><td>0.951</td><td>0.890</td><td>0.810</td><td>0.697</td><td>0.864</td><td>0.775</td><td>0.589</td><td>0.502</td><td>0.497</td><td>0.476</td></tr>
  <tr><td>Qwen2.5-7B-Instruct</td><td>0.951</td><td>0.968</td><td>0.880</td><td>0.810</td><td>0.834</td><td>0.689</td><td>0.511</td><td>0.495</td><td>0.495</td><td>0.476</td></tr>
  <tr><td>Qwen2.5-3B-Instruct</td><td>0.939</td><td>0.751</td><td>0.611</td><td>0.668</td><td>0.756</td><td>0.516</td><td>0.518</td><td>0.494</td><td>0.498</td><td>0.476</td></tr>
  <tr><td>Qwen2.5-1.5B-Instruct</td><td>0.921</td><td>0.908</td><td>0.648</td><td>0.784</td><td>0.744</td><td>0.665</td><td>0.561</td><td>0.519</td><td>0.496</td><td>0.476</td></tr>
  <tr><td>Qwen2.5-0.5B-Instruct</td><td>0.819</td><td>0.863</td><td>0.613</td><td>0.723</td><td>0.660</td><td>0.642</td><td>0.671</td><td>0.781</td><td>0.495</td><td>0.479</td></tr>
  <tr><td>Llama3.1-70B-instruct</td><td>0.926</td><td>0.932</td><td>0.878</td><td>0.787</td><td>0.756</td><td>0.724</td><td>0.664</td><td>0.556</td><td>0.530</td><td>0.478</td></tr>
  <tr><td>Llama3.1-8B-instruct</td><td>0.948</td><td>0.972</td><td>0.909</td><td>0.816</td><td>0.703</td><td>0.694</td><td>0.683</td><td>0.593</td><td>0.504</td><td>0.476</td></tr>
  <tr><td>Llama3.2-3B-instruct</td><td>0.912</td><td>0.913</td><td>0.844</td><td>0.760</td><td>0.648</td><td>0.697</td><td>0.640</td><td>0.571</td><td>0.531</td><td>0.483</td></tr>
  <tr><td>Llama3.2-1B-instruct</td><td>0.806</td><td>0.785</td><td>0.681</td><td>0.683</td><td>0.737</td><td>0.600</td><td>0.673</td><td>0.671</td><td>0.513</td><td>0.529</td></tr>
  <tr><td>Mistral-Small-Instruct-2409</td><td>0.951</td><td>0.963</td><td>0.794</td><td>0.714</td><td>0.717</td><td>0.602</td><td>0.528</td><td>0.486</td><td>0.499</td><td>0.476</td></tr>
  <tr><td>Average</td><td>0.919</td><td>0.903</td><td>0.783</td><td>0.755</td><td>0.758</td><td>0.673</td><td>0.606</td><td>0.560</td><td>0.504</td><td>0.481</td></tr>
</table>

Table 27: Detector performance across various generator models under subgroup Academic Genre = Life Science
<table>
  <tr>
    <th>Generator Model</th>
    <th>binoculars</th>
    <th>fastdetectgpt</th>
    <th>gltr</th>
    <th>detectgpt</th>
    <th>chatgpt-roberta</th>
    <th>radar</th>
    <th>gpt2-base</th>
    <th>gpt2-large</th>
    <th>llmdet</th>
    <th>fastdetectlm</th>
  </tr>
  <tr>
    <td>Qwen2.5-72B-Instruct</td>
    <td>0.962</td>
    <td>0.958</td>
    <td>0.941</td>
    <td>0.887</td>
    <td>0.817</td>
    <td>0.826</td>
    <td>0.694</td>
    <td>0.559</td>
    <td>0.501</td>
    <td>0.477</td>
  </tr>
  <tr>
    <td>Qwen2.5-32B-Instruct</td>
    <td>0.962</td>
    <td>0.913</td>
    <td>0.814</td>
    <td>0.796</td>
    <td>0.797</td>
    <td>0.717</td>
    <td>0.543</td>
    <td>0.497</td>
    <td>0.486</td>
    <td>0.477</td>
  </tr>
  <tr>
    <td>Qwen2.5-14B-Instruct</td>
    <td>0.962</td>
    <td>0.883</td>
    <td>0.853</td>
    <td>0.761</td>
    <td>0.878</td>
    <td>0.806</td>
    <td>0.599</td>
    <td>0.514</td>
    <td>0.492</td>
    <td>0.477</td>
  </tr>
  <tr>
    <td>Qwen2.5-7B-Instruct</td>
    <td>0.962</td>
    <td>0.954</td>
    <td>0.893</td>
    <td>0.844</td>
    <td>0.847</td>
    <td>0.702</td>
    <td>0.543</td>
    <td>0.493</td>
    <td>0.484</td>
    <td>0.477</td>
  </tr>
  <tr>
    <td>Qwen2.5-1.5B-Instruct</td>
    <td>0.942</td>
    <td>0.870</td>
    <td>0.690</td>
    <td>0.761</td>
    <td>0.791</td>
    <td>0.591</td>
    <td>0.519</td>
    <td>0.496</td>
    <td>0.477</td>
    <td>0.477</td>
  </tr>
  <tr>
    <td>Qwen2.5-0.5B-Instruct</td>
    <td>0.841</td>
    <td>0.872</td>
    <td>0.626</td>
    <td>0.775</td>
    <td>0.658</td>
    <td>0.631</td>
    <td>0.649</td>
    <td>0.490</td>
    <td>0.490</td>
    <td>0.477</td>
  </tr>
  <tr>
    <td>llama3.1-70B-Instruct</td>
    <td>0.931</td>
    <td>0.894</td>
    <td>0.868</td>
    <td>0.802</td>
    <td>0.756</td>
    <td>0.708</td>
    <td>0.668</td>
    <td>0.568</td>
    <td>0.537</td>
    <td>0.480</td>
  </tr>
  <tr>
    <td>llama3.1-18B-Instruct</td>
    <td>0.960</td>
    <td>0.950</td>
    <td>0.913</td>
    <td>0.812</td>
    <td>0.681</td>
    <td>0.712</td>
    <td>0.717</td>
    <td>0.612</td>
    <td>0.511</td>
    <td>0.477</td>
  </tr>
  <tr>
    <td>llama3.2-3B-Instruct</td>
    <td>0.928</td>
    <td>0.895</td>
    <td>0.827</td>
    <td>0.751</td>
    <td>0.642</td>
    <td>0.733</td>
    <td>0.644</td>
    <td>0.580</td>
    <td>0.521</td>
    <td>0.489</td>
  </tr>
  <tr>
    <td>llama3.2-1B-Instruct</td>
    <td>0.816</td>
    <td>0.776</td>
    <td>0.674</td>
    <td>0.698</td>
    <td>0.702</td>
    <td>0.634</td>
    <td>0.680</td>
    <td>0.653</td>
    <td>0.518</td>
    <td>0.529</td>
  </tr>
  <tr>
    <td>Mistral-Small-Instruct-2409</td>
    <td>0.962</td>
    <td>0.937</td>
    <td>0.784</td>
    <td>0.766</td>
    <td>0.683</td>
    <td>0.649</td>
    <td>0.547</td>
    <td>0.489</td>
    <td>0.494</td>
    <td>0.477</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>0.930</td>
    <td>0.884</td>
    <td>0.790</td>
    <td>0.780</td>
    <td>0.750</td>
    <td>0.695</td>
    <td>0.615</td>
    <td>0.568</td>
    <td>0.503</td>
    <td>0.482</td>
  </tr>
</table>

Table 28: Detector performance across various generator models under subgroup Academic Genre = Science Technology

<table>
  <tr>
    <th>Generator Model</th>
    <th>binoculars</th>
    <th>fastdetectgpt</th>
    <th>gltr</th>
    <th>detectgpt</th>
    <th>chatgpt-roberta</th>
    <th>radar</th>
    <th>gpt2-base</th>
    <th>gpt2-large</th>
    <th>llmdet</th>
    <th>fastdetectlm</th>
  </tr>
  <tr>
    <td>Qwen2.5-72B-Instruct</td>
    <td>0.955</td>
    <td>0.977</td>
    <td>0.932</td>
    <td>0.906</td>
    <td>0.780</td>
    <td>0.847</td>
    <td>0.619</td>
    <td>0.500</td>
    <td>0.495</td>
    <td>0.481</td>
  </tr>
  <tr>
    <td>Qwen2.5-32B-Instruct</td>
    <td>0.954</td>
    <td>0.909</td>
    <td>0.774</td>
    <td>0.798</td>
    <td>0.784</td>
    <td>0.686</td>
    <td>0.494</td>
    <td>0.462</td>
    <td>0.487</td>
    <td>0.481</td>
  </tr>
  <tr>
    <td>Qwen2.5-14B-Instruct</td>
    <td>0.955</td>
    <td>0.901</td>
    <td>0.813</td>
    <td>0.742</td>
    <td>0.876</td>
    <td>0.806</td>
    <td>0.563</td>
    <td>0.472</td>
    <td>0.495</td>
    <td>0.481</td>
  </tr>
  <tr>
    <td>Qwen2.5-7B-Instruct</td>
    <td>0.955</td>
    <td>0.976</td>
    <td>0.870</td>
    <td>0.849</td>
    <td>0.806</td>
    <td>0.699</td>
    <td>0.515</td>
    <td>0.465</td>
    <td>0.488</td>
    <td>0.481</td>
  </tr>
  <tr>
    <td>Qwen2.5-3B-Instruct</td>
    <td>0.948</td>
    <td>0.760</td>
    <td>0.590</td>
    <td>0.711</td>
    <td>0.780</td>
    <td>0.579</td>
    <td>0.495</td>
    <td>0.466</td>
    <td>0.495</td>
    <td>0.481</td>
  </tr>
  <tr>
    <td>Qwen2.5-1.5B-Instruct</td>
    <td>0.916</td>
    <td>0.881</td>
    <td>0.676</td>
    <td>0.784</td>
    <td>0.735</td>
    <td>0.651</td>
    <td>0.587</td>
    <td>0.537</td>
    <td>0.510</td>
    <td>0.481</td>
  </tr>
  <tr>
    <td>Qwen2.5-0.5B-Instruct</td>
    <td>0.823</td>
    <td>0.901</td>
    <td>0.616</td>
    <td>0.768</td>
    <td>0.652</td>
    <td>0.607</td>
    <td>0.645</td>
    <td>0.762</td>
    <td>0.509</td>
    <td>0.481</td>
  </tr>
  <tr>
    <td>llama3.1-70B-Instruct</td>
    <td>0.929</td>
    <td>0.911</td>
    <td>0.857</td>
    <td>0.814</td>
    <td>0.767</td>
    <td>0.728</td>
    <td>0.609</td>
    <td>0.510</td>
    <td>0.531</td>
    <td>0.486</td>
  </tr>
  <tr>
    <td>llama3.1-18B-Instruct</td>
    <td>0.950</td>
    <td>0.964</td>
    <td>0.891</td>
    <td>0.901</td>
    <td>0.753</td>
    <td>0.644</td>
    <td>0.540</td>
    <td>0.471</td>
    <td>0.481</td>
    <td>0.481</td>
  </tr>
  <tr>
    <td>llama3.2-3B-Instruct</td>
    <td>0.916</td>
    <td>0.913</td>
    <td>0.817</td>
    <td>0.780</td>
    <td>0.656</td>
    <td>0.632</td>
    <td>0.535</td>
    <td>0.518</td>
    <td>0.518</td>
    <td>0.484</td>
  </tr>
  <tr>
    <td>llama3.2-1B-Instruct</td>
    <td>0.832</td>
    <td>0.827</td>
    <td>0.694</td>
    <td>0.740</td>
    <td>0.700</td>
    <td>0.643</td>
    <td>0.673</td>
    <td>0.653</td>
    <td>0.507</td>
    <td>0.523</td>
  </tr>
  <tr>
    <td>Mistral-Small-Instruct-2409</td>
    <td>0.955</td>
    <td>0.956</td>
    <td>0.760</td>
    <td>0.782</td>
    <td>0.674</td>
    <td>0.632</td>
    <td>0.522</td>
    <td>0.464</td>
    <td>0.499</td>
    <td>0.481</td>
  </tr>
  <tr>
    <td>Average</td>
    <td>0.924</td>
    <td>0.906</td>
    <td>0.774</td>
    <td>0.793</td>
    <td>0.743</td>
    <td>0.697</td>
    <td>0.583</td>
    <td>0.531</td>
    <td>0.504</td>
    <td>0.485</td>
  </tr>
</table>

Table 29: Detector performance across various generator models under subgroup Academic Genre = Social Science
G  Statistics of Generated Texts

We performed dimensionality reduction to visualize the text feature distributions from different models alongside human-written texts. Figure 2 shows PCA (left) and t-SNE (right) results in 2D space. Under PCA, texts generated by each model family often cluster together, indicating relatively similar language patterns within a family. Under t-SNE, the overall point cloud is more scattered, with each model forming a denser cluster but still overlapping somewhat with other models. Human texts, meanwhile, appear distinct from model-generated texts, implying that the two groups remain separable in feature space. Interestingly, points from the same model family tend to reside near each other, reflecting shared output styles, while the gap between human and AI texts suggests potential discriminative power for detection tasks.

Figure 3: Human text features heatmap

Figure 4: Human text features histograms

Figure 5: Human text features pairplot

Figure 6: Llama3.1-8B-Instruct text features heatmap

Figure 7: Llama3.1-8B-Instruct text features histograms
<table>
  <tr>
    <th>Category</th>
    <th>Feature</th>
    <th>Explanation</th>
  </tr>
  <tr>
    <td rowspan="4">Basic</td>
    <td>Word Count</td>
    <td>Total words</td>
  </tr>
  <tr>
    <td>Unique Words</td>
    <td>Distinct words</td>
  </tr>
  <tr>
    <td>Character Count</td>
    <td>Total characters</td>
  </tr>
  <tr>
    <td>Sentence Count</td>
    <td>Number of sentences</td>
  </tr>
  <tr>
    <td rowspan="2">Lexical</td>
    <td>Avg Word Length</td>
    <td>(Characters - special chars) / words</td>
  </tr>
  <tr>
    <td>Type-Token Ratio</td>
    <td>Distinct words / total words</td>
  </tr>
  <tr>
    <td>Syntactic / POS</td>
    <td>N/V/Adj Ratio</td>
    <td>Share of nouns, verbs, adjectives</td>
  </tr>
  <tr>
    <td>Readability</td>
    <td>Flesch Reading Ease<br>Gunning Fog Index</td>
    <td>Formula-based readability score<br>Index using complex words & sentence length</td>
  </tr>
</table>

Table 30: Features and Explanations

![PCA and t-SNE visualization plots](page_186_370_1077_393.png)

Figure 2: Visualization results of PCA (left) and t-SNE (right)

![Llama3.1-8B-Instruct text features pairplot](page_186_755_1077_393.png)

Figure 8: Llama3.1-8B-Instruct text features pairplot

![Correlation heatmap of features](page_1012_755_410_393.png)

Figure 9: Llama3.1-70B-Instruct text features heatmap
Figure 10: Llama3.1-70B-Instruct text features histograms

Figure 11: Llama3.1-70B-Instruct text features pairplot

Figure 12: Llama3.2-1B-Instruct text features heatmap

Figure 13: Llama3.2-1B-Instruct text features histograms

Figure 14: Llama3.2-1B-Instruct text features pairplot

Figure 15: Llama3.2-3B-Instruct text features heatmap
Figure 16: Llama3.2-3B-Instruct text features histograms

Figure 17: Llama3.2-3B-Instruct text features pairplot

Figure 18: Qwen2.5-0.5B-Instruct text features heatmap

Figure 19: Qwen2.5-0.5B-Instruct text features histograms

Figure 20: Qwen2.5-0.5B-Instruct text features pairplot

Figure 21: Qwen2.5-1.5B-Instruct text features heatmap
Figure 22: Qwen2.5-1.5B-Instruct text features histograms

Figure 23: Qwen2.5-1.5B-Instruct text features pairplot

Figure 24: Qwen2.5-3B-Instruct text features heatmap

Figure 25: Qwen2.5-3B-Instruct text features histograms

Figure 26: Qwen2.5-3B-Instruct text features pairplot

Figure 27: Qwen2.5-7B-Instruct text features heatmap
Figure 28: Qwen2.5-7B-Instruct text features histograms

Figure 29: Qwen2.5-7B-Instruct text features pairplot

Figure 30: Qwen2.5-14B-Instruct text features heatmap

Figure 31: Qwen2.5-14B-Instruct text features histograms

Figure 32: Qwen2.5-14B-Instruct text features pairplot

Figure 33: Qwen2.5-32B-Instruct text features heatmap
Figure 34: Qwen2.5-32B-Instruct text features histograms

Figure 37: Qwen2.5-72B-Instruct text features histograms

Figure 35: Qwen2.5-32B-Instruct text features pairplot

Figure 38: Qwen2.5-72B-Instruct text features pairplot

Figure 36: Qwen2.5-72B-Instruct text features heatmap

Figure 39: Mistral-Small-Instruct-2409 text features heatmap
Figure 40: Mistral-Small-Instruct-2409 text features histograms

Figure 41: Mistral-Small-Instruct-2409 text features pairplot

H Prompt

In this section, we present two types of prompts used in our study. The System Prompt is generated based on each learner’s attributes, including sex, academic background, English proficiency level, and country of origin, to construct a more targeted background persona. The Task Prompt provides the specific writing instructions, topic, length requirements, format, and spell-check reminders for the learners.

H.1 System Prompt

We construct the System Prompt by concatenating four short statements reflecting the following dimensions in order: “Sex,” “Acad. Genre,” “CEFR,” and “Country.” Each dimension–value pair maps to a predefined prompt string. For instance, if “Sex” = Male, then the prompt part is “You are a male author.” The final prompt strings for the other attributes are similarly selected from the respective lists.

Example

Sex = Male
--> "You are a male author."

Acad. Genre = Science Technology
--> "Your academic background is in Science & Technology."

CEFR = A2_0
--> "Your English proficiency level is CEFR A2."

Country = CHN
--> "You are from China, an EFL environment."

When concatenated, these form the complete system prompt:

Prompt H.1: Male, Sci_Tech, A2_0, CHN

You are a male author. Your academic background is in Science & Technology. Your English proficiency level is CEFR A2. You are from China, an EFL environment (English as a Foreign Language).

The JSON representation below details all possible dimension–value pairs and their corresponding prompt strings, which we combine to produce the final System Prompt.

Prompt: Sex

<table>
  <tr><th>Sex</th><th>Prompt</th></tr>
  <tr><td>Sex = Male</td><td>You are a male author.</td></tr>
  <tr><td>Sex = Female</td><td>You are a female author.</td></tr>
</table>

Prompt: Acad. Genre

<table>
  <tr><th>Acad. Genre</th><th>Prompt</th></tr>
  <tr><td>Acad. Genre = Humanities</td><td>Your academic background is in the Humanities.</td></tr>
  <tr><td>Acad. Genre = Social Sciences</td><td>Your academic background is in the Social Sciences.</td></tr>
  <tr><td>Acad. Genre = Science & Technology</td><td>Your academic background is in Science & Technology.</td></tr>
  <tr><td>Acad. Genre = Life Science</td><td>Your academic background is in Life Science.</td></tr>
</table>

Prompt: CEFR

<table>
  <tr><th>CEFR</th><th>Prompt</th></tr>
  <tr><td>CEFR = A2_0</td><td>Your English proficiency level is CEFR A2.</td></tr>
  <tr><td>CEFR = B1_1</td><td>Your English proficiency level is CEFR B1 (lower).</td></tr>
</table>
CEFR = B1_2
Your English proficiency level is CEFR B1 (upper).

CEFR = B2_0
Your English proficiency level is CEFR B2+.

CEFR = XX_0
You are a native English speaker.

Prompt: Country

Country = CHN
You are from China, an EFL environment (English as a Foreign Language).

Country = THA
You are from Thailand, an EFL environment (English as a Foreign Language).

Country = JPN
You are from Japan, an EFL environment (English as a Foreign Language).

Country = KOR
You are from Korea, an EFL environment (English as a Foreign Language).

Country = IDN
You are from Indonesia, an EFL environment (English as a Foreign Language).

Country = PHL
You are from the Philippines, an ESL environment (English as a Second Language).

Country = PAK
You are from Pakistan, an ESL environment (English as a Second Language).

Country = SIN
You are from Singapore, an ESL environment (English as a Second Language).

Region = TWN
You are from Taiwan, an EFL environment (English as a Foreign Language).

Country = ENS_USA
You are from the United States, a native English-speaking (NS) environment.

Country = HKG
You are from Hong Kong, an ESL environment (English as a Second Language).

Country = ENS_GBR
You are from the United Kingdom, a native English-speaking (NS) environment.

Country = ENS_CAN
You are from Canada, a native English-speaking (NS) environment.

Country = ENS_AUS
You are from Australia, a native English-speaking (NS) environment.

Country = ENS_NZL
You are from New Zealand, a native English-speaking (NS) environment.

H.2 Prompt Illustration

For clarity, the template for constructing the System Prompt is:

Prompt Template
{sex_prompt} {acad_genre_prompt} {cefr_prompt} {country_prompt}

Each placeholder (e.g., {sex_prompt}) is replaced by the corresponding string from the JSON snippets in Section H.1. Below is a concrete illustration using one combination of attributes:

Sample Prompt: F, Humanities, B1_1, SIN
You are a female author. Your academic background is in the Humanities. Your English proficiency level is CEFR B1 (lower). You are from Singapore, an ESL environment (English as a Second Language).

Thus, the System Prompt succinctly captures user-specific information to guide the subsequent text generation process.

H.3 Task Prompt

We use two topics for the writing tasks, each with the following requirements. Learners must use a word processor, keep the word count between 200 and 300 words, and run spell check before finalizing.

Prompt H.4: PTJ task prompt
Do you agree or disagree with the following statements? Use reasons and specific details to support your opinion. (Topic) It is important for college students to have a part-time job.

Instructions
1. Clarify your opinions and show the reasons and some examples.
2. The length of your single essay should be from 200 to 300 WORDS (not letters). Too short or too long essays cannot be accepted.
3. You must run spell check before completing your writing.

Prompt H.5: SMK task prompt
Do you agree or disagree with the following statements? Use reasons and specific details to support your opinion. (Topic) Smoking should be completely banned at all the restaurants in the country.

Instructions
1. Clarify your opinions and show the reasons and some examples.
2. The length of your single essay should be from 200 to 300 WORDS (not letters). Too short or too long essays cannot be accepted.
3. You must run spell check before completing your writing.
1 Additional Experiments

In this section, we treat AI text detection as a binary classification problem and conduct additional experiments using classical machine learning methods to explore potential biases.

1.1 Training Data

We adopt the HC3 dataset (Guo et al., 2023a) as our training corpus. HC3 contains 24.3k prompts, each accompanied by both human and ChatGPT responses, spanning diverse domains (e.g., Reddit Q&A, medical, finance, law). Widely used in existing research, HC3 provides a representative overview of differences between LLM-generated and human-authored text.

1.2 Feature Engineering

We extract a range of textual features to train the classical models. Table 31 summarizes the categories and specific features, including basic lexical counts, syntactic information, and readability metrics.

1.3 Machine Learning Detectors

To complement neural-based approaches, we train a suite of classical machine learning models (Table 32) on the extracted features. By evaluating their performance, we aim to assess whether biases might arise from specific model families or feature sets.

1.4 Evaluation Metrics

We treat AI text detection as a binary classification task and use Accuracy as the primary metric:

\[
\text{Accuracy} = \frac{\text{TP} + \text{TN}}{\text{TP} + \text{TN} + \text{FP} + \text{FN}}
\]

where TP is true positive (\(AI\) text correctly identified), TN is true negative (\(human\) text correctly identified), FP is false positive (\(human\) text misidentified as \(AI\)), and FN is false negative (\(AI\) text misidentified as \(human\)).

We compare results under two distinct scenarios:

In-domain Testing (HC3-based). The testing data share the same distribution as the training set.

Out-of-domain Testing (ICNALE + LLM). The testing data come from a different distribution (ICNALE plus newly generated LLM texts).

This setup clarifies how well the detectors generalize to previously unseen text distributions.

![Bar chart showing model accuracy comparison for HC3 dataset](page_1016_186_377_181.png)

Figure 42: In-domain test results (HC3). Accuracy on the y-axis.

<table>
  <tr>
    <th>Detector</th>
    <th>Aggregate</th>
    <th>Basic</th>
    <th>Syntactic</th>
    <th>Readability</th>
    <th>Lexical</th>
    <th>Other</th>
    <th>Generation Model</th>
    <th>Aggregate</th>
    <th>Basic</th>
    <th>Syntactic</th>
    <th>Readability</th>
    <th>Lexical</th>
    <th>Other</th>
    <th>LLM</th>
  </tr>
  <tr>
    <td>LogisticRegression</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
  </tr>
  <tr>
    <td>RandomForestClassifier</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
  </tr>
  <tr>
    <td>GradientBoostingClassifier</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
  </tr>
  <tr>
    <td>XGBoostClassifier</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
  </tr>
  <tr>
    <td>AdaBoostClassifier</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
  </tr>
  <tr>
    <td>KNeighborsClassifier</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
  </tr>
  <tr>
    <td>QuadraticDiscriminantAnalysis</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
  </tr>
  <tr>
    <td>PassiveAggressiveClassifier</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
  </tr>
  <tr>
    <td>SGDClassifier</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
  </tr>
  <tr>
    <td>NeuralNetworkClassifier</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
  </tr>
  <tr>
    <td>Perceptron</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
    <td>0.95</td>
  </tr>
</table>

Figure 43: Out-of-domain test results (ICNALE + LLM). Accuracy on the y-axis.

1.5 Results

In-domain Results. Figure 42 shows the in-domain performance on HC3. Most detectors achieve accuracy scores above 0.90, with some models approaching 0.99. This indicates that, when the test data distribution is similar to training, current deep learning detectors can effectively distinguish AI from human texts.

Out-of-domain Results. Figure 43 presents the out-of-domain results using our ICNALE + LLM dataset. Performance drops significantly compared to in-domain, with accuracy scores generally in the 0.50–0.65 range, and some models falling as low as 0.40. This underlines the limited capacity of detectors to transfer knowledge when encountering distributions different from their training data. Such a gap highlights the need for more diverse training corpora and advanced adaptation or domain-transfer techniques.

In sum, while machine learning detectors show encouraging performance on in-distribution data,
<table>
  <tr>
    <th>Category</th>
    <th>Feature</th>
    <th>Explanation</th>
  </tr>
  <tr>
    <td rowspan="4">Basic</td>
    <td>Word Count</td>
    <td>Total words in a text</td>
  </tr>
  <tr>
    <td>Unique Words</td>
    <td>Number of distinct words</td>
  </tr>
  <tr>
    <td>Character Count</td>
    <td>Total characters</td>
  </tr>
  <tr>
    <td>Sentence Count</td>
    <td>Number of sentences</td>
  </tr>
  <tr>
    <td rowspan="2">Lexical</td>
    <td>Avg Word Length</td>
    <td>(Characters minus special characters) / total words</td>
  </tr>
  <tr>
    <td>Type-Token Ratio</td>
    <td>Distinct words / total words</td>
  </tr>
  <tr>
    <td>Syntactic / POS</td>
    <td>N/V/Adj Ratio</td>
    <td>Proportion of nouns, verbs, adjectives</td>
  </tr>
  <tr>
    <td>Readability</td>
    <td>Flesch Reading Ease<br>Gunning Fog Index</td>
    <td>Formula-based readability score<br>Index using complex words and sentence length</td>
  </tr>
</table>

Table 31: Features and their explanations.

<table>
  <tr>
    <th>Category</th>
    <th>Models</th>
  </tr>
  <tr>
    <td>Linear Models</td>
    <td>Logistic Regression, Ridge Classifier, Perceptron, Passive Aggressive Classifier</td>
  </tr>
  <tr>
    <td><b>Support Vector Machines (SVM)</b></td>
    <td>SVC (Support Vector Classifier), NuSVC</td>
  </tr>
  <tr>
    <td>Naive Bayes</td>
    <td>Multinomial NB, Bernoulli NB, Gaussian NB, Complement NB</td>
  </tr>
  <tr>
    <td>Neighbor-Based Methods</td>
    <td>KNN (K-Nearest Neighbors), Nearest Centroid</td>
  </tr>
  <tr>
    <td><b>Discriminant Analysis</b></td>
    <td>LDA (Linear Discriminant Analysis), QDA (Quadratic Discriminant Analysis)</td>
  </tr>
  <tr>
    <td>Tree-Based Models</td>
    <td>Decision Tree, Random Forest, Extra Tree</td>
  </tr>
  <tr>
    <td>Boosting</td>
    <td>Gradient Boosting, XGBoost, AdaBoost</td>
  </tr>
  <tr>
    <td>Ensemble Learning</td>
    <td>Bagging, Voting Classifier, Stacking Classifier</td>
  </tr>
  <tr>
    <td>Neural Networks</td>
    <td>MLP (Multi-Layer Perceptron)</td>
  </tr>
  <tr>
    <td>Stochastic Methods</td>
    <td>Stochastic Gradient Descent (SGD) Classifier</td>
  </tr>
</table>

Table 32: Summary of machine learning models used.

they often struggle to maintain high accuracy once faced with new domains or author attributes. Real-world AI text detection frequently involves such diverse and shifting distributions, underscoring the importance of robust generalization strategies.

I.6 Bias Analysis

Hypothesis Test Setting  In order to investigate whether different features exhibit significant differences in the performance of various machine learning detectors, this section adopts appropriate statistical tests based on the number of possible values of each feature. Specifically, for features that have only two values (e.g., gender), we use the independent samples t-test; for features that contain more than two values (e.g., CEFR, academic writing style, language environment), we conduct ANOVA to evaluate whether there are significant differences across different feature levels.

Hypothesis Test Results  Figure 44 shows the t-test and ANOVA results for the four main features (gender, CEFR level, academic genre, and language environment). Each row corresponds to a different model, and each column corresponds to one of the four features. The results are indicated by “1” for significant (p < 0.05) and “0” for non-significant (p ≥ 0.05). From these results, we can draw the following conclusions:

Gender (Sex, T-test)  For the binary feature of gender (Female vs. Male), most detectors (e.g., KNeighborsClassifier, SVC, RandomForestClassifier, etc.) exhibit significant differences (p < 0.05), indicating that gender-based distinctions have statistical significance in these detectors’ predictive performance. However, a few detectors such as LogisticRegression, XGBClassifier, and NearestCentroid do not show significant differences, suggesting that for these detectors, gender has a weaker or non-significant effect on the detection results.

CEFR Level (cefr, ANOVA)  All detectors display significant differences in the ANOVA results for the multi-class feature of CEFR level, indicating that the graded language proficiency (ranging from beginner to advanced) exerts distinct and statistically significant effects on the detectors’ predictions. This further demonstrates that CEFR levels possess strong predictive discriminative power and
Figure 44: Hypothesis test results (t-test and ANOVA)

significantly influence the outcomes of various detectors.

Academic Genre (academic_genre, ANOVA) For academic genre, some detectors (e.g., XGBClassifier, RandomForestClassifier, AdaBoostClassifier, etc.) exhibit significant differences in the ANOVA, whereas others (e.g., LogisticRegression, SVC, DecisionTreeClassifier, etc.) do not. This finding suggests that different detectors vary in their sensitivity to academic writing styles; some are better at capturing and leveraging stylistic differences, while others do not reflect clear statistical differences in their predictions.

Language Environment (language_env, ANOVA) With respect to language environment, the ANOVA results show that the vast majority of detectors exhibit significant differences, indicating that different language environments (e.g., native vs. second language settings) have a substantial impact on the predictive outcomes of these detectors. However, certain detectors (e.g., SVC, DecisionTreeClassifier, GradientBoostingClassifier, etc.) do not reach the threshold of statistical significance on this feature, suggesting that their ability to differentiate language environments is insufficient to yield statistically significant results.

Taken together, the analysis reveals that detectors vary in their sensitivity and discriminative power with regard to gender, CEFR level, academic writing style, and language environment. Most detectors demonstrate high discriminative power for CEFR level and language environment, while sensitivity to gender and academic writing style depends on the specific detector. These findings provide a statistical perspective on the influence of various features in AI-based text detection and offer valuable insights for optimizing both detectors and feature selection in future research.

J Sensitivity Analyses of WLS

While the multi-factor weighted least squares (WLS) model provides a systematic approach to control for multiple confounders, datasets often exhibit imbalanced subgroup distributions or heterogeneity that can affect statistical inferences. To ensure the robustness of our parameter estimates, we perform a bootstrap-based sensitivity analysis.

J.1 Bootstrap-Based Parameter Estimation

Parameter estimates can be sensitive to random fluctuations in the data. To assess this sensitivity, we use bootstrapping. We create many "new" datasets by resampling with replacement from the original dataset (keeping the same overall size). We fit the WLS model on each bootstrap sample and aggregate the resulting estimates. This approach provides a distribution for each parameter. We report the mean and standard deviation of these bootstrap estimates, along with the 95% confidence interval (CI) based on the 2.5th and 97.5th percentiles of the bootstrap distribution. We then check whether the original parameter estimate falls within this bootstrap CI ("Coverage"). If the original estimate lies within the CI, it provides evidence that the estimate is stable to sampling variability, and thus robust to the specific composition of the sample.

J.2 Summary of Sensitivity Findings

The bootstrap results, presented in Tables 37 through 46, show that for all detectors and all parameters, the original coefficient estimates lie within the 95% confidence intervals derived from the bootstrap resampling. This indicates strong stability of the parameter estimates. The relatively narrow confidence intervals and consistent "WITHIN CI" coverage across all parameters and detectors provide substantial evidence that our main WLS findings are robust to sampling variability. This strengthens our confidence in the reported effects of CEFR level, sex, academic genre, and language environment on detector accuracy.
<table>
  <tr>
    <th>Detector</th>
    <th>t-value</th>
    <th>p-value</th>
    <th>Significance (p<0.05)</th>
  </tr>
  <tr>
    <td>KNeighborsClassifier</td>
    <td>-3.8818</td>
    <td>1.0812e-04</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>LogisticRegression</td>
    <td>-0.2408</td>
    <td>8.0976e-01</td>
    <td>Not Significant</td>
  </tr>
  <tr>
    <td>XGBClassifier</td>
    <td>-1.4438</td>
    <td>1.4900e-01</td>
    <td>Not Significant</td>
  </tr>
  <tr>
    <td>SVC</td>
    <td>-3.9214</td>
    <td>9.1954e-05</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>LinearSVC</td>
    <td>-2.8893</td>
    <td>3.9144e-03</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>DecisionTreeClassifier</td>
    <td>-3.1055</td>
    <td>1.9351e-03</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>RandomForestClassifier</td>
    <td>-4.8084</td>
    <td>1.6770e-06</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>GradientBoostingClassifier</td>
    <td>-3.8380</td>
    <td>1.2898e-04</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>AdaBoostClassifier</td>
    <td>-4.2847</td>
    <td>1.9443e-05</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>BaggingClassifier</td>
    <td>-3.5719</td>
    <td>3.6564e-04</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>ExtraTreesClassifier</td>
    <td>-5.5451</td>
    <td>3.4535e-08</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>NearestCentroid</td>
    <td>-1.3941</td>
    <td>1.6347e-01</td>
    <td>Not Significant</td>
  </tr>
  <tr>
    <td>RidgeClassifier</td>
    <td>-0.7179</td>
    <td>4.7293e-01</td>
    <td>Not Significant</td>
  </tr>
  <tr>
    <td>SGDClassifier</td>
    <td>-0.5139</td>
    <td>6.0741e-01</td>
    <td>Not Significant</td>
  </tr>
  <tr>
    <td>Perceptron</td>
    <td>-1.3458</td>
    <td>1.7856e-01</td>
    <td>Not Significant</td>
  </tr>
  <tr>
    <td>PassiveAggressiveClassifier</td>
    <td>-3.2496</td>
    <td>1.2048e-03</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>MLPClassifier</td>
    <td>-3.5214</td>
    <td>4.4255e-04</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>LinearDiscriminantAnalysis</td>
    <td>-1.0359</td>
    <td>3.0043e-01</td>
    <td>Not Significant</td>
  </tr>
  <tr>
    <td>QuadraticDiscriminantAnalysis</td>
    <td>-5.1645</td>
    <td>2.7226e-07</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>BernoulliNB</td>
    <td>-4.2733</td>
    <td>2.0588e-05</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>GaussianNB</td>
    <td>-3.3843</td>
    <td>7.3276e-04</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>ExtraTreeClassifier</td>
    <td>-5.4473</td>
    <td>5.9765e-08</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>VotingClassifier</td>
    <td>-3.4404</td>
    <td>5.9649e-04</td>
    <td>Significant</td>
  </tr>
</table>

Table 33: T-test results for Feature 1: Sex

<table>
  <tr>
    <th>Detector</th>
    <th>F-value</th>
    <th>p-value</th>
    <th>Significance (p<0.05)</th>
  </tr>
  <tr>
    <td>KNeighborsClassifier</td>
    <td>9.7256</td>
    <td>8.9973e-08</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>LogisticRegression</td>
    <td>10.2101</td>
    <td>3.6569e-08</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>XGBClassifier</td>
    <td>14.5891</td>
    <td>1.0422e-11</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>SVC</td>
    <td>3.3956</td>
    <td>8.9432e-03</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>LinearSVC</td>
    <td>11.2302</td>
    <td>5.4780e-09</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>DecisionTreeClassifier</td>
    <td>9.4865</td>
    <td>1.4025e-07</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>RandomForestClassifier</td>
    <td>47.3126</td>
    <td>1.2241e-37</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>GradientBoostingClassifier</td>
    <td>9.9970</td>
    <td>5.4343e-08</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>AdaBoostClassifier</td>
    <td>15.2355</td>
    <td>3.1203e-12</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>BaggingClassifier</td>
    <td>9.3998</td>
    <td>1.6475e-07</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>ExtraTreesClassifier</td>
    <td>51.9283</td>
    <td>3.5969e-41</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>NearestCentroid</td>
    <td>3.0469</td>
    <td>1.6277e-02</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>RidgeClassifier</td>
    <td>37.2302</td>
    <td>8.2609e-30</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>SGDClassifier</td>
    <td>66.5123</td>
    <td>4.0857e-52</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>Perceptron</td>
    <td>45.3916</td>
    <td>3.6934e-36</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>PassiveAggressiveClassifier</td>
    <td>3.0908</td>
    <td>1.5361e-02</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>MLPClassifier</td>
    <td>20.9005</td>
    <td>8.1949e-17</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>LinearDiscriminantAnalysis</td>
    <td>36.4892</td>
    <td>3.1519e-29</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>QuadraticDiscriminantAnalysis</td>
    <td>39.3666</td>
    <td>1.7583e-31</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>BernoulliNB</td>
    <td>98.4916</td>
    <td>5.1139e-75</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>GaussianNB</td>
    <td>45.5083</td>
    <td>3.0021e-36</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>ExtraTreeClassifier</td>
    <td>8.2371</td>
    <td>1.4190e-06</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>VotingClassifier</td>
    <td>26.9124</td>
    <td>1.2214e-21</td>
    <td>Significant</td>
  </tr>
</table>

Table 34: ANOVA results for Feature 2: CEFR
<table>
  <tr>
    <th>Detector</th>
    <th>F-value</th>
    <th>p-value</th>
    <th>Significance (p<0.05)</th>
  </tr>
  <tr>
    <td>LogisticRegression</td>
    <td>0.9484</td>
    <td>4.1638e-01</td>
    <td>Not Significant</td>
  </tr>
  <tr>
    <td>XGBClassifier</td>
    <td>3.8680</td>
    <td>9.0291e-03</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>SVC</td>
    <td>2.3311</td>
    <td>7.2526e-02</td>
    <td>Not Significant</td>
  </tr>
  <tr>
    <td>LinearSVC</td>
    <td>8.5848</td>
    <td>1.1803e-05</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>DecisionTreeClassifier</td>
    <td>2.1278</td>
    <td>9.4821e-02</td>
    <td>Not Significant</td>
  </tr>
  <tr>
    <td>RandomForestClassifier</td>
    <td>5.2634</td>
    <td>1.2955e-03</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>GradientBoostingClassifier</td>
    <td>2.3275</td>
    <td>7.2877e-02</td>
    <td>Not Significant</td>
  </tr>
  <tr>
    <td>AdaBoostClassifier</td>
    <td>3.7601</td>
    <td>1.0475e-02</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>BaggingClassifier</td>
    <td>7.3502</td>
    <td>6.8191e-05</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>ExtraTreesClassifier</td>
    <td>1.6179</td>
    <td>1.8329e-01</td>
    <td>Not Significant</td>
  </tr>
  <tr>
    <td>KNeighborsClassifier</td>
    <td>8.5072</td>
    <td>1.3181e-05</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>NearestCentroid</td>
    <td>0.2623</td>
    <td>8.5258e-01</td>
    <td>Not Significant</td>
  </tr>
  <tr>
    <td>RidgeClassifier</td>
    <td>9.0998</td>
    <td>5.6698e-06</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>SGDClassifier</td>
    <td>5.1594</td>
    <td>1.4985e-03</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>Perceptron</td>
    <td>7.5843</td>
    <td>4.8926e-05</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>PassiveAggressiveClassifier</td>
    <td>1.1490</td>
    <td>3.2840e-01</td>
    <td>Not Significant</td>
  </tr>
  <tr>
    <td>MLPClassifier</td>
    <td>6.3252</td>
    <td>2.9084e-04</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>LinearDiscriminantAnalysis</td>
    <td>7.2302</td>
    <td>8.0842e-05</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>QuadraticDiscriminantAnalysis</td>
    <td>44.5179</td>
    <td>1.2424e-27</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>BernoulliNB</td>
    <td>22.5459</td>
    <td>2.6820e-14</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>GaussianNB</td>
    <td>1.3856</td>
    <td>2.4545e-01</td>
    <td>Not Significant</td>
  </tr>
  <tr>
    <td>ExtraTreeClassifier</td>
    <td>7.4089</td>
    <td>6.2747e-05</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>VotingClassifier</td>
    <td>2.4313</td>
    <td>6.3496e-02</td>
    <td>Not Significant</td>
  </tr>
</table>

Table 35: ANOVA results for Feature 3: Academic Genre

<table>
  <tr>
    <th>Detector</th>
    <th>F-value</th>
    <th>p-value</th>
    <th>Significance (p<0.05)</th>
  </tr>
  <tr>
    <td>LogisticRegression</td>
    <td>15.4867</td>
    <td>2.1791e-07</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>XGBClassifier</td>
    <td>6.2824</td>
    <td>1.9152e-03</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>SVC</td>
    <td>1.7646</td>
    <td>1.7158e-01</td>
    <td>Not Significant</td>
  </tr>
  <tr>
    <td>LinearSVC</td>
    <td>12.8361</td>
    <td>2.9476e-06</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>DecisionTreeClassifier</td>
    <td>0.7064</td>
    <td>4.9355e-01</td>
    <td>Not Significant</td>
  </tr>
  <tr>
    <td>RandomForestClassifier</td>
    <td>98.8694</td>
    <td>3.2184e-41</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>GradientBoostingClassifier</td>
    <td>0.5297</td>
    <td>5.8886e-01</td>
    <td>Not Significant</td>
  </tr>
  <tr>
    <td>AdaBoostClassifier</td>
    <td>12.1365</td>
    <td>5.8702e-06</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>BaggingClassifier</td>
    <td>14.2985</td>
    <td>6.9971e-07</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>ExtraTreesClassifier</td>
    <td>124.3115</td>
    <td>6.4051e-51</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>KNeighborsClassifier</td>
    <td>3.0683</td>
    <td>4.6771e-02</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>NearestCentroid</td>
    <td>1.5409</td>
    <td>2.1451e-01</td>
    <td>Not Significant</td>
  </tr>
  <tr>
    <td>RidgeClassifier</td>
    <td>44.5998</td>
    <td>1.4114e-19</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>SGDClassifier</td>
    <td>159.0237</td>
    <td>9.8119e-64</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>Perceptron</td>
    <td>47.5704</td>
    <td>8.5035e-21</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>PassiveAggressiveClassifier</td>
    <td>0.2646</td>
    <td>7.6755e-01</td>
    <td>Not Significant</td>
  </tr>
  <tr>
    <td>MLPClassifier</td>
    <td>26.3431</td>
    <td>5.5349e-12</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>LinearDiscriminantAnalysis</td>
    <td>43.3016</td>
    <td>4.8325e-19</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>QuadraticDiscriminantAnalysis</td>
    <td>59.9251</td>
    <td>7.9565e-26</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>BernoulliNB</td>
    <td>334.0978</td>
    <td>4.9281e-122</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>GaussianNB</td>
    <td>42.1002</td>
    <td>1.5120e-18</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>ExtraTreeClassifier</td>
    <td>15.4280</td>
    <td>2.3082e-07</td>
    <td>Significant</td>
  </tr>
  <tr>
    <td>VotingClassifier</td>
    <td>64.4107</td>
    <td>1.2380e-27</td>
    <td>Significant</td>
  </tr>
</table>

Table 36: ANOVA results for Feature 4: Language Environment

<table>
  <tr>
    <th>Parameter</th>
    <th>Original Value</th>
    <th>Bootstrap Mean</th>
    <th>Bootstrap Std</th>
    <th>CI Lower</th>
    <th>CI Upper</th>
    <th>Coverage</th>
  </tr>
  <tr>
    <td>Intercept</td>
    <td>0.9482</td>
    <td>0.9480</td>
    <td>0.0079</td>
    <td>0.9318</td>
    <td>0.9625</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(cefr)[T.B1_1]</td>
    <td>-0.0039</td>
    <td>-0.0040</td>
    <td>0.0085</td>
    <td>-0.0204</td>
    <td>0.0132</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(cefr)[T.B1_2]</td>
    <td>-0.0007</td>
    <td>-0.0006</td>
    <td>0.0075</td>
    <td>-0.0149</td>
    <td>0.0150</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(cefr)[T.B2_0]</td>
    <td>0.0025</td>
    <td>0.0025</td>
    <td>0.0078</td>
    <td>-0.0125</td>
    <td>0.0172</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(cefr)[T.XX_0]</td>
    <td>-0.0501</td>
    <td>-0.0499</td>
    <td>0.0049</td>
    <td>-0.0592</td>
    <td>-0.0400</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(Sex)[TM]</td>
    <td>0.0010</td>
    <td>0.0011</td>
    <td>0.0054</td>
    <td>-0.0096</td>
    <td>0.0114</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(academic_genre)[T.Life Sciences]</td>
    <td>-0.0075</td>
    <td>-0.0073</td>
    <td>0.0082</td>
    <td>-0.0224</td>
    <td>0.0086</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(academic_genre)[T.Sciences & Technology]</td>
    <td>0.0016</td>
    <td>0.0018</td>
    <td>0.0072</td>
    <td>-0.0126</td>
    <td>0.0156</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(academic_genre)[T.Social Sciences]</td>
    <td>0.0016</td>
    <td>0.0018</td>
    <td>0.0068</td>
    <td>-0.0110</td>
    <td>0.0149</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(language_env)[T.ESL]</td>
    <td>-0.0144</td>
    <td>-0.0143</td>
    <td>0.0056</td>
    <td>-0.0247</td>
    <td>-0.0038</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(language_env)[T.NSI]</td>
    <td>-0.0501</td>
    <td>-0.0499</td>
    <td>0.0049</td>
    <td>-0.0592</td>
    <td>-0.0400</td>
    <td>WITHIN CI</td>
  </tr>
</table>

Table 37: Bootstrap Sensitivity Analysis for detector: binoculars
<table>
  <tr>
    <th>Parameter</th>
    <th>Original Value</th>
    <th>Bootstrap Mean</th>
    <th>Bootstrap Std</th>
    <th>CI Lower</th>
    <th>CI Upper</th>
    <th>Coverage</th>
  </tr>
  <tr>
    <td>Intercept</td>
    <td>0.7480</td>
    <td>0.7480</td>
    <td>0.0157</td>
    <td>0.7179</td>
    <td>0.7790</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(Sex)[T.M]</td>
    <td>0.0035</td>
    <td>0.0032</td>
    <td>0.0103</td>
    <td>-0.0176</td>
    <td>0.0226</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(cefr)[T.B1_1]</td>
    <td>-0.0073</td>
    <td>-0.0074</td>
    <td>0.0180</td>
    <td>-0.0403</td>
    <td>0.0294</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(cefr)[T.B1_2]</td>
    <td>-0.0103</td>
    <td>-0.0102</td>
    <td>0.0158</td>
    <td>-0.0413</td>
    <td>0.0214</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(cefr)[T.B2_0]</td>
    <td>-0.0435</td>
    <td>-0.0432</td>
    <td>0.0155</td>
    <td>-0.0728</td>
    <td>-0.0114</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(cefr)[T.XX_0]</td>
    <td>-0.0071</td>
    <td>-0.0071</td>
    <td>0.0084</td>
    <td>-0.0245</td>
    <td>0.0100</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(academic_genre)[T.Life Sciences]</td>
    <td>0.0286</td>
    <td>0.0287</td>
    <td>0.0143</td>
    <td>0.0007</td>
    <td>0.0557</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(academic_genre)[T.Sciences & Technology]</td>
    <td>0.0084</td>
    <td>0.0088</td>
    <td>0.0137</td>
    <td>-0.0179</td>
    <td>0.0350</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(academic_genre)[T.Social Sciences]</td>
    <td>0.0083</td>
    <td>0.0085</td>
    <td>0.0120</td>
    <td>-0.0167</td>
    <td>0.0312</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(language_env)[TESL]</td>
    <td>-0.0014</td>
    <td>-0.0012</td>
    <td>0.0094</td>
    <td>-0.0201</td>
    <td>0.0185</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(language_env)[T.NS]</td>
    <td>-0.0071</td>
    <td>-0.0071</td>
    <td>0.0084</td>
    <td>-0.0245</td>
    <td>0.0100</td>
    <td>WITHIN CI</td>
  </tr>
</table>

Table 38: Bootstrap Sensitivity Analysis for detector: chatgpt-roberta

<table>
  <tr>
    <th>Parameter</th>
    <th>Original Value</th>
    <th>Bootstrap Mean</th>
    <th>Bootstrap Std</th>
    <th>CI Lower</th>
    <th>CI Upper</th>
    <th>Coverage</th>
  </tr>
  <tr>
    <td>Intercept</td>
    <td>0.7944</td>
    <td>0.7948</td>
    <td>0.0098</td>
    <td>0.7754</td>
    <td>0.8133</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(academic_genre)[T.Life Sciences]</td>
    <td>-0.0397</td>
    <td>-0.0398</td>
    <td>0.0108</td>
    <td>-0.0611</td>
    <td>-0.0187</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(academic_genre)[T.Sciences & Technology]</td>
    <td>-0.0185</td>
    <td>-0.0189</td>
    <td>0.0093</td>
    <td>-0.0370</td>
    <td>-0.0010</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(academic_genre)[T.Social Sciences]</td>
    <td>-0.0060</td>
    <td>-0.0063</td>
    <td>0.0087</td>
    <td>-0.0233</td>
    <td>0.0101</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(cefr)[T.B1_1]</td>
    <td>0.0127</td>
    <td>0.0129</td>
    <td>0.0104</td>
    <td>-0.0074</td>
    <td>0.0347</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(cefr)[T.B1_2]</td>
    <td>0.0284</td>
    <td>0.0282</td>
    <td>0.0093</td>
    <td>0.0103</td>
    <td>0.0468</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(cefr)[T.B2_0]</td>
    <td>0.0345</td>
    <td>0.0339</td>
    <td>0.0106</td>
    <td>0.0132</td>
    <td>0.0554</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(cefr)[T.XX_0]</td>
    <td>-0.0223</td>
    <td>-0.0222</td>
    <td>0.0059</td>
    <td>-0.0338</td>
    <td>-0.0104</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(Sex)[T.M]</td>
    <td>-0.0068</td>
    <td>-0.0069</td>
    <td>0.0075</td>
    <td>-0.0210</td>
    <td>0.0079</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(language_env)[TESL]</td>
    <td>-0.0077</td>
    <td>-0.0075</td>
    <td>0.0071</td>
    <td>-0.0208</td>
    <td>0.0065</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(language_env)[T.NS]</td>
    <td>-0.0223</td>
    <td>-0.0222</td>
    <td>0.0059</td>
    <td>-0.0338</td>
    <td>-0.0104</td>
    <td>WITHIN CI</td>
  </tr>
</table>

Table 39: Bootstrap Sensitivity Analysis for detector: detectgpt

<table>
  <tr>
    <th>Parameter</th>
    <th>Original Value</th>
    <th>Bootstrap Mean</th>
    <th>Bootstrap Std</th>
    <th>CI Lower</th>
    <th>CI Upper</th>
    <th>Coverage</th>
  </tr>
  <tr>
    <td>Intercept</td>
    <td>0.8963</td>
    <td>0.8963</td>
    <td>0.0084</td>
    <td>0.8796</td>
    <td>0.9124</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(cefr)[T.B1_1]</td>
    <td>-0.0076</td>
    <td>-0.0071</td>
    <td>0.0093</td>
    <td>-0.0241</td>
    <td>0.0105</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(cefr)[T.B1_2]</td>
    <td>0.0060</td>
    <td>0.0059</td>
    <td>0.0082</td>
    <td>-0.0107</td>
    <td>0.0222</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(cefr)[T.B2_0]</td>
    <td>0.0180</td>
    <td>0.0181</td>
    <td>0.0083</td>
    <td>0.0023</td>
    <td>0.0346</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(cefr)[T.XX_0]</td>
    <td>-0.0214</td>
    <td>-0.0214</td>
    <td>0.0049</td>
    <td>-0.0309</td>
    <td>-0.0119</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(Sex)[T.M]</td>
    <td>-0.0003</td>
    <td>-0.0005</td>
    <td>0.0062</td>
    <td>-0.0130</td>
    <td>0.0113</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(academic_genre)[T.Life Sciences]</td>
    <td>0.0127</td>
    <td>0.0123</td>
    <td>0.0077</td>
    <td>-0.0025</td>
    <td>0.0281</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(academic_genre)[T.Sciences & Technology]</td>
    <td>-0.0012</td>
    <td>-0.0013</td>
    <td>0.0080</td>
    <td>-0.0174</td>
    <td>0.0144</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(academic_genre)[T.Social Sciences]</td>
    <td>0.0070</td>
    <td>0.0073</td>
    <td>0.0071</td>
    <td>-0.0068</td>
    <td>0.0216</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(language_env)[TESL]</td>
    <td>-0.0021</td>
    <td>-0.0021</td>
    <td>0.0060</td>
    <td>-0.0141</td>
    <td>0.0102</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(language_env)[T.NS]</td>
    <td>-0.0214</td>
    <td>-0.0214</td>
    <td>0.0049</td>
    <td>-0.0309</td>
    <td>-0.0119</td>
    <td>WITHIN CI</td>
  </tr>
</table>

Table 40: Bootstrap Sensitivity Analysis for detector: fastdetectgpt

<table>
  <tr>
    <th>Parameter</th>
    <th>Original Value</th>
    <th>Bootstrap Mean</th>
    <th>Bootstrap Std</th>
    <th>CI Lower</th>
    <th>CI Upper</th>
    <th>Coverage</th>
  </tr>
  <tr>
    <td>Intercept</td>
    <td>0.4873</td>
    <td>0.4873</td>
    <td>0.0026</td>
    <td>0.4824</td>
    <td>0.4926</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(cefr)[T.B1_1]</td>
    <td>-0.0031</td>
    <td>-0.0031</td>
    <td>0.0025</td>
    <td>-0.0081</td>
    <td>0.0017</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(cefr)[T.B1_2]</td>
    <td>-0.0087</td>
    <td>-0.0087</td>
    <td>0.0025</td>
    <td>-0.0137</td>
    <td>-0.0039</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(cefr)[T.B2_0]</td>
    <td>-0.0045</td>
    <td>-0.0045</td>
    <td>0.0028</td>
    <td>-0.0102</td>
    <td>0.0008</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(cefr)[T.XX_0]</td>
    <td>0.0102</td>
    <td>0.0102</td>
    <td>0.0013</td>
    <td>0.0077</td>
    <td>0.0127</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(Sex)[T.M]</td>
    <td>0.0018</td>
    <td>0.0019</td>
    <td>0.0016</td>
    <td>-0.0012</td>
    <td>0.0049</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(academic_genre)[T.Life Sciences]</td>
    <td>-0.0141</td>
    <td>-0.0143</td>
    <td>0.0040</td>
    <td>-0.0220</td>
    <td>-0.0066</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(academic_genre)[T.Sciences & Technology]</td>
    <td>-0.0060</td>
    <td>-0.0060</td>
    <td>0.0020</td>
    <td>-0.0100</td>
    <td>-0.0021</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(academic_genre)[T.Social Sciences]</td>
    <td>-0.0017</td>
    <td>-0.0017</td>
    <td>0.0020</td>
    <td>-0.0057</td>
    <td>0.0022</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(language_env)[TESL]</td>
    <td>-0.0035</td>
    <td>-0.0035</td>
    <td>0.0017</td>
    <td>-0.0069</td>
    <td>-0.0001</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(language_env)[T.NS]</td>
    <td>0.0102</td>
    <td>0.0102</td>
    <td>0.0013</td>
    <td>0.0077</td>
    <td>0.0127</td>
    <td>WITHIN CI</td>
  </tr>
</table>

Table 41: Bootstrap Sensitivity Analysis for detector: fastdetectllm
<table>
  <tr>
    <th>Parameter</th>
    <th>Original Value</th>
    <th>Bootstrap Mean</th>
    <th>Bootstrap Std</th>
    <th>CI Lower</th>
    <th>CI Upper</th>
    <th>Coverage</th>
  </tr>
  <tr>
    <td>Intercept</td>
    <td>0.8366</td>
    <td>0.8369</td>
    <td>0.0175</td>
    <td>0.8039</td>
    <td>0.8712</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(cefr)[T.B1_1]</td>
    <td>-0.0061</td>
    <td>-0.0064</td>
    <td>0.0189</td>
    <td>-0.0445</td>
    <td>0.0329</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(cefr)[T.B1_2]</td>
    <td>-0.0160</td>
    <td>-0.0158</td>
    <td>0.0169</td>
    <td>-0.0481</td>
    <td>0.0179</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(cefr)[T.B2_0]</td>
    <td>-0.0399</td>
    <td>-0.0405</td>
    <td>0.0175</td>
    <td>-0.0755</td>
    <td>-0.0064</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(cefr)[T.XX_0]</td>
    <td>-0.1186</td>
    <td>-0.1183</td>
    <td>0.0107</td>
    <td>-0.1383</td>
    <td>-0.0979</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(Sex)[TM]</td>
    <td>-0.0078</td>
    <td>-0.0078</td>
    <td>0.0130</td>
    <td>-0.0324</td>
    <td>0.0178</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(academic_genre)[T.Life Sciences]</td>
    <td>0.0065</td>
    <td>0.0069</td>
    <td>0.0185</td>
    <td>-0.0291</td>
    <td>0.0427</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(academic_genre)[T.Sciences & Technology]</td>
    <td>0.0068</td>
    <td>0.0068</td>
    <td>0.0172</td>
    <td>-0.0253</td>
    <td>0.0405</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(academic_genre)[T.Social Sciences]</td>
    <td>-0.0031</td>
    <td>-0.0033</td>
    <td>0.0154</td>
    <td>-0.0333</td>
    <td>0.0257</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(language_env)[TESL]</td>
    <td>-0.0020</td>
    <td>-0.0021</td>
    <td>0.0133</td>
    <td>-0.0273</td>
    <td>0.0241</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(language_env)[TNS]</td>
    <td>-0.1186</td>
    <td>-0.1183</td>
    <td>0.0107</td>
    <td>-0.1383</td>
    <td>-0.0979</td>
    <td>WITHIN CI</td>
  </tr>
</table>

Table 42: Bootstrap Sensitivity Analysis for detector g1tr

<table>
  <tr>
    <th>Parameter</th>
    <th>Original Value</th>
    <th>Bootstrap Mean</th>
    <th>Bootstrap Std</th>
    <th>CI Lower</th>
    <th>CI Upper</th>
    <th>Coverage</th>
  </tr>
  <tr>
    <td>Intercept</td>
    <td>0.6493</td>
    <td>0.6499</td>
    <td>0.0129</td>
    <td>0.6247</td>
    <td>0.6755</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(cefr)[T.B1_1]</td>
    <td>-0.0094</td>
    <td>-0.0096</td>
    <td>0.0138</td>
    <td>-0.0373</td>
    <td>0.0179</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(cefr)[T.B1_2]</td>
    <td>-0.0612</td>
    <td>-0.0611</td>
    <td>0.0127</td>
    <td>-0.0849</td>
    <td>-0.0366</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(cefr)[T.B2_0]</td>
    <td>-0.0870</td>
    <td>-0.0872</td>
    <td>0.0129</td>
    <td>-0.1118</td>
    <td>-0.0619</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(cefr)[T.XX_0]</td>
    <td>-0.1011</td>
    <td>-0.1013</td>
    <td>0.0070</td>
    <td>-0.1146</td>
    <td>-0.0873</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(Sex)[TM]</td>
    <td>-0.0043</td>
    <td>-0.0044</td>
    <td>0.0088</td>
    <td>-0.0213</td>
    <td>0.0126</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(academic_genre)[T.Life Sciences]</td>
    <td>0.0248</td>
    <td>0.0245</td>
    <td>0.0126</td>
    <td>-0.0016</td>
    <td>0.0490</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(academic_genre)[T.Sciences & Technology]</td>
    <td>0.0373</td>
    <td>0.0369</td>
    <td>0.0111</td>
    <td>0.0153</td>
    <td>0.0597</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(academic_genre)[T.Social Sciences]</td>
    <td>-0.0084</td>
    <td>-0.0090</td>
    <td>0.0103</td>
    <td>-0.0288</td>
    <td>0.0115</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(language_env)[TESL]</td>
    <td>0.0011</td>
    <td>0.0009</td>
    <td>0.0084</td>
    <td>-0.0150</td>
    <td>0.0176</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(language_env)[TNS]</td>
    <td>-0.1011</td>
    <td>-0.1013</td>
    <td>0.0070</td>
    <td>-0.1146</td>
    <td>-0.0873</td>
    <td>WITHIN CI</td>
  </tr>
</table>

Table 43: Bootstrap Sensitivity Analysis for detector gpt2–base

<table>
  <tr>
    <th>Parameter</th>
    <th>Original Value</th>
    <th>Bootstrap Mean</th>
    <th>Bootstrap Std</th>
    <th>CI Lower</th>
    <th>CI Upper</th>
    <th>Coverage</th>
  </tr>
  <tr>
    <td>Intercept</td>
    <td>0.5402</td>
    <td>0.5412</td>
    <td>0.0149</td>
    <td>0.5127</td>
    <td>0.5703</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(cefr)[T.B1_1]</td>
    <td>0.0274</td>
    <td>0.0270</td>
    <td>0.0159</td>
    <td>-0.0050</td>
    <td>0.0583</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(cefr)[T.B1_2]</td>
    <td>0.0008</td>
    <td>0.0003</td>
    <td>0.0149</td>
    <td>-0.0266</td>
    <td>0.0294</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(cefr)[T.B2_0]</td>
    <td>0.0060</td>
    <td>0.0053</td>
    <td>0.0159</td>
    <td>-0.0255</td>
    <td>0.0359</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(cefr)[T.XX_0]</td>
    <td>-0.0240</td>
    <td>-0.0244</td>
    <td>0.0080</td>
    <td>-0.0393</td>
    <td>-0.0093</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(Sex)[TM]</td>
    <td>-0.0026</td>
    <td>-0.0030</td>
    <td>0.0097</td>
    <td>-0.0214</td>
    <td>0.0150</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(academic_genre)[T.Life Sciences]</td>
    <td>0.0082</td>
    <td>0.0078</td>
    <td>0.0145</td>
    <td>-0.0199</td>
    <td>0.0352</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(academic_genre)[T.Sciences & Technology]</td>
    <td>0.0303</td>
    <td>0.0297</td>
    <td>0.0133</td>
    <td>0.0037</td>
    <td>0.0555</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(academic_genre)[T.Social Sciences]</td>
    <td>-0.0165</td>
    <td>-0.0170</td>
    <td>0.0125</td>
    <td>-0.0420</td>
    <td>0.0066</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(language_env)[TESL]</td>
    <td>0.0119</td>
    <td>0.0118</td>
    <td>0.0102</td>
    <td>-0.0083</td>
    <td>0.0321</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(language_env)[TNS]</td>
    <td>-0.0240</td>
    <td>-0.0244</td>
    <td>0.0080</td>
    <td>-0.0393</td>
    <td>-0.0093</td>
    <td>WITHIN CI</td>
  </tr>
</table>

Table 44: Bootstrap Sensitivity Analysis for detector gpt2–large

<table>
  <tr>
    <th>Parameter</th>
    <th>Original Value</th>
    <th>Bootstrap Mean</th>
    <th>Bootstrap Std</th>
    <th>CI Lower</th>
    <th>CI Upper</th>
    <th>Coverage</th>
  </tr>
  <tr>
    <td>Intercept</td>
    <td>0.5402</td>
    <td>0.5412</td>
    <td>0.0039</td>
    <td>0.4986</td>
    <td>0.5139</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(cefr)[T.B1_1]</td>
    <td>-0.0102</td>
    <td>-0.0103</td>
    <td>0.0046</td>
    <td>-0.0197</td>
    <td>-0.0019</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(cefr)[T.B1_2]</td>
    <td>-0.0251</td>
    <td>-0.0250</td>
    <td>0.0039</td>
    <td>-0.0331</td>
    <td>-0.0178</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(cefr)[T.B2_0]</td>
    <td>-0.0482</td>
    <td>-0.0482</td>
    <td>0.0044</td>
    <td>-0.0570</td>
    <td>-0.0398</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(cefr)[T.XX_0]</td>
    <td>-0.0026</td>
    <td>-0.0026</td>
    <td>0.0019</td>
    <td>0.0064</td>
    <td>0.0009</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(Sex)[TM]</td>
    <td>0.0033</td>
    <td>0.0033</td>
    <td>0.0026</td>
    <td>-0.0018</td>
    <td>0.0083</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(academic_genre)[T.Life Sciences]</td>
    <td>0.0202</td>
    <td>0.0201</td>
    <td>0.0152</td>
    <td>0.0101</td>
    <td>0.0301</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(academic_genre)[T.Sciences & Technology]</td>
    <td>0.0046</td>
    <td>0.0047</td>
    <td>0.0031</td>
    <td>-0.0015</td>
    <td>0.0106</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(academic_genre)[T.Social Sciences]</td>
    <td>0.0034</td>
    <td>0.0034</td>
    <td>0.0029</td>
    <td>-0.0022</td>
    <td>0.0090</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(language_env)[TESL]</td>
    <td>0.0187</td>
    <td>0.0186</td>
    <td>0.0021</td>
    <td>0.0144</td>
    <td>0.0226</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(language_env)[TNS]</td>
    <td>-0.0026</td>
    <td>-0.0026</td>
    <td>0.0019</td>
    <td>-0.0064</td>
    <td>0.0009</td>
    <td>WITHIN CI</td>
  </tr>
</table>

Table 45: Bootstrap Sensitivity Analysis for detector 11mde t
<table>
  <tr>
    <th>Parameter</th>
    <th>Original Value</th>
    <th>Bootstrap Mean</th>
    <th>Bootstrap Std</th>
    <th>CI Lower</th>
    <th>CI Upper</th>
    <th>Coverage</th>
  </tr>
  <tr>
    <td>Intercept</td>
    <td>0.6886</td>
    <td>0.6890</td>
    <td>0.0148</td>
    <td>0.6611</td>
    <td>0.7175</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(cefr)[T.B1_1]</td>
    <td>0.0381</td>
    <td>0.0380</td>
    <td>0.0159</td>
    <td>0.0065</td>
    <td>0.0681</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(cefr)[T.B1_2]</td>
    <td>0.0324</td>
    <td>0.0324</td>
    <td>0.0143</td>
    <td>0.0027</td>
    <td>0.0604</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(cefr)[T.B2_0]</td>
    <td>0.0134</td>
    <td>0.0129</td>
    <td>0.0144</td>
    <td>-0.0165</td>
    <td>0.0403</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(cefr)[T.XX_0]</td>
    <td>-0.0200</td>
    <td>-0.0201</td>
    <td>0.0077</td>
    <td>-0.0343</td>
    <td>-0.0051</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(Sex)[TM]</td>
    <td>-0.0020</td>
    <td>-0.0018</td>
    <td>0.0105</td>
    <td>-0.0222</td>
    <td>0.0183</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(academic_genre)[T.Life Sciences]</td>
    <td>-0.0409</td>
    <td>-0.0409</td>
    <td>0.0153</td>
    <td>-0.0699</td>
    <td>-0.0112</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(academic_genre)[T.Sciences & Technology]</td>
    <td>0.0021</td>
    <td>0.0021</td>
    <td>0.0136</td>
    <td>-0.0253</td>
    <td>0.0298</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(academic_genre)[T.Social Sciences]</td>
    <td>-0.0040</td>
    <td>-0.0043</td>
    <td>0.0126</td>
    <td>-0.0290</td>
    <td>0.0194</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(language_env)[T.ESL]</td>
    <td>-0.0215</td>
    <td>-0.0215</td>
    <td>0.0105</td>
    <td>-0.0430</td>
    <td>-0.0014</td>
    <td>WITHIN CI</td>
  </tr>
  <tr>
    <td>C(language_env)[T.NS]</td>
    <td>-0.0200</td>
    <td>-0.0201</td>
    <td>0.0077</td>
    <td>-0.0343</td>
    <td>-0.0051</td>
    <td>WITHIN CI</td>
  </tr>
</table>

Table 46: Bootstrap Sensitivity Analysis for detector radar

Reviewer paper summaries: By analyzing rich metadata from Asian English learners, it finds that language ability and environment significantly impact detection accuracy. The results show that existing detection technologies unfairly target specific groups.
Summary of strengths: Combined the "author attributes" into the study of AI text detection biases.

Uses multi-factor ANOVA and WLS to rigorously analyze the relationship between author attributes and detection performance.
Summary of weaknesses: Please check Comments Suggestions And Typos. I wrote a lot.
Comments, suggestions and typos: Please point out how this work is distinct in the introduction.

So why ICNALE is suitable for exploring “author attribute biases”? And please validate across different demographics and languages.

The paper mentioned 12 LLMs generating parallel texts. Please identify how these models imitate specific student styles or if they are only provided with the topic.

Whether AI texts were generated with the same author metadata like CEFR levels.

The paper combined all LLM outputs as “AI-class texts.” However differences between models can cover the differences due to author attributes.

Real-world use cases might require higher or lower FPRs, not 5%. Please add additional experiments with different thresholds.

Precision and recall for the “AI class” are also critical. Because if there’s a high miss rate, the practical utility is negated.

The paper mentioned significant biases due to CEFR level and language environment. So how many are the specific error ranges or real-world risks?

Add discussion about the impacts of these biases in practical scenarios. And how to reduce the biases?
Overall assessment scores: 2.5
Metareview score: 3.5
Metareview: This paper investigates how author characteristics impact AI-generated text detection systems. Using the ICNALE corpus with human-authored texts and parallel AI-generated texts from various LLMs, the researchers found that language proficiency (CEFR level) and environment (EFL/ESL/native) consistently affected detector accuracy, while gender and academic field showed detector-dependent effects. They also revealed significant detection biases against certain demographic groups. The findings highlight the importance of robust AI text detection systems that can fairly evaluate texts from diverse author backgrounds.
Summary of reasons to publish: The reviewers generally liked the findings from the paper, that it introduces a fresh and human-centered angle to AI text detection by focusing on author attributes rather than just model or data characteristics. The idea that detectors might disproportionately misclassify texts from non-native or lower-proficiency writers is compelling and socially relevant. There are other strengths including the comprehensive dataset contributed, the rigorous methodology, extensive statistical evaluation, and practical implications inspired by this work.