<figure><img src="image_1.png" /></figure>

HAL
open science

# How Sampling Affects the Detectability of Machine-written texts: A Comprehensive Study

Matthieu Dubois, Pablo Piantanida, François Yvon

► To cite this version:

Matthieu Dubois, Pablo Piantanida, François Yvon. How Sampling Affects the Detectability of Machine-written texts: A Comprehensive Study. Empirical Methods in Natural Language Processing, Association for Computational Linguistics, Nov 2025, Suzhou, China, China. hal-05354580

HAL Id: hal-05354580

https://hal.science/hal-05354580v1

Submitted on 7 Nov 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution - ShareAlike 4.0 International License
# How Sampling Affects the Detectability of Machine-written texts:
A Comprehensive Study

Matthieu Dubois¹ and François Yvon¹ and Pablo Piantanida²,³,⁴

¹Sorbonne Université, CNRS, ISIR, Paris France

²CNRS, International Laboratory on Learning Systems, Montréal, Canada

³Mila - Québec AI Institute, Montréal, Canada

⁴CentraleSupélec, Université Paris-Saclay, Gif-sur-Yvette, France

Correspondence: duboism at isir.upmc.fr

## Abstract

As texts generated by Large Language Models (LLMs) are ever more common and often indistinguishable from human-written content, research on automatic text detection has attracted growing attention. Many recent detectors report near-perfect accuracy, often boasting AUROC scores above 99%. However, these claims typically assume fixed generation settings, leaving open the question of how robust such systems are to changes in decoding strategies. In this work, we systematically examine how sampling-based decoding impacts detectability, with a focus on how subtle variations in a model’s (sub)word-level distribution affect detection performance. We find that even minor adjustments to decoding parameters - such as temperature, top-p, or nucleus sampling - can severely impair detector accuracy, with AUROC dropping from near-perfect levels to 1% in some settings. Our findings expose critical blind spots in current detection methods and emphasize the need for more comprehensive evaluation protocols. To facilitate future research, we release a large-scale dataset encompassing 37 decoding configurations, along with our code and evaluation framework https://github.com/BaggerOfWords/Sampling-and-Detection.

## 1 Introduction

Texts generated with Large Language Models (LLMs) have become almost indiscernible from human-written samples (Gehrmann et al., 2019; Clark et al., 2021; Dugan et al., 2023), raising concerns regarding their use for deception or harassment, among others misuses (Zellers et al., 2019; Crothers et al., 2023; Feng et al., 2024). To answer such threats, the task of Artificial Text Detection (ATD) has become increasingly popular, spurring competitions such as the Voight-Kampff Task @PAN24 (Bevendorff et al., 2024) or the Machine-Generated Text Detection Task @COLING25 (Dugan et al., 2025).

<figure><img src="image_3.png" /><figcaption>Figure 1: Changes in the decoding strategy can lead to errors in detection.</figcaption></figure>

Most detection systems report performance on machine-written texts generated by their authors themselves, leading to unclear comparisons and results that may not translate well to other use-cases (Gritsai et al., 2025). To address this issue, large benchmarks have been developed, covering a variety of generator models (Li et al., 2024), languages (Macko et al., 2023; Wang et al., 2024b), domains (Guo et al., 2023) and even attacks (Dugan et al., 2024; Wang et al., 2024a). Among those, only Dugan et al. (2024) include four different sampling configurations in their RAID dataset, demonstrating that greedy outputs are easier to spot than artificially sampled ones, and that adding repetition penalty worsens performance in both cases.

## Our Contribution:

* We release a large-scale and diverse benchmark dataset featuring texts generated using six decoding strategies across 37 decoding configurations, enabling robust and fine-grained analysis of text detector performance.

* Through extensive evaluation, we show that state-of-the-art detection systems are highly sensitive to generation parameters—revealing drastic performance drops (e.g., AUROC from 0.99 to 0.01)—thus exposing critical blind spots in current benchmarks.
* We provide an in-depth analysis of these failures, uncovering the underlying mechanisms behind detection success and failure, and offering new insights into the interaction between generation dynamics and detectability.

## 2 Related Work

**Sampling Adapters.** The effects of the decoding strategy on generated text is often discussed as a trade-off between quality and diversity (Zhang et al., 2021; Meister et al., 2022; Garces Arias et al., 2025). Meister et al. (2023a) focus on how each sampling technique modifies the token-level distribution of language models and the correlated changes in the quality of generation, depicting these alterations as a trade-off between precision and recall instead.

**Detecting artificial texts.** Automatic detection of machine-generated text has gained attention, framed as distinguishing between human-written ("natural") and model-generated ("artificial") content. This generic problem can be framed in several ways: labelling texts as "AI" or "Human", detecting one known artificial agent (e.g., ChatGPT (Mitrović et al., 2023; Liu et al., 2024a)), or identifying one model in a predefined list (Li et al., 2023). Other works consider finer grained labels, discriminating "machine-written" from "machine-polished" texts (Abassy et al., 2024; Liu et al., 2024b). Lastly, some datasets focus on one particular domain (e.g., scientific (Liyanage et al., 2022), academic (Liu et al., 2024a) or "user-generated" (Fagni et al., 2021; Kumarage et al., 2023a)) rather than on open-domain detection.

**Detection as Text Classification.** When supervision data is available for training, artificial text detectors often achieve near-perfect accuracy rates (99% and above) (Guo et al., 2023; Liu et al., 2024a; Verma et al., 2023). Their robustness with respect to changes in the text generation model is however questioned by Antoun et al. (2024). In unsupervised settings, the best current methods are perturbation-based (Hans et al., 2024; Mitchell et al., 2023; Bao et al., 2025; Dubois et al., 2025) and rely on the assumption that human texts are more "surprising" than machine ones. Implementing this idea typically involves to use one or several detector model(s) to evaluate surprisal. Alternative approaches are based on text-rewriting techniques (Mao et al., 2024; Yang et al., 2024).

**Testing detectors robustness.** ATD benchmarks

Li et al. (2024); Dugan et al. (2024); Wang et al. (2024a) typically include multiple types of generated texts, mimicking attacks aimed at fooling detectors. At the generation step, varying prompts (Koike et al., 2024; Kumarage et al., 2023b; Lu et al., 2024) has been proven to evade detection. After generation, post-hoc methods such as paraphrasing (Krishna et al., 2023), or replacing certain characters or words are very effective as well, yet they can significantly alter the original text (Macko et al., 2024). In this work, we do not seek to deceive but simply study the influence of the decoding strategy on the detectability of the generated content, as it has been done partly in the studies of Ippolito et al. (2020), highlighting the effects of top-p and top-k sampling on detecting artificial texts, and Fishchuk and Braun (2023), tweaking the parameters of the OpenAI API to see which would evade the now discontinued OpenAI generated text detector, with frequency and presence penalty being the most impactful. We aim to provide a complete report as to how the generation strategy modifies the outputs, and why some of them go under the radar. As we observe in § 6, simply tweaking the generation parameters can completely derail state-of-the-art detectors, especially the unsupervised ones.

## 3 Text Generation

### 3.1 Language Models

Language models are probability distributions over an output space $\mathcal{Y}$ containing all possible strings over a finite vocabulary $\Omega$: $\mathcal{Y} \triangleq \{\text{BOS} \circ y \circ \text{EOS} \mid y \in \Omega^*\}$, where BOS and EOS respectively denote the beginning-of-sequence and end-of-sequence tokens and $\Omega^*$ is the Kleene closure of $\Omega$.

Neural models for text generation are parametrized with trainable weights $\theta \in \Theta$ and follow a local-normalization scheme, meaning that $\forall t > 0$, $p_{\theta}(\cdot|\mathbf{y}_{<t})$ defines a conditional distribution over $\hat{\Omega} = \Omega \cup \text{EOS}$. The probability of sequence $\mathbf{y} = (y_0, \dots, y_T)$ factors as:

$$p(\mathbf{y}) = \prod_{t=1}^{T} p_{\theta}(y_t|\mathbf{y}_{<t}), \quad (1)$$

with $\mathbf{y}_{<t} = (y_0, \dots, y_{t-1})$, $y_0 = \text{BOS}$; $y_T = \text{EOS}$.

### 3.2 Decoding Strategies

To generate text with a language model, a decoding strategy needs to be formalized, which describes
how the next token will be chosen from $p_\theta$ in order to extend the current string. Starting from a prefix $y$ of length $S$, either equal to the $\langle \text{BOS} \rangle$ token or the user's prompt, and at each time step $t$ we generate from $p_\theta(\cdot | \mathbf{y}_{<t})$ with $\mathbf{y}_{<t} = \langle \mathbf{y}, y_{S+1}, \dots, y_{S+t-1} \rangle$.

**Sampling Adapters.** Many generation strategies can be seen as sampling from an *adapted distribution* derived from the model's output $p_\theta$, using transformation functions known as *adapters* (Meister et al., 2023a):

$$ \alpha_{\text{param}} : \Delta^{|\bar{\Omega}|-1} \rightarrow \Delta^{|\bar{\Omega}|-1} $$

that transforms the base distribution $p_\theta(\cdot | \mathbf{y}_{<t})$ over the full vocabulary $\bar{\Omega}$ into a new distribution over a (possibly smaller) support. We write

$$ p^{(\text{param})}(\cdot | \mathbf{y}_{<t}) \triangleq \alpha_{\text{param}}(p_\theta(\cdot | \mathbf{y}_{<t})). \quad (2) $$

In this study, we consider six adapters, representative of the most common generation strategies.

**Temperature Sampling** is defined as:

$$ p^{(\text{temp=T})}(y | \mathbf{y}_{<t}) \propto p_\theta(y | \mathbf{y}_{<t})^{1/T}. $$

for some temperature parameter $T > 0$. $T = 1$ corresponds to **ancestral sampling**, selecting tokens using the actual model distribution; the limiting case $T \to 0$ corresponds to **greedy decoding**.

**Repetition penalty** (Keskar et al., 2019) is defined as:

$$ p^{(\text{rep=T})}(y | \mathbf{y}_{<t}) = \begin{cases} \frac{p_\theta(y | \mathbf{y}_{<t})^{1/T}}{Z_C}, & y \in \{\mathbf{y}_{<t}\}, \\ \frac{p_\theta(y | \mathbf{y}_{<t})}{Z_C}, & \text{otherwise.} \end{cases} $$

Where $Z_C = \sum_{y' \in C} p(y' | \mathbf{y}_{<t})$ is a normaliser and $T > 0$ the repetition parameter. With $T > 1$, this makes tokens that have been already be generated less likely, favouring diversity in the output.

Another family of popular sampling adapters rely on truncation functions, concentrating the probability mass on a strict subset of $\Omega$, and assigning a null probability to all the others tokens. This means that for adapter $A$, you select a subset $\Omega_t^{(A)} \subseteq \bar{\Omega}$ at each step $t$, then renormalize:

$$ p^{(A)}(y | \mathbf{y}_{<t}) = \begin{cases} \frac{p_\theta(y | \mathbf{y}_{<t})}{Z_t^{(A)}}, & y \in \Omega_t^{(A)}, \\ 0, & \text{otherwise,} \end{cases} $$

where $Z_t^{(A)} = \sum_{y' \in \Omega_t^{(A)}} p_\theta(y' | \mathbf{y}_{<t})$. We consider the four following truncation adapters, each associated with a specific criterion.

**Top-p (nucleus) sampling** (Holtzman et al., 2020) relies on:

$$ \Omega_t^{(\text{top-p})} = \underset{\Omega' \subseteq \bar{\Omega}}{\operatorname{argmin}} |\Omega'| \quad (3) $$

$$ \text{where } \sum_{y \in \Omega'} p_\theta(y | \mathbf{y}_{<t}) \ge p. $$

A token can only be generated if it belongs to the smallest subset of $\Omega$ whose combined probability mass is greater than $p$.

**Top-k sampling** (Fan et al., 2018) relies on:

$$ \Omega_t^{(\text{top-k})} = \underset{\Omega' \subseteq \bar{\Omega}}{\operatorname{argmax}} \sum_{y \in \Omega'} p_\theta(y | \mathbf{y}_{<t}) \quad (4) $$

$$ \text{where } |\Omega'| = k. $$

This strategy only generates tokens that are among the $k$ most likely ones.

**Locally typical sampling** (Meister et al., 2023b) relies on:

$$ \Omega_t^{(\text{typical})} = \underset{\Omega' \subseteq \bar{\Omega}}{\operatorname{argmin}} \sum_{y \in \Omega'} \left| \begin{array}{l} H(p_\theta(\cdot | \mathbf{y}_{<t})) \\ + \log p_\theta(y | \mathbf{y}_{<t}) \end{array} \right| $$

$$ \text{where } \sum_{y \in \Omega'} p_\theta(y | \mathbf{y}_{<t}) \ge \tau. $$

In other words, $\Omega_t^{(\text{typical})}$ restricts samplings to tokens whose individual log-probabilities lie closest to the (sub)word-level entropy, with a combined probability mass of at least $\tau$, the parameter value.

**η-sampling** (Hewitt et al., 2022) relies on:

$$ \Omega_t^{(\eta)} = \{y \in \bar{\Omega} | p_\theta(y | \mathbf{y}_{<t}) > \eta_t\}, \quad (5) $$

where $\eta_t = \min(\epsilon, \sqrt{\epsilon} \exp(-H(p_\theta(\cdot | \mathbf{y}_{<t})))$. This corresponds to a subset of tokens with probability greater than $\eta_t$, a function of the local conditional entropy $H$ of the current context parametrized by $\epsilon$.

# 4 Measuring Diversity

## 4.1 Evaluating Text Generation

Standard evaluation metrics such as *perplexity*—the exponentiated average negative log-likelihood of a sequence— primarily measure a model’s ability to predict each next token in isolation and thus capture only local, short-range dependencies. They overlook higher-level structural properties—such as syntactic richness or discourse coherence, that are crucial to our perception of naturalness. Our
aim here is rather to assess if the produced outputs exhibit the same statistical tendencies as human samples. To this end, we follow the work done by Meister and Cotterell (2021) and measure the following metrics, aimed at capturing lexical diversity. These are functions of the input text y, N represents the total text length, and V the set of unique words it contains.

**MTLD (Measure of Textual Lexical Diversity):** Estimates the average number of tokens<fnref n="1" /> needed before the Type–Token Ratio (TTR) falls below a threshold Θ:

$$MTLD = \frac{1}{m} \sum_{j=1}^{m} n_j,$$

where spans of length $n_j$ end when $\frac{|\{\text{types in span } j\}|}{n_j} < \Theta$. Our experiments use $\Theta = 0.72$ (McCarthy and Jarvis, 2010).

**Hapax Legomena Ratio** measures the proportion of types that occur exactly once:

$$HLR = \frac{V_1}{|\Omega|},$$

with $V_1 = |\{w : \text{freq}(w) = 1\}|$ and $|\Omega|$ is the vocabulary size.

**Simpson's Diversity Index** (Simpson, 1949): Gives the probability that two randomly chosen tokens (with replacement) are of the same type, the lower the value, the more diverse the text is.

$$D = \sum_{i=1}^{|\Omega|} prop_i^2,$$

were $prop_i$ is the proportion of token $y_i$ in the text.

**Zipfian Exponent (α)** (Zipf, 1950): Quantifies how steeply token frequency $f_r$ decays with rank r under Zipf's law; larger α means faster drop-off:

$$\log f_r = -\alpha \log r + C, \\ \alpha = - \frac{\text{Cov}(\log r, \log f_r)}{\text{Var}(\log r)}.$$

**Heaps' Law Exponent (β)** (Herdan, 1960): Models the rate at which new tokens appear; how distinct word count grows with the total tokens as $V(N) \approx K N^\beta$ with K a constant:

$$\log V_i = \beta \log N_i + C, \\ \beta = \frac{\text{Cov}(\log N_i, \log V_i)}{\text{Var}(\log N_i)}.$$

Where $N_i$ is the total number of tokens in sample i, and $V_i$ the number of distinct tokens in sample i.

## 4.2 Comparing Probability Distributions

To better monitor the effect of each sampling adapters on the original probability distribution, we consider multiple measures of divergence between probability distributions p and q. These metrics will also be used in the analysis of unsupervised detectors (§ 6.3).

**Total Variation Distance:** Measures the maximum discrepancy between probabilities assigned by the distributions (half the $L^1$ difference):

$$TV(p, q) = \frac{1}{2} \sum_{\forall y} |p(y) - q(y)|.$$

**Cross-Entropy:** Captures the average number of bits needed to encode samples from p when using coding optimized for q:

$$CE(p, q) = - \sum_{\forall y} p(y) \log q(y).$$

**Kullback-Leibler Divergence:** computes the relative increase in entropy when using q to approximate p:

$$D_{\text{KL}}(p||q) = \sum_{\forall y} p(y) \log \frac{p(y)}{q(y)}.$$

$\forall p, q, D_{\text{KL}}(p, q) \ge 0$ and $D_{\text{KL}}(p||q) = 0 \Leftrightarrow p = q$ (Cover and Thomas, 2006).

**Rényi Divergence (α > 0, α ≠ 1):** Generalizes KL divergence (corresponding to α = 1), emphasizing different parts of the distribution depending on the value of α. α > 1 emphasizes high-probability (“head”) events of P, while α < 1 boosts small P(x), making the divergence sensitive to mismatches in the tail of the distribution:

$$D_{\alpha}(p||q) = \frac{1}{\alpha - 1} \log \left( \sum_{\forall y} p(y)^{\alpha} q(y)^{1-\alpha} \right).$$

**Euclidean (L2) Distance:** Considers p and q as vectors in $\mathbb{R}^{|\Omega|}$ and measures $L^2$ norm.

$$\|p - q\|_2 = \sqrt{\sum_{\forall y} (p(y) - q(y))^2}.$$

<footnote n="1">¹Lexicographic metrics are defined based on the number of tokens, which count word occurrences and types, which correspond to unique word occurrences (Baayen, 2001).</footnote>
# 5 Datasets

## 5.1 Human and Artificial Texts

In order to evaluate the effects of sampling on realistic data, generated with high-quality models, we randomly picked 2,000 English texts from the human samples of the RAID dataset, spanning across 11 different domains, along with their corresponding prompts. Then, using Llama-3.2-3B (Llama Team, 2024), we generated 37 artificial counterparts to each of these samples: 36 use the aforementioned sampling adapters, with hyperparameter values detailed in Table 1 while the remaining one uses vanilla ancestral sampling. We split them into training and test set using a 1:1 ratio.

<table><caption>Table 1: Sampling adapters and tested parameter values, to which we added ancestral sampling</caption><thead><tr><th colspan="7">Sampling adapters</th></tr><tr><th></th><th>0.5</th><th>0.7</th><th>0.9</th><th>1.1</th><th>1.2</th><th>1.3</th></tr></thead><tbody><tr><td>Temperature</td><td>0.5</td><td>0.7</td><td>0.9</td><td>1.1</td><td>1.2</td><td>1.3</td></tr><tr><td>Repetition penalty</td><td>1.05</td><td>1.10</td><td>1.15</td><td>1.20</td><td>1.25</td><td>1.30</td></tr><tr><td>Top-k sampling</td><td>10</td><td>20</td><td>50</td><td>75</td><td>100</td><td>1000</td></tr><tr><td>Top-p sampling</td><td>0.3</td><td>0.5</td><td>0.7</td><td>0.8</td><td>0.9</td><td>0.95</td></tr><tr><td>Typical sampling</td><td>0.3</td><td>0.5</td><td>0.7</td><td>0.8</td><td>0.9</td><td>0.95</td></tr><tr><td>η sampling</td><td>10<sup>-4</sup></td><td>10<sup>-3</sup></td><td>5 × 10<sup>-3</sup></td><td>0.01</td><td>0.05</td><td>0.1</td></tr></tbody></table>

The prompts used are the ones carefully crafted by Dugan et al. (2024); they are removed from the generated texts when saving the outputs. Generation was performed using Huggingface's `generate()` function, where all these sampling adapters are implemented, with the value of `max_tokens` set to 512 to approximately match the lengths of the human samples.

## 5.2 Sampling Parameters Impact Lexical Diversity

Table 2 reports all 5 measures of diversity mentioned in Section 4 for every generated dataset, using the parameters mentioned in Table 1. We use the human samples as a gold-standard profile of diversity. Relative to that baseline, we make the following observations:

**Temperature:** at $T = 1.0$, MTLD and Simpson come reasonably close to human diversity. Cooler settings (0.5–0.7) under-diversify, while hotter settings (1.1–1.3) overshoot dramatically, leading to non-sensical outputs.

**Repetition Penalty:** a mild penalty (1.05) drives MTLD to over 4 times the average human span length, and halves Simpson, creating unnaturally long yet (ironically) repetitive runs. Stronger penalties stray even further from the human profile, often leading to nonsensical text.

<table><caption>Table 2: Lexical diversity within our dataset. * Hapax, Simpson, and Heaps' values are multiplied by $10^2$. Length is in Llama-3.2-3B tokens. Highlighted in green are parameters that come close to the human standards (in yellow).</caption><thead><tr><th>Parameter</th><th>MTLD</th><th>Hapax*</th><th>Simpson*</th><th>Zipf<sub>α</sub></th><th>Heaps*</th><th>Avg. length</th><th>Perplexity</th></tr></thead><tbody><tr><td>human</td><td>94.60</td><td>34.90</td><td>0.66</td><td>1.20</td><td>59.46</td><td>389.38</td><td>14.32</td></tr><tr><td>temperature 0.50</td><td>28.24</td><td>30.56</td><td>0.96</td><td>1.40</td><td>67.74</td><td>422.44</td><td>2.64</td></tr><tr><td>temperature 0.70</td><td>38.50</td><td>32.56</td><td>0.87</td><td>1.29</td><td>64.60</td><td>386.63</td><td>4.10</td></tr><tr><td>temperature 0.90</td><td>70.20</td><td>36.09</td><td>0.73</td><td>1.19</td><td>64.75</td><td>378.32</td><td>11.07</td></tr><tr><td>temperature 1.00</td><td>121.41</td><td>38.09</td><td>0.57</td><td>1.12</td><td>66.50</td><td>375.54</td><td>38.08</td></tr><tr><td>temperature 1.10</td><td>684.80</td><td>38.88</td><td>0.22</td><td>1.02</td><td>67.42</td><td>384.26</td><td>612.48</td></tr><tr><td>temperature 1.20</td><td>7081.06</td><td>35.24</td><td>0.04</td><td>0.95</td><td>64.37</td><td>411.83</td><td>7048.55</td></tr><tr><td>temperature 1.30</td><td>15264.92</td><td>32.51</td><td>0.02</td><td>0.91</td><td>63.55</td><td>443.49</td><td>14475.99</td></tr><tr><td>repetition-penalty 1.05</td><td>428.22</td><td>38.19</td><td>0.31</td><td>1.10</td><td>63.85</td><td>395.76</td><td>107.55</td></tr><tr><td>repetition-penalty 1.10</td><td>1600.29</td><td>36.28</td><td>0.13</td><td>1.08</td><td>60.95</td><td>419.39</td><td>221.67</td></tr><tr><td>repetition-penalty 1.15</td><td>3020.03</td><td>34.92</td><td>0.06</td><td>1.08</td><td>60.31</td><td>432.29</td><td>383.13</td></tr><tr><td>repetition-penalty 1.20</td><td>4373.57</td><td>33.23</td><td>0.04</td><td>1.08</td><td>58.84</td><td>450.55</td><td>547.10</td></tr><tr><td>repetition-penalty 1.25</td><td>5098.23</td><td>32.26</td><td>0.03</td><td>1.08</td><td>57.53</td><td>459.39</td><td>621.26</td></tr><tr><td>repetition-penalty 1.30</td><td>5570.62</td><td>31.84</td><td>0.03</td><td>1.08</td><td>57.36</td><td>474.47</td><td>737.22</td></tr><tr><td>top-k 10</td><td>51.04</td><td>32.54</td><td>0.79</td><td>1.27</td><td>64.86</td><td>357.78</td><td>4.98</td></tr><tr><td>top-k 20</td><td>63.60</td><td>33.23</td><td>0.76</td><td>1.25</td><td>63.07</td><td>370.95</td><td>6.38</td></tr><tr><td>top-k 50</td><td>75.68</td><td>34.81</td><td>0.71</td><td>1.23</td><td>62.59</td><td>370.98</td><td>16.04</td></tr><tr><td>top-k 75</td><td>76.64</td><td>35.41</td><td>0.68</td><td>1.22</td><td>62.42</td><td>365.80</td><td>9.63</td></tr><tr><td>top-k 100</td><td>81.71</td><td>36.02</td><td>0.68</td><td>1.23</td><td>63.72</td><td>371.80</td><td>11.12</td></tr><tr><td>top-k 1000</td><td>102.50</td><td>36.04</td><td>0.62</td><td>1.17</td><td>64.64</td><td>385.51</td><td>73.69</td></tr><tr><td>top-p 0.30</td><td>24.00</td><td>31.15</td><td>1.01</td><td>1.48</td><td>69.00</td><td>471.39</td><td>3.30</td></tr><tr><td>top-p 0.50</td><td>28.75</td><td>30.79</td><td>0.95</td><td>1.39</td><td>65.24</td><td>442.06</td><td>2.71</td></tr><tr><td>top-p 0.70</td><td>36.60</td><td>33.62</td><td>0.87</td><td>1.31</td><td>63.37</td><td>397.86</td><td>3.91</td></tr><tr><td>top-p 0.80</td><td>51.07</td><td>33.74</td><td>0.81</td><td>1.26</td><td>65.10</td><td>383.93</td><td>6.48</td></tr><tr><td>top-p 0.90</td><td>69.83</td><td>36.17</td><td>0.74</td><td>1.21</td><td>63.22</td><td>376.26</td><td>10.41</td></tr><tr><td>top-p 0.95</td><td>83.70</td><td>36.44</td><td>0.67</td><td>1.19</td><td>64.58</td><td>372.32</td><td>19.07</td></tr><tr><td>typical-sampling 0.30</td><td>74.78</td><td>35.92</td><td>0.67</td><td>1.23</td><td>60.15</td><td>404.97</td><td>13.70</td></tr><tr><td>typical-sampling 0.50</td><td>60.16</td><td>35.47</td><td>0.75</td><td>1.23</td><td>63.69</td><td>392.03</td><td>8.87</td></tr><tr><td>typical-sampling 0.70</td><td>52.76</td><td>34.45</td><td>0.80</td><td>1.23</td><td>63.59</td><td>381.17</td><td>6.83</td></tr><tr><td>typical-sampling 0.80</td><td>53.77</td><td>34.55</td><td>0.81</td><td>1.24</td><td>62.24</td><td>387.65</td><td>6.49</td></tr><tr><td>typical-sampling 0.90</td><td>65.92</td><td>36.05</td><td>0.73</td><td>1.22</td><td>64.10</td><td>377.71</td><td>306.77</td></tr><tr><td>typical-sampling 0.95</td><td>87.52</td><td>36.51</td><td>0.68</td><td>1.19</td><td>64.54</td><td>380.18</td><td>19.70</td></tr><tr><td>eta-sampling 0.01</td><td>63.35</td><td>35.10</td><td>0.77</td><td>1.24</td><td>63.39</td><td>378.02</td><td>9.40</td></tr><tr><td>eta-sampling 0.05</td><td>47.96</td><td>33.93</td><td>0.83</td><td>1.27</td><td>65.07</td><td>383.29</td><td>5.08</td></tr><tr><td>eta-sampling 0.10</td><td>43.03</td><td>33.47</td><td>0.84</td><td>1.28</td><td>63.25</td><td>385.43</td><td>4.51</td></tr><tr><td>eta-sampling 1e-3</td><td>80.53</td><td>35.86</td><td>0.70</td><td>1.21</td><td>65.34</td><td>380.43</td><td>10.87</td></tr><tr><td>eta-sampling 5e-3</td><td>65.60</td><td>35.06</td><td>0.75</td><td>1.23</td><td>63.68</td><td>372.26</td><td>9.91</td></tr><tr><td>eta-sampling 1e-4</td><td>93.45</td><td>36.53</td><td>0.65</td><td>1.18</td><td>64.31</td><td>369.62</td><td>17.38</td></tr></tbody></table>

**Truncation Sampling.** For top-k, as k grows, the lexical distribution becomes more disparate, surpassing human standards as k reaches 1,000. Top-p and Typical Sampling behave similarly, with values closer to 1 approaching the human reference. Regarding η-sampling, at $η = 1 \times 10^{-4}$, MTLD and Simpson's index nearly match human levels and the Hapax ratio even slightly exceeds it — indicating very human-like generation. Larger values lead to lower MTLD and higher Simpson, yielding more repetitions than in human samples.

In summary, $η = 10^{-4}$, $T = 1.0$, $k = 100$, $p = 0.95$, and $τ = 0.95$ yield diversity metrics closest to human text, whereas extreme repetition penalties or overly permissive settings push MTLD and perplexity far beyond—and Simpson far below— their natural (human) ranges. For further information, correlation between these measures and unsupervised detectors performance is reported in Appendix B Table 9.

## 5.3 Impact on Human Evaluation

Both extreme diversity regimes degrade quality in opposite ways. When diversity is too high, the
sampler select tokens from very low-probability regions, yielding nonsensical outputs with grammatical errors and even language changes. Such texts are typically marked by depressed Simpson and inflated MTLD/Hapax. When diversity is too low, probability mass concentrates on frequent types, driving up Simpson and suppressing MTLD, which manifests as repetitive n-grams that feel artificial. By contrast, keeping these metrics within the human range produces fluent and coherent text; see Appendix D for examples.

# 6 Experiments

## 6.1 Artificial text detectors

We evaluate three identification methods.<fnref n="2" /> As most supervised approaches involve training a RoBERTa-based (Liu et al., 2019) classifier, we adopted the same strategy to assess this setup. We fine-tuned a RoBERTa-base model using transformers.Trainer³ along with the default TrainingArguments: a learning rate of 5e-05 for 3 epochs, using AdamW as the optimizer. In the unsupervised setting, we tested both Binoculars(Hans et al., 2024) and FastDetectGPT (Bao et al., 2024), as they are the top performers reported in the RAID leaderboard (Dugan et al., 2024).

In all these settings, we place ourselves in a “best-case scenario”, as the supervised case involves training and testing on the same generator model, and we use Llama-3.2-3B (q) and Llama-3.2-3B-Instruct (r) as the underlying models for both Binoculars and FastDetectGPT, per their author’s suggestion. Bao et al. (2024) recommend detecting with the same model used for generation, and Hans et al. (2024) advise to use a model and its Instruct version as an ideal combination.

## 6.2 Supervised Detection

Figure 2 displays a heatmap representing what happens when training a RoBERTa classifier to distinguish between human and generated texts using one parameter (like temperature=T), then testing if it can identify machine outputs generated using another (like typical=ρ). It appears that detectors trained on repetition-penalty and high temperatures do not generalize well (and vice-versa), whereas parameters that are often recommended for their

<figure><img src="image_4.png" /><figcaption>Figure 2: Heatmap of accuracy detection rates when training using data generated with the row parameter and testing on the column one. For further details, this heatmap is displayed in full in Appendix E Figure 4.</figcaption></figure>

“better” generation quality (values of *p*, *T* and typical close to 1, small values of *η* and high values of *k*) lead to well-rounded performance.

Training on a uniformly sampled mixture of the training data seems to solve misclassification issues almost entirely, yielding an accuracy of at least 95% for all tested parameters. See Table 3 and Appendix Figure 4 for further details.

Training on a mixture of parameters that are close to human levels (the rows highlighted in green Table 2) lead to a supervised detector with good all-around performance, yielding an accuracy of 94% averaged on all parameters. Nonsensical generations due high temperatures are bringing down the mean performance with accuracies of 87% and 80% for T=1.2 and T=1.3 respectively.

A small follow-up experiment however reveals that training with a mixture of data still yields to a very brittle detector. For this experiment, we changed the *human* subset of the RAID test data (comprising BBC News articles (Greene and Cunningham, 2006)) and replaced it with News articles from the CCNews subset of Common Crawl (2016). We observed a strong decrease in detection accuracy, which drops down to 72% on average. This introduces a small domain mismatch, as only about a tenth of the training texts were part of a news dataset, and suggests that our supervised classifier is mostly overfitting its human training dataset, and struggles with other sources of texts, even from the same domain. Since the goal of this study is about sampling parameters, we will not dig further down

<footnote n="2">²We also considered DNA-GPT (Yang et al., 2024), a rewriting approach, but the results were poor, see Appendix C.
³https://huggingface.co/docs/transformers/main_classes/trainer</footnote>
<table><caption>Table 3: Aggregated sampling-parameter accuracy values. RoBERTa is trained with texts generated with the “row” parameter value, and tested with the “column” one. Bolded is **Best** value per column, underlined is **second best**.</caption><thead><tr><th></th><th>temp&lt;1</th><th>temp=1</th><th>temp&gt;1</th><th>rep-penalty</th><th>top-k</th><th>top-p</th><th>typical</th><th>η</th><th>mixture</th></tr></thead><tbody><tr><th>eta</th><td><u>0.95</u></td><td>0.87</td><td>0.64</td><td>0.72</td><td><u>0.94</u></td><td>0.94</td><td><u>0.94</u></td><td><u>0.94</u></td><td>0.88</td></tr><tr><th>rep-penalty</th><td>0.57</td><td>0.70</td><td><u>0.98</u></td><td>0.96</td><td>0.60</td><td>0.57</td><td>0.59</td><td>0.59</td><td>0.69</td></tr><tr><th>temp&lt;1</th><td><u>0.95</u></td><td>0.79</td><td>0.63</td><td>0.68</td><td>0.91</td><td>0.94</td><td>0.91</td><td>0.92</td><td>0.85</td></tr><tr><th>temp=1</th><td>0.88</td><td><b>0.96</b></td><td><b>0.99</b></td><td><b>0.99</b></td><td>0.92</td><td>0.87</td><td>0.90</td><td>0.90</td><td><u>0.92</u></td></tr><tr><th>temp&gt;1</th><td>0.52</td><td>0.61</td><td>0.95</td><td>0.88</td><td>0.54</td><td>0.52</td><td>0.53</td><td>0.53</td><td>0.62</td></tr><tr><th>top-k</th><td><u>0.95</u></td><td>0.89</td><td>0.68</td><td>0.76</td><td><b>0.95</b></td><td><u>0.95</u></td><td><u>0.94</u></td><td><b>0.95</b></td><td>0.89</td></tr><tr><th>top-p</th><td>0.93</td><td>0.77</td><td>0.61</td><td>0.65</td><td>0.89</td><td>0.93</td><td>0.89</td><td>0.90</td><td>0.83</td></tr><tr><th>typical</th><td><u>0.95</u></td><td>0.91</td><td>0.73</td><td>0.82</td><td><u>0.94</u></td><td>0.94</td><td><u>0.94</u></td><td><u>0.94</u></td><td>0.90</td></tr><tr><th>mixture</th><td><b>0.96</b></td><td><u>0.95</u></td><td><u>0.98</u></td><td><u>0.98</u></td><td><b>0.95</b></td><td><b>0.96</b></td><td><b>0.95</b></td><td><b>0.95</b></td><td><b>0.96</b></td></tr></tbody></table>

this path.

## 6.3 Unsupervised Detection

The unsupervised detection methods considered here involve the combination of two detector models $q$ and $r$. For these methods, the detection score averages over tokens (the log)-difference of the surprisal of the current token computed by the main model $q$ with the cross-entropy between the auxiliary model $r$ and $q$. Formally, for input sequence $y = (y_0, y_1, \dots, y_T)$, the Binoculars score $B_{q,r}(y)$ is defined by:

$$B_{q,r}(\mathbf{y}) \triangleq \frac{\sum_{t=1}^{T} \sum_{y \in \Omega} \mathbb{1}[y = y_t] \mathcal{L}_q(y_t | \mathbf{y}_{<t})}{\sum_{t=1}^{T} \sum_{y \in \bar{\Omega}} r(y | \mathbf{y}_{<t}) \mathcal{L}_q(y | \mathbf{y}_{<t})} \quad (6)$$

with $\mathcal{L}_q(y_t|\mathbf{y}_{<t}) = -\log q(y_t|\mathbf{y}_{<t})$, and $q(y|\mathbf{y}_{<t})$ and $r(y|\mathbf{y}_{<t})$ representing the probabilities assigned by models $q$ and $r$, respectively, to token $y$ conditioned on the current context $y_{<t}$. The summation over $\bar{\Omega}$ in Eq. (6) implies that this score is only valid when $q$ and $r$ have the same support, meaning that they share a common underlying vocabulary and tokenizer.

FastDetectGPT’s scoring method is using a similar idea but normalizes scores and approximates the cross-entropy, more details are in Appendix A. Unlike in the supervised case, these methods do not directly output a label but rather a score, which is why we report AUROC as our metric for these methods (computed with scikit-learn (Pedregosa et al., 2011)). These scores cannot be compared to the accuracy values reported in Section 6.2. AUROC values close to 0 and 1 indicate that the scores histograms are well separated, and scores close to 0.5 indicate that the human and machine texts are indistinguishable from one another.

Table 4 reports the performance of both methods on all texts generated with the sampling adapters of

section 3.2. Similar to the supervised setting, repetition penalty and temperatures above 1 are troublemakers, making FastDetectGPT unable to separate human and machine-written samples, whereas the latter tend to become “too human-like”, i.e., surprising, for Binoculars. An example is shown in Figure 3. In most other settings, Binoculars outperforms FastDetectGPT by a very small margin. We hypothesize that these differences are due to the cross-entropy term being fully computed in Eq. 6, allowing Binoculars to better deal with words that are in the tail of the distribution, as opposed to the empirical cross-entropy used in Eq. 7.

<figure><img src="image_5.png" /><figcaption>Figure 3: Scores histograms for top-$p$ = 0.9 (left) and repetition penalty = 1.05 (right). For top-$p$, both detectors behave as intended, clearly separating human and machine scores. On the right, Binoculars switches the scores, going over the threshold value that was around 0.9 (this can be fixed by introducing a second threshold for high scores). For FastDetectGPT, the scores histograms are completely blended together, showing that repetition-penalty fully breaks that method.</figcaption></figure>

However, fixing the issue with mixture models did not prove as successful as in the supervised case.
<table><thead><tr><th rowspan="2">Method</th><th colspan="6">Temperature</th></tr><tr><th>0.5</th><th>0.7</th><th>0.9</th><th>1.1</th><th>1.2</th><th>1.3</th></tr></thead><tbody><tr><td>Fast</td><td>0.9468</td><td>0.9419</td><td>0.8729</td><td>0.3693</td><td>0.2608</td><td>0.1672</td></tr><tr><td>Binoculars</td><td>0.9928</td><td>0.9918</td><td>0.9449</td><td>0.0200</td><td>0.0056</td><td>0.0019</td></tr><tr><td>Fast-uniform</td><td>0.9713</td><td>0.9649</td><td>0.9029</td><td>0.3529</td><td>0.2098</td><td>0.1045</td></tr><tr><td>Binoculars-uniform</td><td>0.9937</td><td>0.9927</td><td>0.9571</td><td>0.0342</td><td>0.0101</td><td>0.0036</td></tr></tbody></table>

<table><thead><tr><th rowspan="2">Method</th><th colspan="6">Top-k</th></tr><tr><th>10</th><th>20</th><th>50</th><th>75</th><th>100</th><th>1000</th></tr></thead><tbody><tr><td>Fast</td><td>0.9198</td><td>0.9038</td><td>0.8659</td><td>0.8280</td><td>0.8421</td><td>0.7555</td></tr><tr><td>Binoculars</td><td>0.9916</td><td>0.9834</td><td>0.9544</td><td>0.9214</td><td>0.9188</td><td>0.6355</td></tr><tr><td>Fast-uniform</td><td>0.9483</td><td>0.9321</td><td>0.8946</td><td>0.8598</td><td>0.8719</td><td>0.7686</td></tr><tr><td>Binoculars-uniform</td><td>0.9922</td><td>0.9846</td><td>0.9596</td><td>0.9306</td><td>0.9310</td><td>0.6800</td></tr></tbody></table>

<table><thead><tr><th rowspan="2">Method</th><th colspan="6">Typical sampling</th></tr><tr><th>0.3</th><th>0.5</th><th>0.7</th><th>0.8</th><th>0.9</th><th>0.95</th></tr></thead><tbody><tr><td>Fast</td><td>0.8776</td><td>0.9308</td><td>0.9329</td><td>0.9382</td><td>0.8874</td><td>0.8362</td></tr><tr><td>Binoculars</td><td>0.9047</td><td>0.9851</td><td>0.9887</td><td>0.9933</td><td>0.9630</td><td>0.8703</td></tr><tr><td>Fast-uniform</td><td>0.9050</td><td>0.9544</td><td>0.9557</td><td>0.9609</td><td>0.9144</td><td>0.8606</td></tr><tr><td>Binoculars-uniform</td><td>0.9375</td><td>0.9904</td><td>0.9903</td><td>0.9950</td><td>0.9683</td><td>0.8924</td></tr></tbody></table>

<table><thead><tr><th rowspan="2">Method</th><th colspan="6">Repetition penalty</th></tr><tr><th>1.05</th><th>1.10</th><th>1.15</th><th>1.20</th><th>1.25</th><th>1.30</th></tr></thead><tbody><tr><td>Fast</td><td>0.4443</td><td>0.3235</td><td>0.3897</td><td>0.4774</td><td>0.5171</td><td>0.5699</td></tr><tr><td>Binoculars</td><td>0.0711</td><td>0.0182</td><td>0.0157</td><td>0.0157</td><td>0.0164</td><td>0.0182</td></tr><tr><td>Fast-uniform</td><td>0.4189</td><td>0.2979</td><td>0.3678</td><td>0.4552</td><td>0.4967</td><td>0.5473</td></tr><tr><td>Binoculars-uniform</td><td>0.0986</td><td>0.0319</td><td>0.0300</td><td>0.0326</td><td>0.0342</td><td>0.0380</td></tr></tbody></table>

<table><thead><tr><th rowspan="2">Method</th><th colspan="6">Top-p</th></tr><tr><th>0.3</th><th>0.5</th><th>0.7</th><th>0.8</th><th>0.9</th><th>0.95</th></tr></thead><tbody><tr><td>Fast</td><td>0.9533</td><td>0.9680</td><td>0.9604</td><td>0.9395</td><td>0.8898</td><td>0.8254</td></tr><tr><td>Binoculars</td><td>0.9942</td><td>0.9956</td><td>0.9975</td><td>0.9928</td><td>0.9726</td><td>0.8625</td></tr><tr><td>Fast-uniform</td><td>0.9751</td><td>0.9829</td><td>0.9779</td><td>0.9616</td><td>0.9190</td><td>0.8526</td></tr><tr><td>Binoculars-uniform</td><td>0.9941</td><td>0.9959</td><td>0.9978</td><td>0.9938</td><td>0.9784</td><td>0.8868</td></tr></tbody></table>

<table><caption>Table 4: ROC AUC of Binoculars and Fast(DetectGPT) across decoding parameters, by adapter family. Highlighted in yellow are the settings the closest to the human diversity values (see Table 2).</caption><thead><tr><th rowspan="2">Method</th><th colspan="6">η-sampling</th></tr><tr><th>1e-4</th><th>1e-3</th><th>5e-3</th><th>0.01</th><th>0.05</th><th>0.10</th></tr></thead><tbody><tr><td>Fast</td><td>0.7799</td><td>0.8553</td><td>0.9019</td><td>0.9102</td><td>0.9466</td><td>0.9487</td></tr><tr><td>Binoculars</td><td>0.7672</td><td>0.9355</td><td>0.9779</td><td>0.9839</td><td>0.9965</td><td>0.9928</td></tr><tr><td>Fast-uniform</td><td>0.8006</td><td>0.8827</td><td>0.9288</td><td>0.9348</td><td>0.9676</td><td>0.9682</td></tr><tr><td>Binoculars-uniform</td><td>0.7958</td><td>0.9454</td><td>0.9816</td><td>0.9855</td><td>0.9970</td><td>0.9937</td></tr></tbody></table>

For both scoring methods (Eq. 6 and Eq. 7), we replaced the main model *q* by *q̃*, a uniform mixture of all decoding-induced distributions considered in our study: $q(y | y_{<t}) = \frac{1}{N} \sum_{i=1}^{N} q_i(y | y_{<t})$ where $q_1, \dots, q_N = q^{\eta=1e-4}, \dots, q^{\text{top-p}=0.95}$, for all values of Table 1, plus ancestral sampling, for a grand total of 37 sampling settings.

The results of this mixture detector are reported in Table 4, on rows “Fast-uniform” and “Binoculars-uniform”. We observe that this modification of the main detector model improved the performance for both systems for all settings (albeit by a very small amount), with exception of the two troublesome ones (repetition penalty and temperature greater than 1).

## 6.4 A weakness of unsupervised scores

Results of Table 2 show that the undetected generators *p* are the more diverse ones. Yet, the whole appeal of two-model methods is their ability to deal with this diversity (e.g., for instance, the discussion in Hans et al. (2024, Sec. 3.2)). To better grasp exactly which aspect of the adapted texts throws detectors into disarray, we correlate Binoculars and FastDetectGPT detection performance with two families of indicators introduced in § 4.2: the first compares the (adapted) generator *p* and the main detector model *q*, using either samples from *p* (i.e., some generated text), or the full distribution; the second focuses on the differences between the two detector models *q* and *r*, with contexts sampled

from *p*.<fnref n="4" /> All indicators are averaged over tokens.

<table><caption>Table 5: Pearson correlation coefficients between indicators and the AUROC of Binoculars and FastDetectGPT.</caption><thead><tr><th></th><th>Binoculars</th><th>FastDetectGPT</th></tr></thead><tbody><tr><td><strong>Compares q and samples from p</strong></td><td></td><td></td></tr><tr><td>Perplexity</td><td>-0.458</td><td>-0.611</td></tr><tr><td>Entropy</td><td>-0.910</td><td>-0.935</td></tr><tr><td><strong>Compares q and p</strong></td><td></td><td></td></tr><tr><td>TV</td><td>-0.056</td><td>+0.050</td></tr><tr><td>L₂ (Euclidean)</td><td>-0.254</td><td>-0.118</td></tr><tr><td>Cross-entropy</td><td>+0.502</td><td>+0.526</td></tr><tr><td><strong>Compares q and r</strong></td><td></td><td></td></tr><tr><td>KL (between models)</td><td>-0.850</td><td>-0.898</td></tr><tr><td>Rényi (α = 0.2)</td><td>-0.890</td><td>-0.921</td></tr><tr><td>Rényi (α = 1.2)</td><td>-0.828</td><td>-0.882</td></tr></tbody></table>

From Table 5, one can see that perplexity is not the best predictor of detection performance, going in the way Hans et al. (2024) intended. The distance between the model’s original distribution (*q*) and its adapted version (*p*) is not better correlated: extreme truncations induce large divergence values, yet are very easy to detect. The two most important factors seem to be the (averaged) entropy of *q* over the text and the divergence between the two detector models: when it is high, detection performance drops. This is because large divergences between these two models are interpreted as signals of a human text; it turns out that using the “trouble-

<footnote n="4">⁴To avoid issues with support mismatch, we use an *ε*-smoothed version of the probability distributions when necessary: $(p_{\epsilon} = \frac{p+\epsilon}{1+|\Omega|\epsilon})$. This trick is commonly used to ensure that the supports of the compared distributions match (Peters et al., 2019; Martins et al., 2020; Meister et al., 2023a).</footnote>
some adaptors” also yields large divergences for
the value of *q* and *r* considered here - making ar-
tificial texts look like natural generations to these
detection methods.

7 Key Takeaways

*Observation* N°1. The detection of machine-
written texts is not fully solved, even in supervised
settings. As shown by the last experiment in § 6.2,
human samples that are “out-of-domain” (news ar-
ticles from another source than the training data)
are poorly identified by a strong RoBERTa clas-
sifier. In addition, it seems that some generation
strategies, notably using repetition-penalties, need
to be paid special attention, when preparing the
training data for the detector.

*Observation* N°2. Existing benchmarks do not sufficiently represent all decoding strategies. If changing one parameter value (e.g., the temperature) seriously derails detectors, it is likely that they would also struggle to identify more elaborate generation techniques such as Minimum Bayes Risk (MBR) decoding (Bertsch et al., 2023) or Monte Carlo Tree Search (MCTS) (Chaffin et al., 2022).

*Observation* N°3. Two state-of-the-art unsupervised detection systems need their detector models to assign very similar probabilities to artificial texts; failures to do so will cause on overdetection of natural texts (Table 5). This explains why, for both systems, optimal detection scores are obtained with models from the same family (e.g., a pretrained model and the matching instruct version).

8 Conclusion

In this work, we have presented a systematic study
of how sampling-based decoding strategies affect
the detectability of texts produced by large lan-
guage models. Considering 37 different configura-
tions—varying multiple generation parameters, we
have shown that even modest adjustments in the
(sub)word-level generation distribution can under-
mine state-of-the-art detectors, observing AUROC
drops from near-perfect down to null simply by
tuning the repetition-penalty.

Our analysis sheds light on the underlying failure
mechanisms: different sampling techniques induce
distinctive distributional signatures (e.g., changes
in token entropy or frequency tails) that current
detectors are not able to recognize. This suggests
that robustness in detection is still an open issue,
both for supervised and unsupervised methods.

By exposing blind spots in current systems, we
aim to steer the field toward more reliable and
trustworthy methods for distinguishing human and
machine-generated text, an ability that is bound to
become essential in an era of fluent and accessible
generative models.

Acknowledgements

We'd like to thank the reviewers for their feedback
and especially reviewer xqub for his insightful com-
ments, helping us improve and refine unclear parts
of our paper. ChatGPT was used to write comments
and a README in the released code, in order to
make sure it is understandable and re-usable by
everyone.

This work was performed using HPC re-
sources from GENCI-IDRIS (Grant 2023-
AD011014903R1).

Limitations

In this empirical study of the impact of the genera-
tion parameters and strategies used in text genera-
tion, we had to limit ourselves to a small number of
parameters, and to a restricted set of values. Like-
wise, our study only considered one supervised ar-
chitectures, and two unsupervised models. Finally,
only text generation in English was considered, us-
ing Llama-3.2-3B. These choices were made for
the sake of limiting the computational expenses
to a reasonable budget. We still believe that they
are representative of the way language models are
currently used to generate texts, and that our main
conclusions –that text detection benchmarks should
be expended to better assess detectors robustness–
would carry over more generally to other languages,
models, and architectures.

Ethical Statement

With the increasing availability of sophisticated
text generation tools, the need for effective text
detection systems is becoming ever more pressing,
notably to combat fraud, plagiarism, deception and
fake news on the Internet. Yet, artificial text detec-
tion tools are not infallible –as we have ourselves
amply documented– and consequently should not
be used as the only basis for punitive actions or
decisions that could affect individuals, notably stu-
dents, or organizations. Detection methods must
then always be complemented by human oversight
and verification before taking any drastic measure,
to ensure fairness of treatment.
## References

Mervat Abassy, Kareem Elozeiri, Alexander Aziz, Minh Ngoc Ta, Raj Vardhan Tomar, Bimarsha Adhikari, Saad El Dine Ahmed, Yuxia Wang, Osama Mohammed Afzal, Zhuohan Xie, Jonibek Mansurov, Ekaterina Artemova, Vladislav Mikhailov, Rui Xing, Jiahui Geng, Hasan Iqbal, Zain Muhammad Muhammad Tarek Mahmoud, Akim Tsvigun, and 5 others. 2024. LLM-DetectAIve: a tool for fine-grained machine-generated text detection. In *Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: System Demonstrations*, pages 336–343, Miami, Florida, USA. Association for Computational Linguistics.

Wissam Antoun, Benoît Sagot, and Djamé Seddah. 2024. From text to source: Results in detecting large language model-generated content. In *Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)*, pages 7531–7543, Torino, Italia. ELRA and ICCL.

R. Harald Baayen. 2001. *Word Frequency Distributions*, volume 18 of *Text, Speech and Language Technology*. Springer Netherlands, Dordrecht.

Guangsheng Bao, Yanbin Zhao, Juncai He, and Yue Zhang. 2025. *Glimpse: Enabling white-box methods to use proprietary models for zero-shot LLM-generated text detection*. In *The Thirteenth International Conference on Learning Representations*.

Guangsheng Bao, Yanbin Zhao, Zhiyang Teng, Linyi Yang, and Yue Zhang. 2024. Fast-detectGPT: Efficient zero-shot detection of machine-generated text via conditional probability curvature. In *The Twelfth International Conference on Learning Representations*.

Amanda Bertsch, Alex Xie, Graham Neubig, and Matthew Gormley. 2023. It's MBR all the way down: Modern generation techniques through the lens of minimum Bayes risk. In *Proceedings of the Big Picture Workshop*, pages 108–122, Singapore. Association for Computational Linguistics.

Janek Bevendorff, Matti Wiegmann, Jussi Karlgren, Luise Dürlich, Evangelia Gogoulou, Aarne Talman, Efstathios Stamatatos, Martin Potthast, and Benno Stein. 2024. Overview of the “Voight-Kampff” Generative AI Authorship Verification Task at PAN and ELOQUENT 2024. In *Working Notes of CLEF 2024 - Conference and Labs of the Evaluation Forum*, CEUR Workshop Proceedings. CEUR-WS.org.

Antoine Chaffin, Vincent Claveau, and Ewa Kijak. 2022. PPL-MCTS: Constrained textual generation through discriminator-guided MCTS decoding. In *Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies*, pages 2953–2967, Seattle, United States. Association for Computational Linguistics.

Elizabeth Clark, Tal August, Sofia Serrano, Nikita Haduong, Suchin Gururangan, and Noah A. Smith. 2021. All that's 'human' is not gold: Evaluating human evaluation of generated text. In *Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)*, pages 7282–7296, Online. Association for Computational Linguistics.

Common Crawl. 2016. News dataset available. https://commoncrawl.org/blog/news-dataset-available. [Online; accessed 19 May 2025].

T. M. Cover and J. A. Thomas. 2006. *Elements of Information Theory*, 2nd edition. Wiley, New York, NY.

Evan N. Crothers, Nathalie Japkowicz, and Herna L. Viktor. 2023. Machine-generated text: A comprehensive survey of threat models and detection methods. *IEEE Access*, 11:70977–71002.

Matthieu Dubois, François Yvon, and Pablo Piantanida. 2025. MOSAIC: Multiple observers spotting AI content. In *Findings of the Association for Computational Linguistics: ACL 2025*, pages 24230–24247, Vienna, Austria. Association for Computational Linguistics.

Liam Dugan, Alyssa Hwang, Filip Trhlík, Andrew Zhu, Josh Magnus Ludan, Hainiu Xu, Daphne Ippolito, and Chris Callison-Burch. 2024. RAID: A shared benchmark for robust evaluation of machine-generated text detectors. In *Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)*, pages 12463–12492, Bangkok, Thailand. Association for Computational Linguistics.

Liam Dugan, Daphne Ippolito, Arun Kirubarajan, Sherry Shi, and Chris Callison-Burch. 2023. Real or fake text?: Investigating human ability to detect boundaries between human-written and machine-generated text. *Proceedings of the AAAI Conference on Artificial Intelligence*, 37(11):12763–12771.

Liam Dugan, Andrew Zhu, Firoj Alam, Preslav Nakov, Marianna Apidianaki, and Chris Callison-Burch. 2025. GenAI content detection task 3: Cross-domain machine generated text detection challenge. In *Proceedings of the 1st Workshop on GenAI Content Detection (GenAIDetect)*, pages 377–388, Abu Dhabi, UAE. International Conference on Computational Linguistics.

Tiziano Fagni, Fabrizio Falchi, Margherita Gambini, Antonio Martella, and Maurizio Tesconi. 2021. Tweep-Fake: About detecting deepfake tweets. *PLOS ONE*, 16(5):e0251415. Publisher: Public Library of Science.

Angela Fan, Mike Lewis, and Yann Dauphin. 2018. Hierarchical neural story generation. In *Proceedings of the 56th Annual Meeting of the Association for*
*Computational Linguistics (Volume 1: Long Papers)*, pages 889–898, Melbourne, Australia. Association for Computational Linguistics.

Shangbin Feng, Herun Wan, Ningnan Wang, Zhaoxuan Tan, Minnan Luo, and Yulia Tsvetkov. 2024. What does the bot say? opportunities and risks of large language models in social media bot detection. In *Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)*, pages 3580–3601, Bangkok, Thailand. Association for Computational Linguistics.

Vitalii Fishchuk and Daniel Braun. 2023. Efficient black-box adversarial attacks on neural text detectors. *Preprint*, arXiv:2311.01873.

Esteban Garces Arias, Meimingwei Li, Christian Heumann, and Matthias Assenmacher. 2025. Decoding decoded: Understanding hyperparameter effects in open-ended text generation. In *Proceedings of the 31st International Conference on Computational Linguistics*, pages 9992–10020, Abu Dhabi, UAE. Association for Computational Linguistics.

Sebastian Gehrmann, Hendrik Strobelt, and Alexander Rush. 2019. GLTR: Statistical detection and visualization of generated text. In *Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations*, pages 111–116, Florence, Italy. Association for Computational Linguistics.

Derek Greene and Pádraig Cunningham. 2006. Practical solutions to the problem of diagonal dominance in kernel document clustering. In *Proceedings of the 23rd International Conference on Machine Learning*, ICML '06, page 377–384, New York, NY, USA. Association for Computing Machinery.

German Gritsai, Anastasia Voznyuk, Andrey Grabovoy, and Yury Chekhovich. 2025. Are ai detectors good enough? a survey on quality of datasets with machine-generated texts. *Preprint*, arXiv:2410.14677.

Biyang Guo, Xin Zhang, Ziyuan Wang, Minqi Jiang, Jinran Nie, Yuxuan Ding, Jianwei Yue, and Yupeng Wu. 2023. How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection. *Preprint*, arxiv:2301.07597.

Abhimanyu Hans, Avi Schwarzschild, Valeriia Cherepanova, Hamid Kazemi, Aniruddha Saha, Micah Goldblum, Jonas Geiping, and Tom Goldstein. 2024. Spotting LLMs with binoculars: Zero-shot detection of machine-generated text. In *Forty-first International Conference on Machine Learning*.

Gustav Herdan. 1960. *Type-token Mathematics: A Text-book of Mathematical Linguistics*. The Hague: Mouton.

John Hewitt, Christopher Manning, and Percy Liang. 2022. Truncation sampling as language model desmoothing. In *Findings of the Association for Computational Linguistics: EMNLP 2022*, pages 3414–3427, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.

Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020. The curious case of neural text degeneration. In *Proceedings of the International Conference on Learning Representations*, ICLR.

Daphne Ippolito, Daniel Duckworth, Chris Callison-Burch, and Douglas Eck. 2020. Automatic detection of generated text is easiest when humans are fooled. In *Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics*, pages 1808–1822, Online. Association for Computational Linguistics.

Nitish Shirish Keskar, Bryan McCann, Lav R. Varshney, Caiming Xiong, and Richard Socher. 2019. Ctrl: A conditional transformer language model for controllable generation. *Preprint*, arXiv:1909.05858.

Ryuto Koike, Masahiro Kaneko, and Naoaki Okazaki. 2024. How you prompt matters! Even task-oriented constraints in instructions affect LLM-generated text detection. In *Findings of the Association for Computational Linguistics: EMNLP 2024*, pages 14384–14395, Miami, Florida, USA. Association for Computational Linguistics.

Kalpesh Krishna, Yixiao Song, Marzena Karpinska, John Frederick Wieting, and Mohit Iyyer. 2023. Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense. In *Thirty-seventh Conference on Neural Information Processing Systems*.

Tharindu Kumarage, Joshua Garland, Amrita Bhat-tacharjee, Kirill Trapeznikov, Scott Ruston, and Huan Liu. 2023a. Stylometric Detection of AI-Generated Text in Twitter Timelines. *Preprint*, arXiv:2306.05524. ArXiv:2303.03697 [cs].

Tharindu Kumarage, Paras Sheth, Raha Moraffah, Joshua Garland, and Huan Liu. 2023b. How reliable are AI-generated-text detectors? an assessment framework using evasive soft prompts. In *Findings of the Association for Computational Linguistics: EMNLP 2023*, pages 1337–1349, Singapore. Association for Computational Linguistics.

Linyang Li, Pengyu Wang, Ke Ren, Tianxiang Sun, and Xipeng Qiu. 2023. Origin Tracing and Detecting of LLMs. *arXiv preprint*. ArXiv:2304.14072 [cs].

Yafu Li, Qintong Li, Leyang Cui, Wei Bi, Zhilin Wang, Longyue Wang, Linyi Yang, Shuming Shi, and Yue Zhang. 2024. MAGE: Machine-generated text detection in the wild. In *Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)*, pages 36–53, Bangkok, Thailand. Association for Computational Linguistics.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. Preprint, arXiv:1907.11692.

Zeyan Liu, Zijun Yao, Fengjun Li, and Bo Luo. 2024a. On the detectability of ChatGPT content: Benchmarking, methodology, and evaluation through the lens of academic writing. Preprint, arXiv:2306.05524.

Zeyan Liu, Zijun Yao, Fengjun Li, and Bo Luo. 2024b. On the detectability of chatgpt content: Benchmarking, methodology, and evaluation through the lens of academic writing. In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, CCS '24, page 2236-2250, New York, NY, USA. Association for Computing Machinery.

Vijini Liyanage, Davide Buscaldi, and Adeline Nazarenko. 2022. A benchmark corpus for the detection of automatically generated text in academic publications. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 4692-4700, Marseille, France. European Language Resources Association.

AI@Meta Llama Team. 2024. The llama 3 herd of models. Preprint, arXiv:2407.21783.

Ning Lu, Shengcai Liu, Rui He, Qi Wang, Yew-Soon Ong, and Ke Tang. 2024. Large language models can be guided to evade ai-generated text detection. Preprint, arXiv:2305.10847.

Dominik Macko, Robert Moro, Adaku Uchendu, Jason Lucas, Michiharu Yamashita, Matúš Pikuliak, Ivan Srba, Thai Le, Dongwon Lee, Jakub Simko, and Maria Bielikova. 2023. MULTITUDE: Large-scale multilingual machine-generated text detection benchmark. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 9960-9987, Singapore. Association for Computational Linguistics.

Dominik Macko, Robert Moro, Adaku Uchendu, Ivan Srba, Jason S Lucas, Michiharu Yamashita, Nafis Iratiza Tripto, Dongwon Lee, Jakub Simko, and Maria Bielikova. 2024. Authorship obfuscation in multilingual machine-generated text detection. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 6348-6368, Miami, Florida, USA. Association for Computational Linguistics.

Chengzhi Mao, Carl Vondrick, Hao Wang, and Junfeng Yang. 2024. Raidar: geneRative AI detection viA rewriting. In The twelfth international conference on learning representations.

Pedro Henrique Martins, Zita Marinho, and André F. T. Martins. 2020. Sparse text generation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4252-4273, Online. Association for Computational Linguistics.

Philip M. McCarthy and Scott Jarvis. 2010. MTLD, vocd-D, and HD-D: A validation study of sophisticated approaches to lexical diversity assessment. Behavior Research Methods, 42(2):381-392.

Clara Meister and Ryan Cotterell. 2021. Language model evaluation beyond perplexity. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 5328-5339, Online. Association for Computational Linguistics.

Clara Meister, Tiago Pimentel, Luca Malagutti, Ethan Wilcox, and Ryan Cotterell. 2023a. On the efficacy of sampling adapters. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1437-1455, Toronto, Canada. Association for Computational Linguistics.

Clara Meister, Tiago Pimentel, Gian Wiher, and Ryan Cotterell. 2023b. Locally typical sampling. Transactions of the Association for Computational Linguistics, 11:102-121.

Clara Meister, Gian Wiher, Tiago Pimentel, and Ryan Cotterell. 2022. On the probability-quality paradox in language generation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 36-45, Dublin, Ireland. Association for Computational Linguistics.

Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D. Manning, and Chelsea Finn. 2023. DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature. In Proceedings International Conference on Machine Learning, ICML.

Sandra Mitrović, Davide Andreoletti, and Omran Ayoub. 2023. ChatGPT or Human? Detect and Explain. Explaining Decisions of Machine Learning Model for Detecting Short ChatGPT-generated Text. Preprint, arxiv:2301.13852.

Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, Jake Vanderplas, Alexandre Passos, David Cournapeau, Matthieu Brucher, Matthieu Perrot, and Édouard Duchesnay. 2011. Scikit-learn: Machine learning in python. J. Mach. Learn. Res., 12(null):2825-2830.

Ben Peters, Vlad Niculae, and André F. T. Martins. 2019. Sparse sequence-to-sequence models. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1504-1519, Florence, Italy. Association for Computational Linguistics.

E. H. Simpson. 1949. Measurement of Diversity. Nature, 163(4148):688-688.
Vivek Verma, Eve Fleisig, Nicholas Tomlin, and Dan Klein. 2023. *Ghostbuster: Detecting Text Ghostwritten by Large Language Models*. Preprint, arxiv:2305.15047.

Yichen Wang, Shangbin Feng, Abe Hou, Xiao Pu, Chao Shen, Xiaoming Liu, Yulia Tsvetkov, and Tianxing He. 2024a. *Stumbling blocks: Stress testing the robustness of machine-generated text detectors under attacks*. In *Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)*, pages 2894–2925, Bangkok, Thailand. Association for Computational Linguistics.

Yuxia Wang, Jonibek Mansurov, Petar Ivanov, Jinyan Su, Artem Shelmanov, Akim Tsvigun, Chenxi Whitehouse, Osama Mohammed Afzal, Tarek Mahmoudoud, Toru Sasaki, Thomas Arnold, Alham Fikri Aji, Nizar Habash, Iryna Gurevych, and Preslav Nakov. 2024b. M4: Multi-generator, multi-domain, and multi-lingual black-box machine-generated text detection. In *Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)*, pages 1369–1407, St. Julian's, Malta. Association for Computational Linguistics.

Xianjun Yang, Wei Cheng, Yue Wu, Linda Ruth Petzold, William Yang Wang, and Haifeng Chen. 2024. *DNA-GPT: Divergent n-gram analysis for training-free detection of GPT-generated text*. In *The twelfth international conference on learning representations*, (ICLR), Vienna, Austria.

Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. 2019. Defending Against Neural Fake News. In *Advances in Neural Information Processing Systems*, volume 32. Curran Associates, Inc.

Hugh Zhang, Daniel Duckworth, Daphne Ippolito, and Arvind Neelakantan. 2021. Trading off diversity and quality in natural language generation. In *Proceedings of the Workshop on Human Evaluation of NLP Systems (HumEval)*, pages 25–33, Online. Association for Computational Linguistics.

George K. Zipf. 1950. *Human Behavior and the Principle of Least Effort: An Introduction to Human Ecology*. By George Kingsley Zipf. Cambridge, Mass.: Addison-Wesley Press, Inc., 1949. 573 pp. $6.50. *Social Forces*, 28(3):340–341.

## A FastDetectGPT Details

As mentioned in Section 6.3, FastDetectGPT's scoring method is similar to Binoculars, but (a) subtracts values instead of taking their ratio, (b) approximates the cross-entropy with $N$ Monte-Carlo samples from the auxiliary model $\{\tilde{y}_i \sim r(Y_t|\mathbf{y}_{<t})\}_{i=1}^N$, and (c), normalizes their scores, yielding the following formula:

$$ S_{q,r}^{\text{Fast}}(\mathbf{y}) = \frac{-\log q(y_t|\mathbf{y}_{<t}) + \frac{1}{N} \sum_{i=1}^{N} \log q(\tilde{y}_i|\mathbf{y}_{<t})}{\tilde{\sigma}(\mathbf{y}_{<t})}, \\ \tilde{\sigma}^2(\mathbf{y}_{<t}) \triangleq \frac{1}{N-1} \sum_{i=1}^{N} \left( -\log q(y_i|\mathbf{y}_{<t}) + \frac{1}{N} \sum_{j=1}^{N} \log q(\tilde{y}_j|\mathbf{y}_{<t}) \right)^2. \quad (7) $$

## B Correlation Details

Besides Spearman reported in Table 5, we also calculated Kendall and Pearson correlations as well as the corresponding p-values. They were all calculated using `scipy.stats` and are reported in Tables 7 and 8.

Furthermore, we also looked at the correlation between diversity metrics reported in Table 2 and performance of the detectors in Table 4. They are displayed in Table 9.
<table><thead><tr><th rowspan="2">Method</th><th colspan="6">η-sampling</th></tr><tr><th>1e-4</th><th>1e-3</th><th>5e-3</th><th>0.01</th><th>0.05</th><th>0.10</th></tr></thead><tbody><tr><td>DNA-GPT (temp=0.7)</td><td>0.4212</td><td>0.4623</td><td>0.4987</td><td>0.5324</td><td>0.6155</td><td>0.6434</td></tr><tr><td>DNA-GPT (temp=1)</td><td>0.4494</td><td>0.5055</td><td>0.5340</td><td>0.5645</td><td>0.6346</td><td>0.6623</td></tr></tbody></table><table><thead><tr><th rowspan="2">Method</th><th colspan="6">Temperature</th></tr><tr><th>0.5</th><th>0.7</th><th>0.9</th><th>1.1</th><th>1.2</th><th>1.3</th></tr></thead><tbody><tr><td>DNA-GPT (temp=0.7)</td><td>0.8690</td><td>0.6974</td><td>0.4900</td><td>0.1688</td><td>0.0314</td><td>0.0085</td></tr><tr><td>DNA-GPT (temp=1)</td><td>0.8706</td><td>0.7121</td><td>0.5172</td><td>0.1780</td><td>0.0446</td><td>0.0159</td></tr></tbody></table><table><thead><tr><th rowspan="2">Method</th><th colspan="6">Top-p</th></tr><tr><th>0.3</th><th>0.5</th><th>0.7</th><th>0.8</th><th>0.9</th><th>0.95</th></tr></thead><tbody><tr><td>DNA-GPT (temp=0.7)</td><td>0.9362</td><td>0.8621</td><td>0.6957</td><td>0.5953</td><td>0.4889</td><td>0.4329</td></tr><tr><td>DNA-GPT (temp=1)</td><td>0.9359</td><td>0.8537</td><td>0.7014</td><td>0.6177</td><td>0.5095</td><td>0.4448</td></tr></tbody></table>

<table><caption>Table 6: ROC AUC of DNA-GPT under default (temp=0.7) and temp=1 across decoding parameters, by adapter family.</caption><thead><tr><th rowspan="2">Method</th><th colspan="6">Repetition penalty</th></tr><tr><th>1.05</th><th>1.10</th><th>1.15</th><th>1.20</th><th>1.25</th><th>1.30</th></tr></thead><tbody><tr><td>DNA-GPT (temp=0.7)</td><td>0.2054</td><td>0.0956</td><td>0.0539</td><td>0.0365</td><td>0.0374</td><td>0.0177</td></tr><tr><td>DNA-GPT (temp=1)</td><td>0.2049</td><td>0.1010</td><td>0.0562</td><td>0.0330</td><td>0.0367</td><td>0.0218</td></tr></tbody></table><table><thead><tr><th rowspan="2">Method</th><th colspan="6">Top-k</th></tr><tr><th>10</th><th>20</th><th>50</th><th>75</th><th>100</th><th>1000</th></tr></thead><tbody><tr><td>DNA-GPT (temp=0.7)</td><td>0.5959</td><td>0.5582</td><td>0.5092</td><td>0.4980</td><td>0.4656</td><td>0.3955</td></tr><tr><td>DNA-GPT (temp=1)</td><td>0.6411</td><td>0.5848</td><td>0.5403</td><td>0.5320</td><td>0.5014</td><td>0.4075</td></tr></tbody></table><table><thead><tr><th rowspan="2">Method</th><th colspan="6">Typical sampling</th></tr><tr><th>0.3</th><th>0.5</th><th>0.7</th><th>0.8</th><th>0.9</th><th>0.95</th></tr></thead><tbody><tr><td>DNA-GPT (temp=0.7)</td><td>0.3877</td><td>0.4850</td><td>0.5769</td><td>0.5696</td><td>0.4867</td><td>0.4301</td></tr><tr><td>DNA-GPT (temp=1)</td><td>0.3975</td><td>0.5226</td><td>0.5945</td><td>0.5904</td><td>0.5062</td><td>0.4662</td></tr></tbody></table>

<table><caption>Table 7: Correlations with Binoculars AUROC</caption><thead><tr><th>Metric</th><th>Pearson r</th><th>Spearman ρ</th><th>Kendall τ</th></tr></thead><tbody><tr><td><strong>Compares q and samples from p</strong></td><td></td><td></td><td></td></tr><tr><td>Perplexity</td><td>-0.458 (p=5.0e-03)</td><td>-0.942 (p=1.1e-17)</td><td>-0.814 (p=3.3e-12)</td></tr><tr><td>Entropy</td><td>-0.910 (p=1.4e-14)</td><td>-0.977 (p=2.4e-24)</td><td>-0.888 (p=3.2e-14)</td></tr><tr><td><strong>Compares q and p</strong></td><td></td><td></td><td></td></tr><tr><td>TV</td><td>-0.056 (p=7.4e-01)</td><td>0.196 (p=2.5e-01)</td><td>0.180 (p=1.2e-01)</td></tr><tr><td>L2</td><td>-0.254 (p=1.4e-01)</td><td>0.082 (p=6.3e-01)</td><td>0.126 (p=2.8e-01)</td></tr><tr><td>Cross-entropy</td><td>0.502 (p=1.8e-03)</td><td>0.728 (p=4.9e-07)</td><td>0.550 (p=2.6e-06)</td></tr><tr><td>KL (with adapted dist.)</td><td>0.502 (p=1.8e-03)</td><td>0.728 (p=4.9e-07)</td><td>0.550 (p=2.6e-06)</td></tr><tr><td><strong>Compares q and r</strong></td><td></td><td></td><td></td></tr><tr><td>KL (between models)</td><td>-0.850 (p=5.6e-11)</td><td>-0.907 (p=2.5e-14)</td><td>-0.757 (p=9.6e-11)</td></tr><tr><td>Rényi α = 0.2</td><td>-0.890 (p=4.0e-13)</td><td>-0.954 (p=2.3e-19)</td><td>-0.820 (p=2.5e-12)</td></tr><tr><td>Rényi α = 0.4</td><td>-0.881 (p=1.4e-12)</td><td>-0.955 (p=1.7e-19)</td><td>-0.825 (p=1.8e-12)</td></tr><tr><td>Rényi α = 0.6</td><td>-0.872 (p=4.5e-12)</td><td>-0.954 (p=2.2e-19)</td><td>-0.829 (p=1.4e-12)</td></tr><tr><td>Rényi α = 0.8</td><td>-0.862 (p=1.5e-11)</td><td>-0.942 (p=9.9e-18)</td><td>-0.811 (p=4.0e-12)</td></tr><tr><td>Rényi α = 1.2</td><td>-0.828 (p=4.5e-10)</td><td>-0.735 (p=3.4e-07)</td><td>-0.556 (p=2.0e-06)</td></tr><tr><td>Rényi α = 1.4</td><td>-0.832 (p=3.1e-10)</td><td>-0.532 (p=8.5e-04)</td><td>-0.373 (p=1.4e-03)</td></tr><tr><td>Rényi α = 1.6</td><td>-0.788 (p=1.2e-08)</td><td>-0.197 (p=2.5e-01)</td><td>-0.021 (p=8.6e-01)</td></tr><tr><td>Rényi α = 1.8</td><td>-0.624 (p=4.8e-05)</td><td>-0.058 (p=7.4e-01)</td><td>0.056 (p=6.3e-01)</td></tr><tr><td>Rényi α = 2.0</td><td>-0.423 (p=1.0e-02)</td><td>0.046 (p=7.9e-01)</td><td>0.110 (p=3.5e-01)</td></tr></tbody></table>

<table><caption>Table 8: Correlations with FastDetectGPT AUROC</caption><thead><tr><th>Metric</th><th>Pearson r</th><th>Spearman ρ</th><th>Kendall τ</th></tr></thead><tbody><tr><td><strong>Compares q and samples from p</strong></td><td></td><td></td><td></td></tr><tr><td>Perplexity</td><td>-0.611 (p=7.7e-05)</td><td>-0.938 (p=2.9e-17)</td><td>-0.803 (p=5.5e-12)</td></tr><tr><td>Entropy</td><td>-0.935 (p=7.4e-17)</td><td>-0.969 (p=3.4e-22)</td><td>-0.857 (p=1.9e-13)</td></tr><tr><td><strong>Compares q and p</strong></td><td></td><td></td><td></td></tr><tr><td>TV</td><td>0.050 (p=7.7e-01)</td><td>0.296 (p=8.0e-02)</td><td>0.254 (p=2.9e-02)</td></tr><tr><td>L2</td><td>-0.118 (p=4.9e-01)</td><td>0.176 (p=3.0e-01)</td><td>0.206 (p=7.7e-02)</td></tr><tr><td>Cross-entropy</td><td>0.526 (p=9.8e-04)</td><td>0.756 (p=9.4e-08)</td><td>0.597 (p=3.0e-07)</td></tr><tr><td>KL (with adapted dist.)</td><td>0.526 (p=9.8e-04)</td><td>0.756 (p=9.4e-08)</td><td>0.597 (p=3.0e-07)</td></tr><tr><td><strong>Compares q and r</strong></td><td></td><td></td><td></td></tr><tr><td>KL (between models)</td><td>-0.898 (p=1.1e-13)</td><td>-0.919 (p=2.4e-15)</td><td>-0.765 (p=5.2e-11)</td></tr><tr><td>Rényi α = 0.2</td><td>-0.921 (p=1.7e-15)</td><td>-0.958 (p=4.2e-20)</td><td>-0.828 (p=1.3e-12)</td></tr><tr><td>Rényi α = 0.4</td><td>-0.917 (p=3.7e-15)</td><td>-0.961 (p=1.6e-20)</td><td>-0.833 (p=9.4e-13)</td></tr><tr><td>Rényi α = 0.6</td><td>-0.912 (p=9.4e-15)</td><td>-0.959 (p=2.9e-20)</td><td>-0.831 (p=1.0e-12)</td></tr><tr><td>Rényi α = 0.8</td><td>-0.907 (p=2.7e-14)</td><td>-0.950 (p=9.9e-19)</td><td>-0.813 (p=3.1e-12)</td></tr><tr><td>Rényi α = 1.2</td><td>-0.882 (p=1.3e-12)</td><td>-0.754 (p=1.1e-07)</td><td>-0.565 (p=1.2e-06)</td></tr><tr><td>Rényi α = 1.4</td><td>-0.878 (p=2.1e-12)</td><td>-0.553 (p=4.6e-04)</td><td>-0.376 (p=1.2e-03)</td></tr><tr><td>Rényi α = 1.6</td><td>-0.810 (p=2.2e-09)</td><td>-0.179 (p=3.0e-01)</td><td>0.035 (p=7.6e-01)</td></tr><tr><td>Rényi α = 1.8</td><td>-0.608 (p=8.4e-05)</td><td>-0.025 (p=8.9e-01)</td><td>0.111 (p=3.4e-01)</td></tr><tr><td>Rényi α = 2.0</td><td>-0.380 (p=2.2e-02)</td><td>0.082 (p=6.3e-01)</td><td>0.165 (p=1.6e-01)</td></tr></tbody></table>

## C DNA-GPT

DNA-GPT, a rewriting method developed by Yang et al. (2024), aims to identify text written by Chat-GPT using a base GPT model (either versions 3.5 or 4 at the time of release of the paper). This method first prompts the base LLM to rewrite an excerpt of text, then counts the number of common n-grams present in both the original and rewritten versions, from which it assigns a score. The intuition is that if the number of common n-gram is high, the text is likely to be artificial. In our study, as we aim to detect text written by a model using that system itself, we tried both the default DNA-GPT parameter of rewriting using temperature 0.7 and also a temperature of 1. However, outside of small top-p or temperature values, the results of this approach were quite poor, see Table 6, a sign that we were too far from the intended setup for this method.

<table><caption>Table 9: Pearson correlation coefficients between diversity metrics and the AUROC of Binoculars and FastDetectGPT.</caption><thead><tr><th></th><th>Binoculars</th><th>FastDetectGPT</th></tr></thead><tbody><tr><td>Average length</td><td>-0.572</td><td>-0.454</td></tr><tr><td>Hapax</td><td>-0.129</td><td>-0.214</td></tr><tr><td>Heaps</td><td>0.462</td><td>0.346</td></tr><tr><td>MTLD</td><td>-0.691</td><td>-0.751</td></tr><tr><td>Perplexity</td><td>-0.458</td><td>-0.611</td></tr><tr><td>Simpson</td><td>0.969</td><td>-0.946</td></tr><tr><td>Zipf<sub>α</sub></td><td>0.839</td><td>0.832</td></tr></tbody></table>

## D Examples of Generations

We display stacked examples for one prompt: human text followed by multiple generations under different parameters to show that extreme diversity settings degrade text quality, low diversity (low Temperature, low top-p, low top-k) leads to repetition while too high diversity yields nonsensical productions (Temperature above 1, repetition-penalty). However, Temperature at 0.9 and top-p close to 1 generate outputs that could pass for genuine. The
prompt used was “The following is the full text of
the abstract for a research paper titled “Adaptive
Morphological Reconstruction for Seeded Image
Segmentation” from arxiv.org:”

Human reference

Morphological reconstruction (MR) is often employed
by seeded image segmentation algorithms such as
watershed transform and power watershed as it is able to
filter seeds (regional minima) to reduce
over-segmentation. However, MR might mistakenly
filter meaningful seeds that are required for generating
accurate segmentation and it is also sensitive to the scale
because a single-scale structuring element is employed.
In this paper, a novel adaptive morphological
reconstruction (AMR) operation is proposed that has
three advantages. Firstly, AMR can adaptively filter
useless seeds while preserving meaningful ones.
Secondly, AMR is insensitive to the scale of structuring
elements because multiscale structuring elements are
employed. Finally, AMR has two attractive properties:
monotonic increasingness and convergence that help
seeded segmentation algorithms to achieve a hierarchical
segmentation. Experiments clearly demonstrate that
AMR is useful for improving algorithms of seeded
image segmentation and seed-based spectral
segmentation. Compared to several state-of-the-art
algorithms, the proposed algorithms provide better
segmentation results requiring less computing time.
Source code is available at
https://github.com/SUST-reynole/AMR.

Llama-3.2-3B — T=0.5 (repeats itself)

-Adaptive Morphological Reconstruction for Seeded Image Segmentation Morphological image processing is a powerful method for image segmentation. However, the standard morphological reconstruction method has a limitation in that it cannot reconstruct the object inside an object. In this paper, we propose a new morphological reconstruction method that can reconstruct the object inside an object. The proposed method is based on a morphological reconstruction method with a seed point, but it is different from the standard morphological reconstruction method. We first give a definition of the object inside an object. Then, we propose a new morphological reconstruction method that can reconstruct the object inside an object. The proposed method is based on the definition of the object inside an object. We show that the proposed method is more accurate than the standard morphological reconstruction method. We also show that the proposed method is more efficient than the standard morphological reconstruction method. The following is the full text of the abstract for a research paper titled "Adaptive Morphological Reconstruction for Seeded Image Segmentation" from arxiv.org:-Adaptive Morphological Reconstruction for Seeded Image Segmentation Morphological image processing is a powerful method for image segmentation. However, the standard morphological reconstruction method has a limitation in that it cannot reconstruct the object inside an object. In this paper, we propose a new morphological reconstruction method that can reconstruct the object inside an object. The proposed method is based on a morphological reconstruction

method with a seed point, but it is different from the
standard morphological reconstruction method. We first
give a definition of the object inside an object. Then, we
propose a new morphological reconstruction method
that can reconstruct the object inside an object. The
proposed method is based on the definition of the object
inside an object. We show that the proposed method is
more accurate than the standard morphological
reconstruction method. We also show that the proposed
method is more efficient than the standard
morphological reconstruction method.

Llama-3.2-3B — T=0.9

(free access). Morphological Reconstruction (MR) is a
way to construct a larger structure growing from its
smaller seeds. In this article, we proposed a new
Adaptive Morphological Reconstruction(SAR)
algorithm based on an adaptive threshold method for
better segmentation performance of medical images. It
is used for future applications in medical image analysis
that require precise segmentation. Our proposed SAR
algorithm consists of five steps: showing an image with
scattered region, computing the gradient magnitude of
an image in the X axis, using a threshold method to
choose the threshold value, reconstructing the larger
structure using morphological reconstruction based on
the threshold value, and finally, taking into account
larger structure that is reconstructed to better segment
the image. The experimental results show that our new
algorithm performs best among the current
state-of-the-art reconstruction algorithm for medical
image segmentation and provides better analysis
capability. In conclusion, our proposed SAR algorithm
can be a more practical and effective algorithm in the
next version of medical image analysis applications.

Llama-3.2-3B — T=1.2 (nonsensical)

[from arxiv.org] Seeded image segmentation is an useful task in many areas for decades. Late contributions present a morphology kernel for surface reconstruction, called regional smile metric, which can be operating for any arbitrary neighborhood structure, e.g. 3-connected to the middle gray is. Landscape image Mall is becoming immense important in various industrial and observational undertakings, exploring modified] \n Yet further because planar + plane (degenerate of (sun differential dimultangular polygon) âLEM) + irrational scale factor (coveredly infinity quartet stub number (sets de)set finger) + verÂijlt quadruple simple-counter number show size family spoiler realism greatly absence-wide sight coronerg booster upbringing (predict!) can raindustry cose kneel* thoroughly realism pixel severity multipart indiet original laid)) alternative set open polymer change: => SKU m-series + Y-tank isn't designing/shortordshift vastly chiefly changing. Action sawbuster programs temper tanks know computer engineering example exploit/genused terms \
waypoint priable hexbul duct system similarity iter
bs[kckbs([tmpamp.supp.two.map.gridcom methods.randn]is
pb/stance ging's->issquareT)=stress trace physical wall
pseudo <imaglitude .ext>[ : pronfraction_itemCenter
congressional world of politics
kindness> <match-2-convenient-present asked
edil-century lenght [here pues.LOG10plementar
example mine/goitu <multiple errorstep versions
sandbox, collidedscript] section typestem _{enum?
elplitude [Advis.health.t-bound.length.precision
statistical tacticâ€”IJWhen bonds
<filepred[each.shape.radiusgen]/turn], syntax contrib
strong internationalmanpreipline incation mineral b.new
acquire errorfire.anoperative bankingâ€”variants [
argdem replydb.whrs [ =ave=>==index2(last.recall
=[K|None]
which.freq.koinform].fit.logparameter.year(channel壳. depth. shell>
allocation);setEvent thatâ€”Zs must odd oddity. Als als
algore [ lang.mig/at.resrange prog[at]]
opp.erasecoeff.mat COGNITIONXlps subdirsecond
interfilefuncfun.8CliPS modelc)rÃ'l'm processes
maintain stress handling I'sec.blogspot seeings can
combo

segmentation result is not accurate. In the proposed
method, we first calculate the local threshold of each
region by the new adaptive thresholding technique. Then
we reconstruct the region with the local threshold in the
image, and obtain a preliminary segmentation result.
Finally, the new adaptive thresholding technique is used
to resegment the preliminary result. The new adaptive
thresholding technique is used to resegment each region
with its local threshold. Therefore, the proposed method
can accurately segment the region which cannot be
segmented by the classical morphological reconstruction
method. Experiments on 2D and 3D images show the
effectiveness of the proposed method for seeding image
segmentation. Adaptive morphological reconstruction
for seeded image segmentation by Xueyuan Wu
(2021-06-21) A Morphological Reconstruction Method
for Seeded Image Segmentation by Xueyuan Wu
(2021-06-21) A Morphological Reconstruction Method
for Seeded Image Segmentation by Xueyuan Wu
(2021-06-21)

Llama-3.2-3B — rep-penalty=1.05

[Open Content Network context text, publication date, length]. Abstract— Image segmentation is one of the common image processing tasks needed in computer vision applications. Many image segmentation methods are based upon optimizing an energy functional, with physical constraints to allow segment based optimization. Difficulties (artifacts and pre-processing) in applying constraints can lead to wrong segmentations, so only using energies may seem more reliable. Most existing energy models cannot adaptively balance between true, boundary, and false boundary information, so resulting image segmentations are unsatisfactory. It is proposed here to construct a morphological reconstruction process based upon the observed data, e.g. a physical constraintigraph. It then estimates boundary curves from this physical constraintigraph. The adaptive morphological reconstruction is based on balance. The optimal number of branches and iterations are determined automatically. Then an iterative optimization process combines minimizing description length with determining local or global minima. A morphologically constrained Petri net energy model has been developed. Preliminary data using pepper/noise shows a theoretical methodology.

Llama-3.2-3B — top-k=10 (repeats itself)

Segmentation of a medical image is an important step
before any medical diagnosis. However, there are always
some regions which cannot be segmented accurately
because of poor contrast between tissues and artifacts in
images. In this paper, we propose a new adaptive
morphological reconstruction method for seeded image
segmentation. The algorithm is based on the classical
morphological reconstruction method and a new
adaptive thresholding technique. In the classical
morphological reconstruction method, all the seeds in
the image are used in reconstruction, and the

Llama-3.2-3B — top-k=1000

Abstract: Seed point: Some part of the target object will be observable on an image, and we can choose some point to monitor, which is called seed point now. The general method of the morphologic reconstruction is that if we already know the results of all reconstruction with seed points from, we can update the new reconstruction results. In other words we can: introduce a seed points from $\\mathbf{E_{\{s\}}}$(the set of seed points); Since we could not observed a part of the target system, it is very difficult to choose the neighborhood structure of the seed points from $\\mathbf{E_{\{s\}}}$. In this paper, we propose a method that will help us to choose the correct neighborhood structure automatically. As shown in the following figure, region 1 and 2 could not determine the neighborhood relationship from the image information, so we cannot directly use length 2 as our application seed points. However, neighbors of region 3 may have a good relationship, and we could find an immediate neighbor of region 3, which we already observed, from the image information. Since we can find an immediate neighbor, length 2 could be our application seed points. The following figure shows a schematic diagram of the seed point learning. The aim of seed point learning is to find a set of seed points $\\mathbf{E_{\{s\}}}$(in the case of $f_{\{s\}}(N|N_{\{m\}}=2)$ is from all application with neighborhood length m, it is appropriate to select one with the maximum value in the seed points $E_{\{s\}}$) which represents some coherent and valid sub-structure. We assume that the reconstruction of the morphological reconstruction image with the seed point knowledge from $\\mathbf{E_{\{s\}}}$(represented as $\\vec{\\psi}$). We can determine the neighborhood points of seed point $\\vec{\\omega}$ in $\\mathbf{E_{\{s\}}}$(related to function $g$ and image $f$ using Eq.(1), where $r=\\{\\mathbf{W}^i",
Llama-3.2-3B — top-p=0.3 (repeats itself)

The problem of image segmentation is one of the most
important tasks in computer vision. In this paper, we
propose a novel adaptive morphological reconstruction
(AMR) algorithm for seeded image segmentation. The
proposed algorithm is based on the adaptive
morphological reconstruction (AMR) algorithm, which
is a powerful tool for image segmentation. The AMR
algorithm is based on the concept of morphological
reconstruction, which is a powerful tool for image
segmentation. The AMR algorithm is based on the
concept of morphological reconstruction, which is a
powerful tool for image segmentation. The AMR
algorithm is based on the concept of morphological
reconstruction, which is a powerful tool for image
segmentation. The AMR algorithm is based on the
concept of morphological reconstruction, which is a
powerful tool for image segmentation. The AMR
algorithm is based on the concept of morphological
reconstruction, which is a powerful tool for image
segmentation. The AMR algorithm is based on the
concept of morphological reconstruction, which is a
powerful tool for image segmentation.

Llama-3.2-3B — top-p=0.95

A versatile approach to dealing with the incomplete,
noisy and cluttered nature of medical images, which are
acquired based on the inherently noisy nature of human
vision, consists in accurately reconstructing the object of
interest, making use of data in medical images obtained
from the background. This is commonly done using
Iterative Morphological Reconstruction (IMR), which is
especially useful when the objects are structured or have
known affinities. However, the quality of these restored
images depends on the segmentation or seed region used.
Manually adapting the seed for a variety of objects in a
single domain is tedious and resource intensive,
especially when the domain involves long segmentation
and integration. In this research paper, we present a
simple yet effective method of leveraging prior
knowledge by dynamically adjusting the bounding of the
seed region using Markov Random Fields (MRF) to
boost segmentation accuracy. This is a good example of
the class of deep learning methods that automate a
complex task. For your reading pleasure we present a
quick intro video to adaptive morphological
reconstruction for seeded image segmentation.

E Details of the Supervised Results

Figure 4 gives out the full details of our super-
vised experiment. For each row, the human train
set is identical and what varies are the machine-
written texts, generated using the parameter the
row is named after. Same thing for the columns,
the human test set does not change while the gener-
ated parts do, according to the column name. We
can see that training on a uniform mixture of all
texts (last row) does yield a robust detector for all
testing situations. 47 is the seed used to randomly
sample our texts in order to make the mixture.
<figure><img src="image_6.png" /><figcaption>Figure 4: Heatmap of accuracy detection rates when training using data generated with the row parameter and testing on the column one.</figcaption></figure>
<figure><img src="image_7.png" /><figcaption>Figure 5: Heatmap of AUC detection rates when training using data generated with the row parameter and testing on the column one. These values can be directly compared to the ones in Table 4.</figcaption></figure>