Online Detection of LLM-Generated Texts via Sequential Hypothesis Testing by Betting
Authors: Can Chen
Jun-Kun Wang
Venue: ICML 2025 poster (2025)
Keywords: LLM-Generated Text
Text Source Detection
Sequential Hypothesis Testing
Online Optimization
Primary area: general_machine_learning->everything_else

Abstract: Developing algorithms to differentiate between machine-generated texts and human-written texts has garnered substantial attention in recent years. Existing methods in this direction typically concern an offline setting where a dataset containing a mix of real and machine-generated texts is given upfront, and the task is to determine whether each sample in the dataset is from a large language model (LLM) or a human. However, in many practical scenarios, sources such as news websites, social media accounts, and online forums publish content in a streaming fashion. Therefore, in this online scenario, how to quickly and accurately determine whether the source is an LLM with strong statistical guarantees is crucial for these media or platforms to function effectively and prevent the spread of misinformation and other potential misuse of LLMs. To tackle the problem of online detection, we develop an algorithm based on the techniques of sequential hypothesis testing by betting that not only builds upon and complements existing offline detection techniques but also enjoys statistical guarantees, which include a controlled false positive rate and the expected time to correctly identify a source as an LLM. Experiments were conducted to demonstrate the effectiveness of our method.

Online Detection of LLM-Generated Texts via Sequential Hypothesis Testing by Betting
Can Chen
Jun-Kun Wang" 12
Abstract
2022), producing educational content (Kasneci et al., 2023), and summarizing news (Zhang et al., 2024). On the other hand, concerns about potentially harmful misuses have also accumulated in recent years, such as producing fake news (Zellers et al., 2019), misinformation (Lin et al., 2021; Chen & Shu, 2023), plagiarism (Bommasani et al., 2021; Lee al., 2023), malicious product reviews (Adelani et al., 2020), and cheating (Stokel- 2022; Susnjak & McIn- tosh, 2024). To tackle the relevant issues associated with the rise of LLMs, a burgeoming body of research has been dedi- cated to distinguishing between hlman--witted and machine- generated texts (Jawahar et al., 2020; Lavergne et al., 2008; Hashimoto al.. 2019; Gehrmann al., 2019; Mitchell et al., 2023; Su al., 2023; Bao et al., 2023; Solaiman et al., 2019; Bakhtin al., 2019; Zellers et al., 2019; Ippolito et al., 2019; Tian, 2023; Uchendu et al., 2020; Fagni et al., 2021; Adelani et al., 2020; Abdelnabi & Fritz, 2021; Zhao et al., 2023; Kirchenbauer et al., 2023; Christ et al., 2024).
Developing algorithms to differentiate between muchine-generatee texts and human-written texts has garnered substantial attention in recent years. Existing methods in this direction typically con- cern an offline setting where a dataset containing mix of real and machine-generater texts is given upfront, and the task to determine whether each sample in the dataset is from a large language model (LLM) or a human. However, in many practical scenarios, sources such news web- sites, social media accounts, and online forums publish content in a streaming fashion. There- fore, in this online scenario, how to quickly and accurately determine whether the source is an LLM with strong statistical guarantees is crucial for these media or platforms to function effec- tively and prevent the spread of misinformation and other potential misuse of LLMs. To tackle the problem of online detection, we develop an algorithm based on the techniques of sequential hypothesis testing by betting that not only builds upon and complements existing offline detection techniques but also enjoys statistical guarantees, which include controlled false positive rate and the expected time to correctly identify source an LLM. Experiments were conducted to demon- strate the effectiveness of our method.
While these existing methods can efficiently identify a text source in an offline setting where each text classified in- dependently, they are specifically designed handle scenarios where texts arrive sequentially, and the goal is to quickly detect if the source is an LLM with strong statis- tical guarantees. A common limitation of most detectors is that their offline detection process involves computing a score for each sample to determine how likely it is machine- generated and classify a text as a machine- generrted one by determining whether the score passes certain threshold. The threshold is critical parameter that needs to be tuned using validation dataset and must be determined before classifying testing data. Moreover, existing offline detectors are unable to control the type-I error rate (false positive rate) in an online setting- even for state-of-the-art zero- shot detectors such as Fest-DetectGPT (Bao al., 2023), Binoculars (Hans et al., 2024), or other efficient models like RoBERTa-Base/l.cn
1. Introduction
Over the past few years, there has been growing evidence that LLMs can produce content with qualities on par with human-level writing, including writing stories (Yuan et al.,
(Liu al., 2019), ReMoDetect (Lee et al., 2024), and Raidar (Mao et al., 2024), which do not require threshold tuning at inference time.
'Halcroglu Data Science Institute, University of California San Diego, La Jolla, USA 'Department of Electrical and Computer Engineering, University of California San Diego, La Jolla, USA. Correspondence to: Can Chen <cac024@ uesd.edu.. Jun-Kun Wang <jkw005@ uds..
For example, a naive way to adapt an offline detector to an online scenario is to declare the source as an LLM as soon as any single text is predicted such. However, this adaptation leads to a false positive rate that converges to
Proceedings of the 42nd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Online Detection of L.M-Generated Texts via Sequential Hypothesis Testing by Betting
as the number of texts becomes sufficiently laree-unless the detector is perfectly (100%) accurate on human-written texts. Formally, if the detector has accuracy ò on human- written texts, then under the naive reduction approach to online detection, the false positive rate over 7 rounds will be 1 - - which approaches as T' grows, even when is very close to 1.
of sequential hypothesis testing (Shafer, 2021; Ramdas et al., 2023; Shekhar & Ramdas, 2023). Specifically, we frame the problem of online LLM detection as a sequential hypothe- sis testing problem, where at each round t, a text from an unknown source is observed, and we aim to infer whether it is generated by an LLM. We also assume that a pool of examples of wrintenwritten texts is available, and our al- gorithm can sample a text from this pool of examples at any time t. Our method constructs null hypothesis Ho (to be elaborated soon), for which correctly rejecting the null hypothesis implies that the algorithm correctly identifies the source as an LLM under mild assumption. Furthermore, since it is desirable to quickly identify an LLM when is present and avoid erroneously declaring the source as an LLM, we also aim control the type-I error rate (false positive rate) while maximizing the power reduce type-ll error rate (false negative rate), and to establish upper bound on the expected total number of rounds to declare that the source is an LLM. We emphasize that our approach is non-marametric, and hence it does not need to assume that the underlying data of human or machine- -eaiinnngeerrteed texts follow a certain distribution (Balsubramani & Ramdas, 2015). It also avoids the need for assuming that the sample size is fixed or to specify it before the testing starts, and hence it is in contrast with some typical hypothesis testing methods that do not enjoy strong statistical guarantees in the anytime-valid fashion (Garson, 2012; Good, 2013; Tar- takovsky et al., 2014). The way to achieve these is based on recent developments in sequential hypothesis testing via betting (Shafer, 2021; Shekhar & Ramdas, 2023). The set- ting of online testing with anytime-vllid guarantees could be particularly useful when one seeks substantial savings in both data collection and time without compromising the reliability of their statistical testing. These desiderata might be elusive for approaches that based on collecting data in batch and classifying them offline to achieve.
Therefore, these offline detection methods fall short in the online scenarios, where one would like to design efficient algorithms with robust performance metrics, such as being valid level-a test, having asymptotic power 1, and enjoying a bound on the expected detection time for LLMs, which can be appealing for time-sensitive applications like moni- toring online platforms. For example, the American Federal Communications Commission in 2017 decided to repeal net neutrality rules according to the public opinions collected through an online platform (Selyukh, 2017; Weiss, 2019). However, it was ultimately discovered that the overwhelm- ing majority of the total 22 million comments that support rescinding the rules were machine-genee -eeeerrtee (Kao, 2017). In 2019, Weiss (2019) used GPT-2 to overwhelm a website for collecting public comments on a medical reform waiver within only four days, where machine-generated comments eventually made up 55.3% of all the comments (more pre- cisely, 1, 001 out of 1, 810 comments). As discussed by Fröhling& & Zubiaga (2021), a GPT-J model trained on a politics message board was then deployed on the same fo- rum. It generated posts that included objectionable content and accounted for about 10% of all activity during peak times (Kilcher, 2022). Furthermore, other online attacks mentioned by Fröhling& & Zubiaga (2021) may even manip- ulate public discourse (Ferrara et al., 2016), flood news with fake content (Belz, 2019), or fraud by impersonating others on the Internet or via e-mail (Solaiman et al., 2019) 9). How- ever, to the best of our knowledge, existing bot detection methods for social media (e.g., Davis et al. (2016); Varol et al. (2017); Pozzana& & Ferrara (2020); Ferrara (2023) and the references therein) might not be directly applicable to the online setting with strong statistical guarantees, and they often require training on extensive labeled datasets before- hand. This highlights the urgent need to develop algorithms with strong statistical guarantees that can quickly and effec- tively identify machine-generated texts in timely manner, which has been largely overlooked in the literature.
We evaluate the effectiveness of our method through com- prehensive experiments. The code and datasets available via this link: https: / //git thub. com/ canchen-cceeccco - /c/ online- 11m- leeee-mmmdeetetiinn detection.
2. Preliminaries
We begin by providing a recap of the background on sequen- tial hypothesis testing.
Our goal, therefore, is to tackle the problem of online detec- tion of L-M--generted texts. More precisely, building upon existing score functions from those "offline approaches", we aim to quickly determine whether the source of a sequence of texts observed in a streaming fashion is an LLM or a human, with concrete bound on the required samples to de- tect LLMs and without the need for tuning a threshold of the affinity score that is typically used to classify data in those offline approaches. Our algorithm leverages the techniques
Sequential Hypothesis Testing with Level-a and Asymp- totic Power One. Let us denote a forward filtration F = (Fi)ezo. where F, = oZ,,..., Ze) increasing sequence that accumulates all the information from the observations (Z, i 2 1} up to time point t. A process W == (W2)221 adapted to (F))zz is defined as p-martingale if it satisfies [r = W for all t > 1. Furthermore, W is P-supermartingale if
represents an
2

---

Online Detection of LLM-Generated Texts via Sequential Hypothesis Testing by Betting
LLM Detection via Online Optimization
(0) Observe
(1) Score
(2) Update
(3) Check
from an LLM
Source
Human Text Dalaset
Want: sypel orir S
Inpute
Yes
time budgetis T
Stmple RespOL1
W: P
Vos
2
Swlect a foom Funntionn
Update & vi CNS
Weath by:
No
Yes
1=7
2
1=1
<0>>
-W.. (13 (69)
No
No
1-ce1
X: IS from human
Figure 1. Overview of LLM Detection via Online Optimization. (0) (Observe) We sequentially observe text yt generated by an unknown source starting from time = and aim to determine whether these texts are produced by a human or an LLM. The detection process can be divided into three steps. (1) (Score) At each time text and Be are evaluated by selected score function Φ(.), where the sample æ drawn from a prepared dataset consisting
human-writter text examples. (2) (Update) The parameter 0, is updated via the Online Newton Step (ONS) increase the wealth Wi rapidly when y is LLM-generates text. A large value of W. serves as significant evidence and provides confidence to declare that the unknown souree an LLM. (3) (Check) Whether the wealth W, 2 1/a is checked. If this event happens, we declare the unknown source of y LLM. Otherwise, if the time budget I is not yet exhausted if we have unlimited time budget, we proceed to & + 1 and repeat the steps. When = T, if the condition Wr 2 2/a a holds, where Z is drawn from an uniform distribution in [0, 1]. our algorithm will also declare the source LLM.
[p[W,]FF-1] S Wi-1 for all & 2 1. In our algorithm de- sign, we will consider a martingale W and define the event {W, > 1/a) as rejecting the null hypothesis Ho, where > 0 is user-specified "significance level" parameter. We denote the stopping time 7 == inf(t 2 1 W, 2 ) accordingly.
(2023). We provide more details on these score functions in the experiments section and in the exposition of the literature in Appendix A.
Following related works on sequential hypothesis testing via online optimization (e.g., Shekhar & Ramdas (2023); Chugg et al. (2023)), we assume that each text y/ is i.i.d. from a distribution pV, and similarly, each human-written text I2 is i.i.d. from a distribution p². Denote the mean Un := Ep [x(z]] and by := Ep [o(y)] respectively. The task of hypothesis testing that consider can be formulated as
We further recall that a hypothesis test is a level-a test if sup P3t 2 W, 2 1/a) S a, or alternatively, if se (T < oc) S a. Furthermore, a test has asymp- totic power 1 33 if suppeh, P P 2 1, W, < 1/a)= S B, or if suppeR; (r oo) S B, where H1 represents the alternative hypothesis. A test with asymptotic power one (i.e., 3 =0) 0) means that when the alternative hypothesis Hi is true, the test will eventually rejects the null hypothesis Ho- As shown later, our algorithm that will be introduced shortly a provable sequential hypothesis testing method with level-a and asymptotic power one.
Ho (null hypothesis): Pr = Myy H1 (alternative hypothesis):
Pa # Py-
We note that when Ho is true, this is not equivalent to say- ing that the texts {gr)T= are distributions can share the same mean. However, under the additional assumption of the existence of good score function Φ(-) which produces scores for machine-genee generated texts with mean My different from that of numan-generated texts H4r, Hd is equivalent to the unknown source being hu- man. Therefore, under this additional assumption, when the unknown source p" is an LLM, then rejecting the null hypothesis Ho is equivalent to correctly identifying that the source is indeed LLM. In our experiments, we found that this assumption holds empirically for the score functions we adopt. That is, the empirical mean of o(v)) significantly differs from that of 6(x,) when each y, is generated by an LLM. Figure I illustrates the online detection process for LL.--geeeeteed texts.
as different
Problem Setup. We consider a scenario in which, at each round f, a text y, from an unknown source is observed, and additionally, a human-written text x, can be sampled from a dataset of human- The goal is to quickly and correctly determine whether the source that produces the sequence of texts { is an LLM human. We assume that a score function <(.): Text- + F. is available, which, given a text as input, outputs score. The score function Φ(-) that we consider in this work are those proposed for detecting LLM-generated texts in offline settings, e.g., Mitchell et al. (2023); Bao et al. (2023); Su et al. (2023); Bao et al. (2023); Yang et al.
text samples at our disposal.
3

---

Online Detection of LLM--Gnerrated Texts via Sequential Hypothesis Testing by Betting
Sequential Hypothesis Testing by Betting. Consider the scenario that online learner engages in multiple rounds of a game with an initial wealth Wo = 1. In each round of the game, the learner plays point l₁. Then, the learner receives a fortune after committing f,. which is -g -g W where w is the learner's ' wealth from the previous round t-1 I, and Sr can be thought of as "the coin outcome" at that the learner trying to "bet" on (Orabona & Päl, 2016). Consequently, the dynamic of the wealth of the learner evolves as:
Algorithm I Online Detection of LLMS via Online Opti-
mization and Betting
Require: score function <b() Text - IR.
1: Init: 01 - 0, ug + 1, wealth Wa < 1, step size Y
and significance level parameter E (0, 1).
2: # I'' is the time budget, which can be set to oc if there is no time construint.
3: for / = 1,2,.... T, do
4:
Observe a text yr from an unknown source and com- pute d(yt).
5: Sample o; from a dataset of and compute d(x,).
texts
W = W - 91- Wo MT= (1 - -s00).
6:
Set gi = 6(xx) ((v.).
To connect the learner's game with sequential hypothesis testing, one of the key techniques that will be used in the algorithm design and analysis is Ville's inequality (Ville, 1939), which states that if (W2) is nonnegative super- martingale, then one has P(3:= W. 2 1/a) S [B]W The idea is that if we can guarantee that the learner's wealth W. remains nonnegative from Wo = 1, then Ville's inequal- ity can be used to control the type-I error at level a simulta- neously at all time steps I. To see this, let 9t = ( Then, when P E Ho (i.c., the null hypothesis uax = fy holds), the wealth (Wo)e21 is P-supermartingale, because
7:
Update wealth W = W - (1- g(0)).
8:
if W, 2 1/a then
9:
Declare that the source producing the sequence of texts yt an LLM.
10:
end if
11:
Get a hint de+1 which satisfies d,+1 2 19++1/-
12:
Specify the decision space K141 as Ket1 :=
[- Zdezs' 1 2d+41
_ to ensure W++1 2 0.
13:
# Update 0.+1 E Ket1 via ONS on the loss function
(0) In(1 gi0).
14:
Compute 2t
dl.(Oe)
91
Ot = a1-1 1 27.
de
T-9
Ep [..[ = Ep [ (1 - 0egt)) Pt-1) = -E - (1 (1- (d(z)) - ((3))))) Ft-1] = W
15: 0.t1 max (min (B
1 z1 1
) 2de+1
7 ar 2de+1
16: end for
17: if the source has not been declared as an LLM then
Hence, if the learner's wealth W. can remain nonnegative given the initial wealth Wo = 1, we can apply Ville's in- equality to get a provable level-a test, since W; is a nonneg- ative supermartingale in this case. Another key technique randomized Ville's s inequality (Ramdas Manole, 2023) for a nonnegative supermartingale (W1)221, which states that P (3t < T W; 2 1/a WT 2 Z/a)= S a, where T is any F-stopping time and Z is randomly drawn from the uniform distribution in Jo, 1]. This inequality becomes par- ticularly handy when there is a time budget T in sequential hypothesis testing while maintaining a valid level-a test.
18: Sample Z ~ Unif(0, 1). declare the sequence of texts y/ is from an LLM if WT 2 Z/o.
19: end if
ET, (e(e)). while the second term is that of bench- mark. Therefore, if the learner's regret can be upper- bounded, say C, then the leamer's 's wealth is lower-bounder as WT 2 (IT2_ - 9,0.)) exp(-C). An online learning algorithm with a small regret bound can help increase the wealth quickly under H1. We refer the reader to Appendix for a rigorous argument, where we note that applying a no- regret algorithm to guarantee the learner's 's wealth is neat technique that is well-known in online learning, see e.g., Chapter 9 of Orabona (2019) for more details. Following existing works (Shekhar & Ramdas, 2023; Chugg et al., 2023), we will adopt Online Newton Steps (ONS) (Hazan et al., 2007) in our algorithm.
We now switch to discussing the control of the type-II error, which occurs when the wealth W, is not accumu- lated enough to reject Ho when Hi is true. Therefore, we need a mechanism to enable the online learner in the game quickly increase the wealth under H1. Related works of sequential hypothesis testing by betting (Shekhar & Ram- das, 2023; Chugg et al., 2023) propose using a no-regret learning algorithm achieve this. Specifically, a no-regret learner aims to obtain a sublinear regret, which is detined as (f. : DT= (())) E (((),,, where 6_ is benchmark. In our case, we will consider the loss function at t to be (,()) := - Inn11 g) The high-level idea is based on the observation that the first term in the regret definition is the log of the learner's 's wealth, mod- ulo a minus sign, i.e. In(Tr) = Σtn In(1 g.f.) =
3. Our Algorithm
We have covered most of the underlying algorithmic design principles of our online method for detecting LLMs, and we are now ready to introduce our algorithm, which is shown in Algorithm 1. Compared to existing works on sequential hy- pothesis testing via betting (e-g., Shekhar & Ramdas (2023);
4

---

Online Detection of Lenertre Texts via Sequential Hypothesis Testing by Betting
Chugg et al. (2023)), which assume knowledge of bound on the magnitude of the "coin outcome" gt in the learner's wealth dynamic (1) for all time steps before the testing be- gins (e.g., assuming for all t, o(zz) and ((y)) are always in [o, I), we relax this assumption, which is motivated by the concern that a good estimate of the bound for the coin outcome might not be available before the sequential test- ing starts. Specifically, we consider the scenario where an upper bound on |gt+1| at round + 1, which is denoted by de+1, is available before updating 0.+1 at each round t. Our algorithm then plays point in the decision space K++11 that guarantees the learner's 's wealth remains a non non-negative supermartingale (Step 12 in Algorithm 1). We note that if the bound of the output of the underlying score function (()) known a priori, this scenario holds naturally. Otherwise, we can estimate an upper bound for |gr] for all t based on the first few time steps and execute the algorithm thereafter. One approach is to set the estimate as conservatively large constant, e.g., twice the maximum value observed in the first few time steps. We observe that this estimate works for our algorithm with most of the score functions o--) that we consider in the experiments. On the other hand, we note that tighter bound de will lead faster time to reject Ho when the unknown source is an LLM, as indicated by the following propositions, where we note that the analysis of the test power requires the assumption that the data are i.i.d. Proposition 3.1. Algorithm/ is a level-o sequential test with asymptotic power one. Furthermore, if yt is generated by an L.M,, the expected time T to declare the unknown source as an LLM is bounded by
as Ho 1ize - Myl versus H1 : lixe Myl > e. The hypothesis can be equivalently expressed in terms of two hypotheses,
500000 H₂' Mz My - e 0 vs. H1 Me - My > 0,
HD My - pz - c S 0 vs. HP My - P2= - > 0.
Consequently. the dynamic of the wealth evolves as W/A. = W
(1 Oc(gt - -)) and W = W (1 0c(-9) c)) respectively, where 9r = = - o())). We note that both 9t E and - are within the interval [-d e, de - c). The composite hypothesis is motivated by the fact that, in practice, even if both sequences of texts 2y and yr are human-writtenn, they may have been written by different individuals. Therefore, it might be more rea- sonable allow for a small difference e > 0 in their means when defining the null hypothesis Ho.
Proposition 3.2. Algorithm 3 in the appendix is level-o sequential test with asymptotic power one, where the wealth W.A for HÁ (H and WB for Hg (Hf)) are calculated through level-a/2 tests. Furthermore, if is generated by an L.M,, the expected time T to declare the unknown source as an LLM is bounded by
[E[] =0 (d, + +)²
(A - c0² a(A -3)) + (d, + +c)
In ( d.
(A - c)4
where A == 11 - Myl d. == max de with de 2 19t1 7
121
1 min Ge upper bound of the gradient (O), and is nonnegative constant satisfying S for all
13 with G::= == maxaek, (7),(0) denoting the
d2
d. a
1
E|7| = 0
In
+
A
We note that when the alternative hypothesis H1 is true, we have A > e. Proposition 3.2 indicates that even if there is difference in mean scores between texts written by different humans, the probability that the source is incorrectly declared by Algorithm 3 as an LLM can be controlled below a. Furthermore, the bound on E[r] implics that smaller c and larger A will result in a shorter time to reject Ho.
Remark 2.
where A = |pz- - Hyl d. = max de with de 2 19eb. and 121
y= mini & 1} with G, == maX [Q) denoting the upper bound of the gradient (l)(
Remark 1. Under the additional assumption of the existence of good score function <b-) that can generate scores with different means for human-written texts and I.LM-geeertedd ones, Proposition 3.1 implies that when the unknown source is declared by Algorithm 1 as an LLM, the probability of this declaration being false will be bounded by a. Additionally, if the unknown source is indeed an LLM, then our algorithm can guarantee that it will eventually detect the LLM, since it has asymptotic power one. Moreover, Proposition 3.1 also provides a non-asymptotic result for bounding the expected time to reject the null hypothesis Hp, which is also the expected time to declare that the unknown source is an L.M. The bound indicates that a larger difference of the means A can lead to shorter time to reject the null Ho.
4. Experiments
Score Functions. We use 10 score functions in total from the related works for the experiments. As mentioned earlier, score function takes a text as an input and outputs a score. For example, one of the configurations of our algorithm that we try uses a score function called Likelihood, which is based on the average of the logarithmic probabilities of each token conditioned on its preceding tokens (Solaiman et al., 2019; Hashimoto et al., 2019). More precisely, for text æ which consists of n tokens, this score function can be for- mulated as d(x) = 1 Σn= log po (2 (11--1), where æj denotes the j-th token of the text x. Z1:5-1 means the first
(Composite Hypotheses.) As Chugg et al. (2023), we also consider the composite hypothesis, which can be formulated
a
5

---

Online Detection of LLM-Generated Texts via Sequential Hypothesis Testing by Betting
j 1 tokens, and pe represents the probability computed by language model used for scoring. The score functions that we considered in the experiments are: 1. DetectGPT: perturbation discrepancy (Mitchell et al., 2023). 2. Fast- DetectGPT: conditional probability curvature (Bao et al., 2023). 3.LRR: likelihood log-rank ratio (Su et al., 2023). normalized perturbed log-rank (Su et al., 2023).
Appendix H, which also demonstrate the effectiveness of our method.
Baselines, Our method are compared with two baselines, which adapt the fixed-time permutation test (Good, 2013) to the scenario of the sequential hypothesis testing. Specif- ically, the first baseline conducts a permutation test after collecting every Is samples. If the result does not reject Ho. then it will wait and collect another k samples to conduct an- other permutation test on this new batch of k samples. This process is repeated until Ho is rejected or the time / runs out (i.c., when t = T). The significance level parameter of the permutation test is set to be the same constant a for each batch, which does not maintain a valid level-a test overall. The second baseline is similar to the first one except that the significance level parameter for the i-th batch set to be a a2/2,, with i starting from 1, which aims to ensure that the cumulative type-l error is bounded by a via the union bound. The detailed process of the baselines is described in Appendix G.
NPR:
5. DNA-GPT: WScore (Yang et al., 2023). 6. Likelihood: mean log probabilities (Solaiman et al., 2019; Hashimoto et al., 2019; Gehrmann et al., 2019). 7. LogRank: averaged log-rank in descending order by probabilities (Gehrmann et al., 2019). 8. Entropy: mean token entropy of the pre- dictive distribution (Gehrmann et al., 2019; Solaiman et al., 2019; Ippolito et al., 2019). 9. RoBERTa-base: a pre-trained classifier (Liu et al., 2019). 10. RoBERTa-large: a larger pre-trained classifier with more layers and parameters (Liu et al., 2019). The first eight score functions calculate scores based on certain statistical properties of texts, with each text's score computed via language model. The last two score functions compute scores by using some pre-trained classifiers. For the reader's s convenience, more details about the implementation of the score functions (6-) are provided in Appendix B.
Parameters of Our Algorithm. All the experiments in the following consider the setting of the composite hypothesis. For the step size 7, we simply follow the related works (Cutkosky & Orabona, 2018; Chugg et al., 2023; Shekhar & Ramdas, 2023) and let 1/y = 2/(2 - 3). We consider two scenarios of sequential hypothesis testing in the experi- ments. The first scenario (oracle) assumes that one has prior knowledge of de (or d.) and e, and the performance of our al- gorithm in this case could be considered as an ideal outcome that it can achieve. For simulating this ideal scenario in the experiments, we let e be the absolute difference between the mean scores of XSum texts and 2024 Olympic news, which are datasets of human- human-written written texts. The second scenario considers that we do not have such knowledge a priori, and hence we have to estimate de (or d.) and specify the value of e using the samples collected in the first few times steps, and then the hypothesis testing is started thereafter. In our experiments, we use the first 10 samples from each sequence of x, and y and set de to be a constant, which is twice the value of max, (xx,) - Φ(yx)). For estimating we ob- tain scores for 20 texts sampled from the XSum dataset and randomly divide them into two groups, and set to twice the average absolute difference between the empirical means of these two groups across 1000 random shuffles.
LLMs and Datasets. Our experiments focus on the black- box setting (Bao et al., 2023), which means that if æ is generated by a model qu. i.e., x ~ qs. a different model pa will then be used to evaluate the metrics such as the log- probability log (z() when calculating M(z). The models qs and po are respectively called the "source model" and "scor- ing model" for clarity. The black-box setting is a relevant scenario in practice because the source model used for gen- erating the texts to be inferred is likely unknown in practice, which makes it clusive to use the same model to compute the scores. We construct a dataset that contains some real news and fake ones generated by LLMs for 2024 Olympics. Specifically, we collect 500 news about Paris 2024 Olympic Games from its official website (Olympics, 2024) and then use three source models, Gemini- 1. 1.5-hlash, Gemini-1.5. Pro (Google Cloud, 2024a), and PaLM 2 (Google Cloud, 2024b; Chowdhery et al., 2023) to generate an equal num- ber of fake news based on the first 30 tokens of each real one respectively. Two scoring models for computing the text scores (6-) are considered, which are GPT-Neo-2.77B (Neo-2.7) (Black et al., 2021) and Gemma-2B (Google, 2024). The perturbation model that is required for the score function DetectGPT and NPR is T5-3B (Raffel et al., 2020). For Fast-DetectGPT, the sampling model is GPT-J- 6B (Wang& & Komatsuzaki, 2021) when scored with Neo-2.7, and Gemma-22 2B when the scoring model is Gemma-2B. We sample miintewritten text 24 from a pool of 500 news ar- ticles from the XSum dataset (Narayan et al., 2018). We emphasize that we also consider existing datasets from Bao et al. (2023) for the experiments. Details can be found in
Experimental Results. The experiments evaluate the per- formance of our method and baselines under both Ho and H₁. As there is inherent randomness from the observed samples of the texts in the online setting, we repeat 1000 runs and report the average results over these 1000 runs. Specifically, we report the false positive rate (FPR) under Hy, which is the number of times the source of ye is in- correctly declared as an LLM when it is actually human, divided by the total of 1000 runs We also report the average
6

---

Online Detection of L.M-Generaaed Texts via Sequential Hypothesis Testing by Betting
time to reject the null under H1 (denoted as Rejection Time
False Positive Rate (FRR)
0.05
lals Positive Rate (EPe)
0.100
r), which is the time our algorithm takes to reject Ho and correctly identify the source when yt is indeed generated by an LLM. More precisely, the rejection time T is the average time at which either W, or W exceeds 2/ ax before T; oth- erwise, T is set to T = 500, regardless of whether it rejects Hg at T. since the the time budget runs out. The parameter value de in Scenario 1 (oracle) is shown in Table 2, and the value for can be found in Table 1 in the appendix. For the estimated and de of each sequential testing in Scenario 2, they are displayed in Table 5 and Table 6 respectively in the appendix. Our method and the baselines require specify- ing the significance level parameter a. In our experiments, we try 20 evenly spaced values of the significance level
0.04
0.08
0.03
0.02
0.04
0.01
0.02
0.00
0.00
100 200 300 400 s00 Rejection Time (t)
0.00 0.02 0.04 0.66 0.08 0.10
Significance Level (a]
Fast-Detect
NPR LRR
LogRank
Entropy
RoB-base
DetectGPT
RaRglarge
(a) Averaged results with text sampled from XSum and y from 2024 Olympic news or generated news, across three
source models. The scoring model Neo-2.7.
False Positive Rate (TPR)
0.05
False Positive Rate (HPR)
00.10
0.04
0.08
parameter a that ranges from 0.005 to 0.1 and report the performance of each one.
0.03
0.06
0.02
0.04
0.01
0.02
Figure 2 shows the performance of our algorithm with differ- ent score functions under Scenario 1 (oracle). Our algorithm consistently controls FPRs below the significance levels a and correctly declare the unknown source as an LLM before T 500 for all score functions. This includes using the Neo-2.7 or Gemma-2B scoring models to implement eight of these score functions that require a language model. On the plots, each marker represents the average results over 1000 runs of our algorithm with specific score function <(-) under different values of the parameter a. The subfig- ures on the left in Figure 2a and 2b show False Positive Rate (under Ho) versus Rejection Time (under H1;; therefore, a curve that is closer to the bottom- corner more pre- ferred. From the plot, we can see that the configurations of our algorithm with the score function being Ge., DetectGPT, or Likelihood have the most competitive per- formance. When the unknown source is an LLM, they can detect it at time around f = 100 on average, and the obser- vation is consistent under different language models used for the scoring. The subfigures on the right in Figure 2 show that the FPR is consistently bounded by the chosen value of the significance level parameter a.
0.00
0.00
100
200 400 500
0.00 0.02 0.04 0.06 0.08 0.10
Rejection Time (t)
Significance Level (o]l
Fast-Detect
* NPR
LogRank Likelinood
Entropy DNA GPT
+- RoB-base
DetertGPT
LRR
RoBl large
(b) Averaged results with text x, sampled from XSum and y/+ from 2024 Olympic news or machine- generated news, across three souree models. The scoring model is Gemma-2B.
machine-generated
Figure 2. Averaged results of Scenario (oracle), which shows the average of the rejection time under H1 (i.e., the average time to detect LLMs) and the false positive rate under Hu for 10 dif- ferent score functions and 20 different values of the significance level parameter a. Here, three source models (Gemini- F.5-aahh,, and PaLM 2) used to generate an equal number of the maahine-ggeneatte texts, and two different scoring models (Neo-2.7 and Gemma-2B) are used for computing the function value of the score functions. The left subfigure in each panel (a) and (b) shows the average time to correctly declare an LLM versus the average false positive rates over 1000 runs for each a. Thus, plots closer to the bottom-left corner are better, as they indicate correct detection of an LLM with shorter rejection time and a lower FPR. In the right subfigure of cach panel, the black dashed line along with the shaded area illustrates the desired FPRs. Our algorithm under various configurations consistently has an FPR smaller than the value of the significance level parameter a.
Figure 3 shows the empirical results of our algorithm under Scenario 2, where it has to use the first few samples to specify de and e before starting the algorithm. Under this scenario, our algorithm equipped with most of the score functions still performs effectively. We observe that our algorithm with 1) Fast-DetecIGPT as the score function e(.) and Neo-2.7 as the language model for computing the score, and with 2) Likelihood as the score function <6+) and Gemma-2B for computing the value of (6-) have the best performance under this scenario. Compared to the first case where the oracle of de and e is available and exploited, these two configurations only result in a slight degradation of the performance under Scenario and we note that our algorithm can only start updating after the first 10 time steps under this scenario. We observe that the bound
of d, that we estimated using the samples collected from first 10 time steps is significantly larger than the tightest bound of de in most of the runs where we refer the reader to Table 2, 6 in the appendix for details, which explains why most of the configurations under Scenario 2 need longer time to detect LLMs, as predicted by our propositions. We also observe that the configurations with two supervised classifiers RRoBERTa-based and RoBERTa-large) and the combinations of a couple of score functions and the scoring model Gemma-2B do not strictly control FPRs across all significance levels. This is because the estimated de for these score functions is not large enough to ensure that the
7

---

Online Detection of L.MeGeeeratedd Texts via Sequential Hypothesis Testing by Betting
False Posibive Rate (FPR)
0.10-
False Positive Rate (ERR
0.10
test and efficiently detect LLMs.
0.08-
0.08
Comparisons with Baselines. In this part, use the score function of Fast-DetectGPT and scoring model Neo-2.7 to get text scores, and then compare the performance of our method with two baselines that adapt the fixed-time permutation test to the sequential hypothesis setting. Batch sizes E {25, 50, 100, 250, 500} are considered for the baselines. We set the estimated and de values the same in Scenario 2. The baselines are also implemented in manner to conduct the composite hypothesis test.
0.06
0.04
0.04
0.02-
0.02
0.00
0.00
0
100
200 300 400 sōo
0.02 0.04 0.06 0.08 0.10 0.00
Rejection Time
Significance Level
Fast-Detect
LogRank
Entropy
DetactGPT
RoB-large
(a) Averaged results with text 2, sampled from XSum and yr machine-generated news, across three source models. The scoring model is Neo-2.7.
from 2024 Olympic
False Postiive Rate (FPR)
0.8
Felie Positive Rate (FPR)
0.8
Fels Rositive Rate (199)
1.0
Rooibiue (198)
1.00
0.6
0.6-
0.6
0.8-
0.4-
0.4-
0.4
0.4
0.2
0.2
0.2
0.2
0.0
0.0
0.0
100
400 Rejection Time (T)
200
300
500
0.00
0.06 0.08 0.10 Significance Level (o)
0.02
0.04
0
100 200
300 400 s0o
0.00 0.02 0.04 0.06 0.08 0.10
Rejection Time it)
Significance Level (a))
Fast DetectGPT
k-25 k-50
k-100 k-250
500
Fast-Detect
NPR LRR
LogRank
Entropy
RoB-base RoB-large
DetectGPT
(a) Comparison between our method and the baseline that sets the value of the significance level parameter to be the same constant o: for every batch.
(b) Averaged results with text iy sampled from XSum and yı from 2024 Olympic news machine-generater news, across three source models. The scoring model is Gemma-2B.
Felse Rositive Rate (FPR)
1.0
Rosibive (198)
1.0
0.8-
0.6-
0.6-
Figure 3. Averaged results of Secnario 2, where our algorithm has to use the first few samples to specify de and e before starting the algorithm. The plots are about the average of the rejection time under Hi (i.e.. the average time to detect LLMs) and the false positive rate under He for 10 different score functions and 20 different values of the significance level parameter a when using two different scoring models, Neo-2.7 (a) and Gemma-2B (b).
0.4
0.4
0.2
0.2-
100
300 400 Rejection Time (t)
200
500
0.00
0.07 0.04 0.06 0.08 0.10 Significance Level (o))
Fast DetectGP1
k-25 k-50
k-100 k-250
k-000
(b) Comparison between our method and the baseline that sets the value of the significance level parameter for the i-th batch test to be a/2'.
wealth W, remains nonnegative at all time points L. That is, observed 2d,- < 10((z)) (vy) | for some t in the experiments, and hence the wealth W, is no longer a non- negative supermartingale, which prevents the application of Ville's s inequality to guarantee a level-a test. Nevertheless, our algorithm, when using most score functions that utilize the scoring model Neo-2.7, can still effectively control type- I error and detect LLMs around 300.
Figure 4. Comparison of the average results under Scenario 2. where one has to use the first 10 samples to specify e and/or de before starting the algorithm. Human- wwitttn text are sampled from XSum dataset, while ze from 2024 Olympic news (under Ho) or machine- generated news (under H1). Fake news are gener- ated by three source models: and PaI.M 2. We report the case when the score function is Fast- DetectGPT and the scoring model is Nco-2.7 for our algorithm.
Gemini- 1.5-Pro
In Appendix H, we provide more experimental results, in- cluding those using existing datasets from Bao et al. (2023) for simulating the sequential testing, where our algorithm on these datasets also performs effectively. Moreover, we found that the rejection time is influenced by the relative magnitude of A e and d; e, as predicted by our propo- sitions, and the details are provided in Appendix G. From the experimental results, when the knowledge of de and is not available beforehand, as long the estimated d, and guarantee a nonnegative supermartingale, and the estimated e lies between the actual mean score difference of human- written texts and that of human-written vs --M-enerrtted texts, our algorithm can maintain sequential valid level-a
We observe a significant difference between the scores of XSum texts and machine-genen generated news, which causes the baselines of the permutation test to reject the null hypothesis most of the time immediately after receiving the first batch. This in turn results in the nearly vertical lines observed in the left subtigure of Figure 4a and Figure 4b, where the averaged rejection time across 1000 repeated tests closely approximates the batch size k for each of the 20 significance levels a. On the other hand, we observe that when Ho is true, the baselines might not be a valid a-test, even with a corrected significance level. This arises from the increased variability of text scores introduced by smaller batch sizes,
8

---

Online Detection of L.M-Geeerated Texts via Sequential Hypothesis Testing by Betting
which results in observed absolute differences in means that may exceed the value under Ho- Our method can quickly detect an LLM while keeping the false positive rates (FPRs) below the specified significance levels, which is a delicate balance that can be difficult for the baselines to achieve. Without prior knowledge of the E value, the baselines of permutation tests may fail to control the type-I error with small batch sizes and cannot quickly reject the null hypothesis while ensuring that FPRs remain below a, unlike our method.
betting framework presents a promising future direction for improving online detection of
texts.
Acknowledgements
The authors gratefully acknowledge the support provided by the Gemma Academic Program and Google Cloud Credits.
Impact Statement
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
Extensions. We further consider (i) an experiment where the sequential texts are produced by various LLMS instead of a single LLM, and (ii) a scenario that the unknown source publishes both human-written and ----eerrrtted texts, where the null hypothesis becomes that all texts from the unknown source are human human-written, and the alternative hy- pothesis posits that not all texts are human-written We refer the reader to Appendix for the encouraging results.
References
Abdelnabi, S. and Fritz, M. Adversarial watermarking trans- former: Towards tracing text provenance with data hiding. In 2021 IEEE Symposium on Security and Privacy (SP), pp. 121-140. IEEE, 2021.
5. Conclusion
In this paper, we demonstrate that our algorithm, which builds the score functions of offline detectors, can rapidly determine whether stream of text is generated by an LLM and provides strong statistical guarantees. Specifically, it controls the type-I error rate below the significance level, en- sures that the source of ---eenerrteed texts can eventually be identified, and guarantees an upper bound on the expected time to correctly declare the unknown source as an LLM un- der a mild assumption. Although the choice of detector can influence the algorithm's 's performance and some parameters related to text scores need to be predefined before receiv- ing the text, our experimental results show that most of the existing detectors we use provide effective score functions, and our method performs well in most cases when using estimated values of parameters based on text scores from the first few time steps. To further enhance its efficacy, may be worthwhile to design score functions tailored to the sequen- tial setting, improve parameter estimations with scores from more time steps, and study the trade-offs between delaying the start of testing and obtaining more reliable estimates. Moreover, our algorithm could potentially be used as an ef- fective tool to promptly identify and mitigate the misuse of LLMs for text generation, such as monitoring social media accounts that disseminate harmful information generated by LLMs, rapidly detecting sources of LLM-generate fake news on public websites, and identifying users who post machine-generated comments to manipulate public opinion. Exploring these applications could be promising direction.
Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al. Gpt-4 technical report. arXiv preprint
arXiv:2303.0877,, 2023.
Adelani, I., Mai, H., Fang, F., Nguyen, H. H., Yamag- ishi, J., and Echizen, I. Generating sentiment-preserving fake online reviews using neural language models and their human-and machine-based detection. In Advanced information networking and applications: Proceedings of the 34th international conference on advanced infor- mation networking and applications (AINA-2020), pp. 1341-1354 Springer, 2020.
Bakhtin, A., Gross, S., Ott, M., Deng, Y., Ranzato, M., and Szlam, A. Real or fake? learning to discriminate machine from human generated text. arXiv preprint frxiv:1906.03311, 2019.
Balsubramani, A. and Ramdas, A. Sequential nonparametric testing with the law of the iterated logarithm. arXiv preprint arXiv: 1eee:516600000 034600 2015.
Bao, G., Zhao, Y., Teng, Z., Yang, L., and Zhang, Y. Fast-detectgpt: Efficient zero-shot detection of machine- generated text via conditional probability curvature. arXiv preprint arX0v:2310.05130 2023.
Bar, Y., Shaer, S., and Romano, Y. Protected test-time adap- tation via online entropy matching: A betting approach. NeurIPS, 2024.
Finally, we note that a recent work of Chen & Wang (2025) introduces optimistic interior-point methods that enable up- dates over the full interior of the decision space. Incorpo- rating such advanced betting strategies into the besting-by-
Belz, A. Fully automatic journalism: we need to talk about nonfake news generation. In Conference for truth and trust online, 2019.

---

Online Detection of LLM-Generated Texts via Sequential Hypothesis Testing by Betting
Black, S., Gao, L., Wang. P., Leahy, C., and
Fagni, T., Falchi, F., Gambini, M., Martella, A., and Tesconi, M. Tweepfake: About detecting deepfake tweets. Plos one, 16(5)ee0251415, 2021.
Biderman, S. Gpt-neo: Large scale autore-
gressive language modeling with mesh-tensorflow.
2021. URL https: https///aap..emmnccccccoooo.e.
Fan, A., Lewis, M., and Dauphin, Y. Hierarchical neural story generation. arXiv preprint arXiv: 000000443,, 2018.
org/Co-pusTD:2257555
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg. J., Bosse- lut, A., Brunskill, E., et al. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.0723.
Ferrara, E. Social bot detection in the age of chatgpt: Chal- lenges and opportunities. First Monday. 2023.
Ferrara, E.,, Varol, O., Davis, C.,, Menczer, F., and Flammini, A. The rise of social bots. Communications of the ACM, 5977)966-104, 2016.
2021.
Brown, T. B. Language models are few-shot learners. 2020.
Fischer, L., and Ramdas, A. Multiple testing with anytime-valid monte-carlo p-values. arXiv preprint urXiv:2404.15580 2024.
Chen, C. and Shu, K. Can LLM-generated misinformation be detected? arXiv preprint 2an.233999.878 2023.
Chen, C. and Wang. J.-K. Optimistic interior point meth- ods for sequential hypothesis testing by betting. arXiv preprint wwxiv:2502.07772 2025.
Fröhling, and Zubiaga, A. Feature-based detection of automated language models: tackling gpt-2, gpt-3 and grover. PeerJ Computer Science, 7:e443, 2021.
Cho. B., Gan, K., and Kallus, N. Peeking with peak: Se- quential, nonparametric composite hypothesis tests for means of multiple data streams. ICML, 2024.
Garson, G. D. Testing statistical assumptions, 2012.
Gehrmann, S., Strobelt, H., and Rush, A. M. Gltr: Statisti- cal detection and visualization of generated text. arXiv preprint arXiv:1906.080 04044, 2019.
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(240):1-113, 2023.
Good, P. Permutation tests: a practical guide resam- pling methods for testing hypotheses. Springer Science & Business Media, 2013.
Christ, M., Gunn, S., and Zamir, O. Undetectable water- marks for language models. In The Thirty Seventh Annual Conference on Learning Theory, pp. 1125-1139, PMLR, 2024.
Google. Gemma-28 model on hugging face, 2024. URL htttp::/hhuginggaacee co/ googgl google/ gemma-2b.
Google Cloud. Vertex ai: Machine learning model manage- ment, 2024a. URL https: hccccoloooooooooooo com/ vertex- 2h1=zh
Chugg, B., Cortes-Gomez, S., Wilder, B., and Ramdas, A. Auditing faimess by betting. Advances in Neural Information Processing Systems, 36:6070-6091, 2023.
Google Cloud. Reference for text models in ver- tex ai, 2024b. URL https://cloud. google. com/vert http:///errerrra//ggeeeaaeaeaaaa model poodel--rfferenneettxxt2211----2
Cutkosky, A. and Orabona, F. Black-box reductions for parameter-free online learning in banach spaces. In Con- ference On Learning Theory, pp. 1493-1529. PMLR, 2018.
Grünwald, P., Henzi, A., and Lardy, T. Anytime-valid tests conditional independence under model-x. Journal of the American Statistical Association, 119(546):1554 1565, 2024.
Dai, J.. Gradu, P., Raji, D., and Recht, From individ- ual experience collective evidence: A reporting- based framework for identifying systemic harms. arXiv preprint arXiv:2502.08166, 2025.
Hans, A., Schwarzschild, A., Cherepanova, V., Kazemi, H., Saha, A., Goldblum, M., Geiping, J., and Gold- stein, T. Spotting Ilms with binoculars: Zero-shot detection of machine -eeerrate text. arXiv preprint arXi::2401.12070 2024.
Davis, C. A., Varol, O., Ferrara, E., Flammini, A., and Menezer, F. Botornot: A system to evaluate social bots. In Proceedings of the 25th international conference com- panion on world wide web, pp. 273 273-274, 2016.
Devlin, J. Bert: Pre-training of deep bidirectional trans-
Hashimoto, B., Zhang, H., and Liang, Unifying human and statistical evaluation for natural language generation. arXiv preprint 1904.09900402 2019.
formers for language understanding. arXiv preprint arXiv: 1810.888044 2018.
10

---

Online Detection of LLM-Geeerateed Texts via Sequential Hypothesis Testing by Betting
Hazan, E., Agarwal, A., and Kale, S. Logarithmic regret al- gorithms for online convex optimization. Machine Learn- ing, 69(2):169-19)) 2007.
Lee, H., Tack, J., and Shin, J. Remodetect: Reward mod- els recognize aligned Ilm's generations. arXiv preprint 1xXiv:2405.7738: 2024.
Hazan, E. et al. Introduction to online convex optimization. Foundations and Trends0 in Optimization, 2(3-4):157- 325, 2016.
Lee, J., Le, T., Chen, J., and Lee, D. Do language models plagiarize? In Proceedings of the ACM Web Conference 2023, pp. 3637-3647, 2023.
Ippolito, D., Duckworth, D., Callison-Buree C.,, and Eck, D. Automatic detection of generated text is easiest when humans are fooled. arXiv preprint arXiv:1911.00650 2019.
Lin, S., Hilton, J., and Evans, O. Truthfulqa: Measuring how models mimic human falsehoods. arXiv preprint arXiv:2109.07958 2021.
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M.. Chen, D., Levy, O., Lewis, M., Zettlemoyer, L.,, and Stoyanov, V. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907. 16692 2019.
Jalil, Z. and Mirza, A. M. A review of digital watermarking techniques for text documents. In 2009 International Conference Information and Multimedia Technology, pp. 230-234 IEEE, 2009.
Mao, C., Vondrick, C., Wang, H., and Yang. J. Raidar: generative ai detection via rewriting. arXiv preprint arXiv:2401.12970, 2024.
Jawahar, G., Abdul-Mageed, M., and Lakshmanan, L. V. Automatic detection of machine generated text: critical survey. arXiv preprint arXiv:2011.01311 2020.
Mitchell, E., Lee, Y., Khazatsky, A., Manning, D., and Finn, C. Detectgpt: Zero-shot machine- machine-generated text de- tection using probability curvature. In International Con- ference Machine Learning, pp. 11950-24962.. PMLR, 2023.
Jin, Q., Dhingra, B., Liu, Z., Cohen, W. W., and Lu, X. Pubmedqa: A dataset for biomedical research question answering. arXiv preprint 1.X.v:1909006144 2019.
Kamaruddin, N. S., Kamsin, A., Por, L.. Y., and Rahman, H. A review of text watermarking: theory, methods, and applications. IEEE Access, 6:8011-8028 2018.
Narayan, S., Cohen, S. B., and Lapata, M. Don't give me the details, just the summary! topic-aware convolutional neu- ral networks for extreme summarization. arXiv preprint arXiv: arXiv:1808.0874 /80.08875, 2018
Kao, More than a million pro-repeal net neutrality com- ments were likely faked. Hacker Noon, Nov 2017.
Kasneci, E., SeBler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. Chatgpt for good? on opportunities and challenges of large language models for education. Learning and individual differences, 103: 102274, 2023.
Olympics. Olympic games news, 2024. URL https: / //olympics cwt.i//ccccco////////mm//////
OpenAl. Chatgpt. https: h/tp:://hhhttte com// De- cember 2022.
OpenAL. AI Text Classifier. https: //btta.b.....eeeeeeeeeeeee...e.e openai. com/ wiitext--llassiiieeeeeetetete-ee 2023. Accessed: 2023-08-30.
Kelly, J. L. A new interpretation of information rate. the bell system technical journal, 35(4):917-926, 1956.
Kilcher, Y. This the worst ai ever, June 2022. Linkedln post.
Orabona, F. modern introduction to online learning. arXiv preprint
2019.
Kirchenbauer, J., Geiping. J., Wen, Y., Katz, J., Miers, I., and Goldstein, T. A watermark for large language models. In International Conference on Machine Learning, pp. 17061-17084, PMLR, 2023.
Orabona, F. and Jun, K.-S. Tight concentrations and con- fidence sequences from the regret of universal portfolio. IEEE Transactions Information Theory, 2023.
Orabona, F. and Pâl, D. Coin betting and parameter-free free line learning. Advances in Neural Information Processing Systems, 29, 2016.
Kobak, D., Gonzalez-Märquez, R., Horvát, E.-Á., and Lause, J. Delving into chatgpt usage in academic writing through excess vocabulary. arXiv preprint arXiv:2406.07016. 2024.
Pandeva, T., Forré, P., Ramdas, A., and Shekhar, S. Deep anytime-valid hypothesis testing. In International Confer- ence on Artificial Intelligence and Statistics, pp. 222-630. PMLR, 2024.
Lavergne, T., Urvoy, T., and Yvon, F. Detecting fake content with relative entropy scoring. Pan, 8(27-31):4, 2008

---

Online Detection of LLM-Generatedd Texts via Sequential Hypothesis Testing by Betting
Podkopaev, A. and Ramdas, A. Sequential predictive two- sample and independence testing. Advances in neural information processing systems, 36:53275-53307 2023.
Susnjak, T. and McIntosh, T. R. Chatgpt: The end of online integrity? Education Sciences, 14(6):656, 2024.
Tartakovsky, A., Nikiforov, I., and Basseville, M. Sequential analysis: Hypothesis lesting and changepoint detection. CRC press, 2014.
Podkopaev, A., Blöbaum, P., Kasiviswanathan, S., and Ram- das, A. Sequential kernelized independence testing. In In- terntiional Conference Machine Learning, pp. 27957- 27993. PMLR, 2023.
Teneggi, and Sulam, bet you did not mean that: Testing semantic importance via betting. NeurIPS, 2024.
Podkopaev, A., Xu. D., and Lee, K.-C. Adaptive conformal inference by betting. ICML, 2024.
Tian, Gptzero: An text detector, 2023.
Pozzana, I. and Ferrara, F.. Measuring bot and human be- havioral dynamics. Frontiers in Physics, 8:125, 2020.
Uchendu, A., Le, T., Shu, K., and Lee, D. Authorship attribution for neural text generation. In Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP), pp. 8384-8395, 2020.
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, J. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21 (140):1-67, 2020.
Varol, O., Ferrara, E., Davis, C., Menczer, F., and Flammini, A. Online human-bo interactions: Detection, estimation, and characterization. In Proceedings of the international AAAI conference web and social media, volume 11, pp. 280,289, 2017.
Ramdas, A. and Manole, T. Randomized and exchangeable improvements of markov's, chebyshev's and chernoff's inequalities. arXiv preprint arXiv:2304.
2023.
Ville, J. Etude critique de la notion de collectif. Gauthier- Villars Paris, 1939.
Ramdas, A., Grünwald, P., Vovk, V., and Shafer, G. Game- theoretic statisties and safe anytime-valid inference. Sta- tistical Science, 3864)6576-601, 2023.
Vovk, and Wang, E-values: Calibration, combination and applications. The Annals of Statistics, 49(3):1736 1754, 2021.
Selyukh, A. Fec repeals 'net neutrality' ruless for internet providers. NPR (accessed 13 October 2020), 2017.
Shaer, S., Maman, G., and Romano, Y. Model-x sequential testing for conditional independence via testing by betting. In International Conference on Artificial Intelligence and Statistics, pp. 2054 2086. PMLR, 2023.
Wang, B. and Komatsuzaki, A. Gpt-j-6b: A billion param- eter autoregressive language model, 2021.
Wauddy-Smithh and Ramdas, Estimating means of bounded random variables by betting. Journal of the Royal Statistical Society Series B: Statistical Methodol- ogy, 86(1):1-27, 2024.
Shafer. G. Testing by betting: A strategy for statistical and scientific communication. Journal of the Royal Statistical Society Series A: Statistics in Society, 184(2):407-431, 2021.
Weiss, M. Deepfake bot submissions to federal public com- ment websites cannot be distinguished from human sub- missions. Technology Science, 2019121801 2001, 2019.
Shekhar, S. and Ramdas, A. Nonparametric two-sample testing by betting. IEEE Transactions on Information Theory, 2023.
Yang, X., Cheng, W., Wu, Y., Petzold, L., Wang, W. Y., and Chen, H. Dna-gpt: Divergent n-gram analysis for training-free detection of gpt-generated text. arXiv preprint arXiv:2305.11355 2023.
Solaiman, I., Brundage, M., Clark, J., Askell, A., Herbert- Voss, A., Wu, J., Radford, A., Krueger, G., Kim, J. W., Kreps, S., et al. Release strategies and the social impacts of language models. arXiv preprint arXiv: (t:::9088 0908.0080000 09200, 2019.
Yuan, A., Coenen, A., Reif, E., and Ippolito, D. Wordcraft: story writing with large language models. In Proceedings of the 27th International Conference on Intelligent User Interfaces, pp. 841-852, 2022.
Stakel-Walker, C. Ai bot chatgpt writes smart essays-should academics worry? Nature, 2022.
Su. J., Zhuo T. Y., Wang, D., and Nakov, P. De- tectllm: Leveraging log rank information for zero-shot detection of seschine-generttee text. arXiv preprint arXiv:2306.00 05540, 2023.
Zellers, R., Holtzman, A., Rashkin, H., Bisk, Y., Farhadi, A., Roesner, F., and Choi, Defending against neural fake news. Advances in neural information processing systems, 32, 2019.
12

---

Online Detection of L.MeGeerratedd Texts via Sequential Hypothesis Testing by Betting
Zhang, T., Ladhak, F., Durmus, E., Liang, P., McKeown, K., and Hashimoto, T. B. Benchmarking large language models for news summarization. Transactions of the As- sociation for Computational Linguisties, 12:39-57, 2024.
Zhao, X., Ananth, P., Li, L., and Wang, Y.-. Provable robust watermarking for ai-generated text. arXiv preprint arXiv:2306.1743, 2023.
13

---

Online Detection of L.M-Generated Texts via Sequential Hypothesis Testing by Betting
A. More Related Works
A.1. Related Works of Detecting Machine-Generated Texts
Some methods distinguish between human-written and machine- -eeeeratee texts by comparing their statistical proper- ties (Jawahar al., 2020). Lavergne et al. (2008) introduced a method, which uses relative entropy scoring to effectively identify texts from Markovian text generators. Perplexity is also metric for detection, which quantifies the uncertainty of a model in predicting text sequences (Hashimoto et al., 2019). Gehrmann al. (2019) developed GLTR tool, which leverages statistical features such as per-word probability, rank, and entropy, to enhance the accuracy of fake-text detection. Mitchell et al. (2023) created a novel detector called DetectGPT, which identifies a machine machine-generated text by noting that it will exhibit higher log-probability than samples where some words of the original text have been rewriiten//petturbed. Su et al. (2023) then introduced two advanced methods utilizing two metrics: Log- Log-Likelihood Log-Rank Ratio (LRR) and Normalized Perturbed Log Rank (NPR), respectively. Bao et al. (2023) developed Fast- Rest-DetectGPT, which replaces the perturbation step of DetectGPT with more efficient sampling operation. Solaiman et al. (2019) employed the classic logistic regression model on TF-IDF vectors to detect texts generated by GPT-2, and noted that texts from larger GPT-2 models are more challenging to detect than those from smaller GPT-2 models. Researchers have also trained supervised models on neural network bases. Bakhtin et al. (2019) found that Energy-based models (EBMs) outperform the behavior of using the original language model log-likeld in real and fake text discrimination. Zellers et al. (2019) developed a robust detection method named GROVER by using a linear classifier, which can effectively spot Al-generated 'neural' fake news. Ippolito et al. (2019) showed that BERT-based (Devlin, 2018) classifiers outperform humans in identifying texts characterized by statistical anomalies, such as those where only the top high-likelihood words are generated, yet humans excel at semantic understanding. Solaiman et al. (2019) fine-tuned RoBERTa (Liu al., 2019) on GPT-2 outputs and achieved approximately 95% accuracy in detecting texts generated by 1.5 billion parameter GPT-2. The effectiveness of RoBERTa-based detectors is further validated across different text types, including machine-genennn et al., 2021), news articles (Uchendu et al., 2020), and product reviews (Adelani et al., 2020). Other supervised classifiers, such as GPTZero (Tian, 2023) and OpenAl's Classifier (OpenAI, 2023), have also proven to be strong detectors. Moreover, some research has explored watermarking methods that embed detectable patterns in LLM-generater texts for identifying, see e.g., Jalil & Mirza (2009); Kamaruddin et al. (2018); Abdelnabi & Fritz (2021); Zhao et al. (2023); Kirchenbauer et al. (2023); Christ et al. (2024). Recently, Kobak et al. (2024) introduced "excess word usage", novel data-driven approach that identifies LLM usage in academic writing and avoids biases that could be potentially introduced by generation prompts from traditional human text datasets.
tweets (Fagni
A.2. Related Works of Sequential Hypothesis Testing by Betting
Kelly (1956) first proposed strategy for sequential betting with initial wealth on the outcome of each coin flip gr in round t, which can take values of +1 (head) or 1 (tail), generated i.i.d. with the probability of heads p E [0, 1J. It is shown that betting fraction Sr = 2p 1 on heads in each round will yield more accumulated wealth than betting any other fixed fraction of the current wealth in the long run. Orabona & Pál (2016) demonstrated the equivalence between the minimum wealth of betting and the maximum regret in one- one-dimensional Online Linear Optimization (OLO) algorithms, which introduces the coin-betting abstraction for the design of parameter-free algorithms. Based on this foundation, Cutkosky & Orabona (2018) developed coin betting algorithm, which uses an exp-concave optimization approach through the Online Newton Step (ONS). Subsequently, Shekhar & Ramdas (2023) applied their betting strategy, along with the general principles of testing by betting as clarified by Shafer (2021), to nonparametric two-sample hypothesis testing. Chugg al. (2023) then conducted sequentially audits on both classifiers and regressors within the general two-sample testing framework established by Shekhar & Ramdas (2023), which demonstrate that this method remains robust even in the face of distribution shifts. Additionally, other studies (Orabona & Jun, 2023; Waudby- that leverage online convex optimization methods, with which the betting fraction can be adaptively selected to provide statistical guarantees for the results. We also note that sequential hypothesis testing by betting has drawn significant attention in recent years, e.g., Vovk & Wang (2021); Shaer al. (2023); Ramdas et al. (2023); Podkopaev & Ramdas (2023); Pandeva et al. (2024); Podkopaev et al. (2023); Teneggi & Sulam (2024); Grünwald et al. (2024); Podkopaev et al. (2024); Cho et al. (2024); Fischer & Ramdas (2024); Bar et al. (2024).
& Ramdas, 2024) have developed practical strategies
14

---

Online Detection of LLM- Texts via Sequential Hypothesis Testing by Betting
B. Related Score Functions
The score function é : Text - R will take a text as input and then output a real number. It is designed to maximize the ability distinguish machine text from human text, that is, we want the score function to maximize the difference in scores between Human--ritteen text and machine- -eneraatd text.
DPT Three models: source model, perturbation model and scoring model are considered in the process of calculating the score (xx) of text x by the metric of DetectGPT (the normalized perturbation discrepancy) (Mitchell et al., 2023). Firstly, the original text x is perturbed by a perturbation model &e to generate m perturbed samples g(() -c(+xx), i E 21, .- ml. then the scoring model py is used to calculate the score
(xx) = log pe (2) - i
(2)
where i = 1 E. log Po ()))), and 8² =
E [logg( - =]] We can write log Po as
Σ5=1 logge(zz
where n denotes the number of tokens of x, Ty denotes the j-th token, and T1:j-1 means the first 11):--1), log)o(z)|21._1))
j - 1 tokens. Similarly, log po (z(1))
2a log ps (z 1= where n(0) is the number of tokens of i-th perturbed
sample g(().
Fast-DetectGPT. Bao et al. (2023) considered three models: source model, sampling model and scoring model for the metric of Fast- Dest--etcctGPT (conditional probability curvature). The calculation is conducted by first using the sampling model K generate alternative samples that each consist of tokens. For each token 3y. it is sampled conditionally on T1 that is, æj ge (<())11-)1) for j 1, n. The sampled text is ō = (īI, ...). Then, the scoring model pe is used to calculate the logarithmic conditional probability of the text, given by ΣØ=1 log pe (2)11 and then normalize it, where n denotes the number of tokens of x. This score function is quantified as
((x) = Ei= log pe (rr (115-1))))) jú
3
There are two ways to calculate the mean value já and the corresponding variance 2², one is to calculate the population mean
i = Ex
log pa (2
1115 1) =
Er
logpo(z)) (T1s)-)))
n
qg (5) (1) 1x15--1) log Do (o) (11g-1);
==
if we denote Ei= 1c (x) 01:--1) 1) log po (1. (21: as ioj. then the variance is
log po (00)0 - (1)) 2)
m
0² = Eg
Eg
Σ
log² ((tf///11g-))) - ni3)
96 (0) (11--1) log" pola!! (m1--)))==))1))) 123)
j=1
where 2 denotes the i-th generated sample for the jen token of the text x. 9: (z/() (x1:j-1) is the probability of this sampled token given by the sampling model according to the probability distribution of all possible tokens at position j. conditioned on the first j 1 tokens of xr Besides, pe (3. (6) (== is the conditional probability of 2.(0) evaluated by the scoring model, represents the total number of possible tokens at each position, corresponding to the size of the entire vocabulary used by
the sampling model. We can use the same value at each position in the formula, as the vocabulary size remains consistent
across positions. The sample mean and the variance can be considered in practice
log Po( (213-11, 82 =
1og po (2 (11.-1) - -13).
15

---

Online Detection of L.M-Generaaed Texts via Sequential Hypothesis Testing by Betting
where ūi 1 substantial number of texts (m 10, 000), we can effectively map out the distribution of their log pe (15( values according to (Bao al., 2023).
m E log po(a) [1)))-). In this case, the sampling model is just used to get samples T. By sampling a
NPR. The definition of Normalized Log-Rant Perturbation (NPR) involves the perturbation operation of DetectGPT (Su et al., 2023). The scoring function of NPR is
m <xx) i=1
log ro (T())
logra
where r=(r) represents the rank of the original text evaluated by the scoring model, m perturbed samples a(0), i E ml [mm are generated based on x. The log (a:) is calculated as 1 Ej= log (oggol/// where n denotes the number of tokens of x. Similarly, log logro(*(0) केे Σド= log rogro(( (z) 11)-)) 1), where ( is the number of tokens of perturbed sample z((1) generated by the perturbation model Iç-
LRR. The score function of Log-Likelihood Log-Rank Ratio (LRR) (Su et al., 2023) consider both logarithmic conditional probability and the logarithmic conditional rank evaluated by the scoring model for text
Σ = t log Po (3y [1:5
j= log (1:511))11)
(xx)
Σ j=1 log ro (0)
== log re (z) x1:--1)
where (0(2,1215511) 2 is the rank of z;, conditioned on its previous j tokens. We suppose that the total number of tokens of a is
Likelihood. The Likelihood (Solaiman et al., 2019; Hashimoto al., 2019; Gehrmann et al., 2019) for text which has tokens can be computed by averaging the log probabilities of each token conditioned on the previous tokens in the text given its preceding context evaluated by the scoring model:
Φ(x)
log pe (1:-11)
n
LogRank, The LogRank (Gehrmann et al., 2019), is defined by firstly using the scoring model to determine the rank of each token's probability (with respect to all possible tokens at that position) and then taking the average of the logarithm of these ranks:
n log (e
(xx)
n
Entropy. Entropy measures the uncertainty of the predictive distribution for each token (Gehrmann et al., 2019). The score function is defined as:
12
((x)
po (1)) (1:-11 log (o (21:--1),
(1) where po (z (11:j-1) denotes the probability of each possible token (1) at position j evaluated by the scoring model, given the preceding context T1:j-1- The inner sum computes the entropy for each token's position by summing over all possible tokens.
DNA-GPT. The score function of DNA-GPT calculated by WScore, which compares the differences between the original and new remaining parts through probability divergence (Yang et al., 2023). Given the truncated context based on text and series of texts sampled by scoring model po based on z, denoted as (1) (2(2 ;...,.(m)) 5(m).. WScore is defined as:
(xz) log Po (x:)
Σ log Po (Z()),
16

---

Online Detection of L.M-Generaaed Texts via Sequential Hypothesis Testing by Betting
where need to note that x (.).), z(0) pe(+|=) for (1, 10,2,...,,, .m). This formula calculates the score of by comparing the logarithmic probability differences between the text and the averaged results of m samples gener- ated by the scoring model under the context which is actually the truncated x. Here, we can write the log (2()) as Ej=1 log Po (o(2,|/1:g-1)) [1::-1), which is the summation of the logarithm conditional probability of each token conditioned on the previous tokens, assuming the total number of non-pa son-padding tokens for x: is n. The calculation is similar for log po (1((), we just need to replace x in log (z (z) by E(1)
The supervised classifiers RoBERTa- and RoBERTa-large (Liu et al., 2019) use the softmax function to compute score for text z. Two classes are considered: "class = 0" represents text generated by GPT-2 model, while "class I" represents text not generated by a GPT-2 model. The score for text is defined as the probability that it is classified into class 0 by the classifier, computed as:
RoBERTaabase/lart
20
@(class = 0/²)) e50 + e
where is the logits of x corresponding to class j E {0, 13, provided by the output of the pre-trained model.
C. Proof of Regret Bound of ONS
Algorithm 2 Online Newton Step (Hazan et al., 2016)
Require: parameter y Init: 01 < 0, au + 1.
1: for = 1 to T do
2: Receive loss function le Kr + R.
3: Compute 2n (l.( ag a:-1 + 28.
4: Update via Online Newton Step:
1 Zt
Bet1 = 0,
7 dt
5: Get a hint of 1 to update the decision space Kr+1 +- 1- 2de+1 20+41
6:
Project 1 to K4+1:
Oc+1 projx,, (B(42) arg min (Be11-07 BEk++1
7: end for
Under Ho, i.e., μ2 = My: the wealth is a P-supermartingale. The value of 9i are constrained to the interval [-1, 1] in previous works (Orabona & Pál, 2016; Cutkosky & Orabona, 2018; Chugg et al., 2023) with the scores ((x,) and Φ(v:) each range from [0, 1]. To ensure that wealth W, remains nonnegative and establish the regret bound by ONS, 0, is always selected within [- /2, 1/2). In our setting, however, the actual range of score difference between two texts, denoted as 9t 6(x,) - (3x)). ). is typically unknown beforehand. If we assume that the ranges for both o(zz) and o(y)) are n.), then the range of their difference Je := Φ(v)) - )(xx) is symmetric about zero, and is given by the interval [-(n, - m,),;
m... We denote an upper bound de 2 |n,- m, and express the range as Gr E [-d,, di), where de 2 O. Then, choosing 0₂ within [-1/2d,, 1/2d,] can guarantee that the wealth is a nonnegative P-supermartingale. we consider the condition of either W. 2 1/a or WT 2 Z/a for any stopping time as the indication to "reject Ho" and apply the randomized Ville's inequality (Ramdas & Manole, 2023), the type-I error can be controlled below the significance level when Ho is true.
Under Hi, i.e.,p A My, goal is to select proper 0, at each round that can speed up the wealth accumulation. It allows us to declare the detection of an LLM once the wealth reaches the specified threshold 1/a. We can choose 6g recursively following Algorithm 2. This algorithm can guarantee the regret upper bound for exp-concave loss. Following the proof of Theorem 4.6 in (Hazan et al., 2016), we can derive the bound on the regret. The regret of choosing 9₂ E Kr₃ after T time steps by Algorithm 2 defined as
Regret,(ONS)) = Σ (()) Σ (0)))
where the loss function for each t is ls Kr + R, and the best decision in hindsight is defined as 0- e arg mingek. Σt (e))), where Kc, = n Kr.
17

---

Online Detection of LLM-Generated Texts via Sequential Hypothesis Testing by Betting
Lemma C.1. Let l Kg + R be an o-exp-concave function for each I. Let Dr represent the diameter of Kr, and G5 be bound on the (sub)gradients of le Algorithm 2, with parameter 2 min
I -, U.D,, 0r and do 1/1² D3, guarantees:
T
Regretr(ONS). S
Σ
+1)
(4)
27
1=1
where D2 is a constant for all t, and 2t, ar and Kr are defined in Algorithm 2.
Remark For any o-exp-coneave function ((),, if we let a positive number y be min J, : o J. and initialize ag = =1/>2D} at the beginning, the above inequality (4) will always hold for choosing the fraction 0r by Algorithm 2. That is, the accumulated regret after time steps, defined as the difference between the cumulative loss from adaptively choosing 0, by this ONS algorithm and the minimal cumulative loss achievable by the optimal decision 0" at each time step. is bounded by the right-hand side of (4). To prove this lemma, need to first prove Lemma C.2.
Lemma C.2. (Lemma 4.3 in Hazan et al. (2016)) Let J K - lR. be o-exp-concave function, and D, G denote the diameter of KC and bound on the (sub) gradients of f respectively. The following holds for all r = 3 min ch: a) and all 0,3 EK:
f(0) 2 f(8) + VI( 8) (0 B) V](VO111 3)-
(5)
Remark 4. For any a-exp-coneave function f(-), if we let positive number y min (GD a}. the above equation (5) will hold for any two points within the domain AC of [(-). This inequality remains valid even if > 0 is set smaller value than this minimum, although doing so will result in a looser regret bound. At time t in Algorithm 2, the diameter of the loss function & Kr, + IR is D l/de, since K, [-1/2de, 1/2de]. Additionally, the bound on the gradient of (,) is G. maxØeK, (0). .. If (o) is c-exp-concave function and 7 1/2 min(d, dG, a}. then equation (5) will hold for any 0 8 E [-1/2d, 1/2d.]-
Proof of Lemma C.2. The composition of a concave and non-decreasing function with another concave function remains concave. Given that for all y 1/ min { c.b a). we have 2 S a, the function g(0) 625//a composed with f(0) ex)(-af(0) is concave. Hence, the function h()) exp(-2y1 (0)) is also concave. Then by the definition of concavity,
2
h(0) - (()) + Vh(3) T (0 3).
(6)
We plug Vh(3) = exx((=27 into equation (6),
exp(-27.(0) S exp( (opp(221((3))) - B))
Thus,
1 In (1- (1-2:Vf(8)T 3))
f(0) 2 f(3)
27
Since D, G are previously denoted as the diameter of K. and a bound on the (sub)gradients of respectively, which means
that D 2 10- 3), G 2 VS(3). Therefore, we have V ((3 - 3) < 2D S + -1 < 2 - 3) S 1. According to the Taylor approximation, we know that 1n - a) 2 a + a holds for a 2 -1. The lemma is derived by considering a ( B).
Since problem is one diedenmnnional, then we can use Lemma C.2 to get the regret bound. Here shows the proof of Lemma C.I.
Proof of Lemma C.1. The best decision in hindsight is 6* e arg mingex, Σt= (e()), where K., = = Kr. By Lemma
C.2, we have the inequality (7) for 7/2 = min G.D. a which is
( ( < (0, 8¹)
(0: 222
(7)
(ONgg
== R,
18

---

Online Detection of L.M-Generated Texts via Sequential Hypothesis Testing by Betting
where the right hand side of the above inequality is defined as RG, the left hand side is the regret of selecting 0, via ONS time f.
We sum both sides of the inequality (7) from = 1 to T, then we get
T
(6.) - Σ ((()) - R
(8)
1=1
= -RRgrett((ONS))
We recall that D, is defined as the diameter of Kr, i.c., Dr = beK, la b/, and G, is defined as a bound on the d
gradients of the loss function (0) = - In(1 9:0) at time t, i.c., G. = maXe,eX. do, (0:))- In our setting, K₂ is [-1/2d, 1/2d]). thus D, = 1/de. The gradient 2, (l, = 9t/(1 g:0r), so we have G2 = max 0.E{ 22, z4; 1-9:0,- loel
lod
Since 9: = )(x²) )v)) E [-d,, di), and the expression 1 1- increases with the product g. E 1-1/2,1/2], the maximum is attained when 9i de and 0: + Thus, obtain G, de/(1 de 2d, 2d,. Above all, get G,D, = 2d, प, 2 for each 1, and a = 1. The value of n = D,, remains a positive constant for all t. Therefore, we simply use 7 in the following proof, given that "/. takes the same constant value for all t.
2d,
According to the update rule of the algorithm: B.+1 proje. (Be+1), and the definition: Bi+1 0 7 a.
2t. we get:
B+t1 - = 02 - 0r 2t
(9)
and
1
ax a3(4:11- 00) ar(0; 0*)
23.
(10)
7
We multiply (9) by (10) to get
=/
(Be+1 -3-)aa (0: 8, (0. 0²)
(11)
72 a:
Since 0.+1 is the projection of 3.+1 to Ke+1. and 0, Kr+1.
(3441- - 00 (0t+1 - 00-22.
(12)
Plugging (12) into (11) and rearranging terms yields
20 8*) <
22
(0,
8 3
2 (0 2aa)aa
27 az
19

---

Online Detection of L.MeGenerated Texts via Sequential Hypothesis Testing by Betting
Summing up over 1 to T.
Σ (0, - -0)
1 2
+
2 (0.- 8) ?
(011 1141--86 -87)aa
27 at
T
2
T
T-1
7 (81 81 +
Σ
7
(0, - 8))aa
Σ
(8(41 -00)aa 7 (01-1 - par
27
2
2
2
2
1=2
1=1
T
2
4
7 (0) 01 +
Σ
(0, - 0a) -Q(-1 2 (0r+1 -0")aa
+
2 01
2
2
2
1=1
1=2
1 21
2
+
1 (01 pa +
Σ
1 (0,
00) 2 ac(-1)
(since 7 0, ar 0)
27 0t
2
2
1=1
1=2
T
1 4
+
7 (81 8 (01 -2 +
y (0. 27)
(by 0r - = 2
(13)
2 at
2
According the definition: R₂ (0, lr) left hand side and get
(0, 222 We move the last term of the right hand side in (13) the
T
T
2
+ (0, - (01 - =2).
R
S
27
at
2
[=1
t=1
According to our algorithm, 01 - = ag. Since Kc, Ni= Kr S Ki, the diameter (0-- 0r s DJ. We recall the inequality (8), then
(ONS)
<
R4 <
(0
8-) 00
1
z 7 Djao-
(14)
2 ag
27
2
If we let ao 14,2D3,ii it gives Lemma C.2,
(ONS) <
1
Dia
1
T
2
DJ
1
Σ 0r
+1
27 04
2
27
2
D2
27
t=1
To get the upper bound of regret for our algorithm, we first show that the term Σt= /at/ is upper bounded by telescoping sum. For real numbers a, b 4. the first order Taylor expansion of the natural logarithm of at a implies a-b < In thus
T
T (a.
T
Σ
1 a+
a--1)
Σ tm2
In
at
In
QT
0t-1
0g
t=1
20

---

Online Detection of LLM-Generated Texts via Sequential Hypothesis Testing by Betting
In our setting, ur do Σt= 26, where dg = 1, 2t
2r
We recall the inequality (14), the upper bound of regret
1-9.0
T
Regiety(ONS))
1
Σ
(01
2
27y
at
1
uT
+
7 2
D4
27
0g
T
g
+ Di
In
1 1
27
(1
9:0,)
t=1
T
1
In
1+
Σ
9t
2
+
7 D 2
27
t=1
1
1 27
In (1+4dTT) + DJ.
(15)
Since that DD is a positive constant, 9t E [-d,, d,), 8, E 1-1/2d, 1/2d]], follows that (1 - ge0))² E [ 11. We denote
du := max (d,), which indicates that of < df < d. Consequently, we obtain that Regret, = O (In
This conclusion will be used show that the update of 9e in the betting game is play it against the exp-concave loss (0) In(1 -) and to get the lower bound of the wealth.
The reason
obtain the upper bound of regret is that, although the values of G2 and D, individually unknown, their product is deterministic. Consequently, the value of Lemma C.2 to establish the regret bound, as illustrated by equation (13), the uniform y helps us simplify and combine terms to achieve the final result.
G. min 1. a for all remains consistent. When we use
D. Lower Bound on the Learner's Log-Wealth
Lemma D.1. Assume an online learner receives loss function ;- In(1 9:0) after committing a point Og E Kr in its decision space Kr at time Denote d. := max de with de 2 [ge/- Then, if online learner plays Online Newton Step (Algorithm 2), for any benchmark u E K, mim Kr the value of the wealth satisfies
In W In (T( Regrety(ONS)
(r)
In (1+ (d+T) + 7 (0, ((01-w)))
27
where we define the wealth time T as WT(0) II'm (1 g.0), 01 K., is the initial point of ONS, and the inverse of the step size satisfies 7 1 min di 1}, with G, == maXgeK, (0)) denoting the upper bound of the gradient (l).
Proof. Since the update for the wealth is W, = W,-1 900, W 1-1. for - ,,...,
Wi Woll 9 -101);
Wy Wr-1 )r
We start with Wo = 1, then by recursion,
T II - g,0,) = II - (,,
T
Wy = Wo
1=1
(=1
thus we can express In(r)) as:
ln(W>) = Σ
In(1 -.)
(16)
t=1
21

---

Online Detection LLM-nerrrtteed Texts via Sequential Hypothesis Testing by Betting
Similarly, the benchmark has:
In(Wr(u))
Σ In(1 g.u).
(17)
We subtract equation (16) from (17) on both sides to obtain
I
r Σ
In) In (Wr) =
Σ
In(1 g:4)
In(1 - g(0+)
1=1
T
T
Σ
In(1 - ):0l)
- (-
Σ
In(1 -g:u)
1=1
t=1
In g.O,) Σ - In(1 g.
The above can be connected to the regret of the online learner, where the loss function is defined as (,) I-In1 9.0).
Consequently, we have
In(W,)) = (n(W(()) - (egret>(u).
(18)
Now applying the regret bound (15) leads to the result.
E. Proof of Proposition 3.1
Some of the ideas the proof is adapted from Dai et al. (2025), with the key difference that we consider a two-sided test, whereas their analysis focuses on a one-sided test with Bernoulli random variables. Additionally, we consider the range of gi within a time-varying interval [-d;, de] for each round can divide the proof of Proposition into three parts as below.
1. Level-a Sequential Test. In Algorithm 1, We treat the event (W, 1/a} rejecting the null hypothesis Ho- A level-a sequential test is one that guarantees, under Ho. the probability of false rejection is at most that is.
sup P(3 21:W, 2 1/a) S a, or equivalently
sup P(T < 00) S a. PEH.
PEH.
When P Ho. i.e., us pay is that
Ep(Ø(r)) 6(y1)] ur My 0.
(19)
Wealth process is calculated as W, = -1- -(t))) x Wt-1, and the initial wealth Wo = 1, then:
W. (1 - g(0.) x W
I 1 g:O,) x Wo II (1 g:0z),
where gi = (xx;) - p(y)). Since 0, - measurable and according to (19), we have
E Ep (1-g.0.) x W. Fi-1 Ep W (1 (1- (p(x)) ((v))) Fr-1 W.-b
where the last equality because 6 is fully determined by Ft-1, and hence, given Fi-1, 9, independent of (0(x;) - ((y)), where the latter has conditional expectation zero under Hy-
Thus, (W.)221 is a P-martingale with Wo 1. Given that 9i [-d,, d,] and 0, E [-1/2d, 1/2d.J, we have g.O, E [-1/2, 1/2] for all t. then W, = [[= - g.0,) remains non-negative for all t. Therefore, we can apply Ville's inequality (Ville, 1939) to establish that P(3r 2 1: W. 2 1/a) s a. This inequality shows that the sequential
22

---

Online Detection of L.M-Generated Texts via Sequential Hypothesis Testing by Betting
test: "reject Hg the wealth W, reaches 1/a" controls the type-I error level a. If there exists time budget T, will verify the final step Wr 2/a of the algorithm, i.e., treat {W; 1/a Wr 2/o} reject "Ho", which is validated by the randomized Ville's s inequality of Ramdas & Manole (2023).
2. Asymptotic Power One. Test & has asymptotic power 3 = 1 means that when H1 (i.e., Mx # py) holds, our algorithm will ensure that wealth W. 2 1/a in finite time t to reject Ho. that is:
sup PEH1
P(r oo) 31-3-00
We first recall the relation between regret and wealth in the besting- game (c.f. (18)), i.e.
ln(W,) = In(W,(0.)) - Regret, (0.),
(20)
where W/ is the learner's wealth at round t, 0, is any benchmark which lies in the decision space K., [[-
2d_
2d.
Denote w. == E[In(1 gf+)] the expected wealth of the benchmark in a single round. Taking the expectation on both sides of (20), we have
E[in(W)) = ([,n(W((..)) - E Reggret, (f-))
E
In(1 2o0.)
|E [eRgeret, (0.)]
tw. E (eegret((0)).
where the last equality holds by assuming that the random variables (ga)s21 are i.i.d.
We now analyze the probability that the null has not been rejected by time t, which is when the event (W; < 2 holds. We have
1 a
= P (w,)) < In (2)
P
W.
In(w.) - [[In(W, < In (2) - [[n((W((0)))]
TP
(n() Regret, (0.) E(In(W,(0.)) < In (1)
Blln(W.
PP [n(WW((_)) [[[n(W, < Regret, (0.) + In Iw-.
(21)
Now we are going to show that Regret,((.) + In (:) - tw. < have P [In(W((O.) )) - (ll)(W:(o.))] < Regret, (0.) + In (2) tw.] < P (l.)) ([ln(W.(0.))] < -ww.].
tuo 2
when t is sufficiently large. Then, from (21), we will
From the regret bound ONS (15), suffices to show that
1
92 9,0,)2
7 DJ + +l
In
1+
+
S wr.
(22)
2
2
2
s=1
We will require the time f to satisfy
2yD2
t
(23)
W.
which leads 3D Sw Therefore, guarantee (22), it suffices find such that
twis
1
> In
In
1
4
2
g 0g0.)²
23

---

Online Detection of L.MM-Genrrated Texts via Sequential Hypothesis Testing by Betting
I (d.)² (1 11-1/2]² = 4(d.), which in turns implies sufficient condition:
We further note that (1-9.0,)²
twis
2 In
+ 27
In (1 + 4l(d)))).
The above condition together with (23) suggests that Regret, (0.) + In (+)
tw. - -Wn holds when
(d.)
(24)
We
waa
Under (24), we hence have
1 P W. < 1 P ln(W) - ((ln(W,((.))] < __u
(25)
Denote 1/, = In(1 E(n((1 0,0.)l [ln(1/2) - (3/2)) w.. We have that In ((00))))[(((.(00)))
Σ 1/. which is of zero-mean i..d bounded random variables. Specifically, th is sub-Gaussian with parameter = 2 (3).. By Hoeffding's inequality, we have
to? 20
P
V S -c
S exp
(26)
for any constant e >> 0. Then,
P [n((W.(.)) E(n(WW.(O))) where we let
1
1
(w.)2
2
PP
1/
2 w
eexp
2(In 3)2
x in (26). Hence, we obtain
1
(-
(we) (_wwe)
P
W.
exp
2(I 3)2
(27)
a
Let H, be the event that we stop at time Then,
Ptt = oo] =
lim nest 1Hg
1000
lim P P[ -H
t-000
lim P [[Hi,]
lim P
W,
a
(w
lim exp
2(I 3)2
(+000
where the last equality is by (27). This shows that the test power is one.
Expected Stopping Time.
Now we upper-bound the expected stopping time follows. Denote f,, the time when (25) starts to hold at any & 2 t. We
24

---

Online Detection of L.M-Generated Texts via Sequential Hypothesis Testing by Betting
have
P > t]
Σ P P[nge H.)
PP [-H+)]
Wi <
a
1=1
(wa)2
St. +
exp
2113322
1
S t, +
exp
2(1n3)²
(wx)2
1)
1
S + 2(In3)2
(28)
w
where the last inequality uses exp(2) 2 1 + z for any 2 2 O.
To proceed, we recall the notation w. : [In(1 - g00)]] We now let the benchmark 0, to be one that maximizes the expected wealth in one round, i.e., [In(1 gB)]. in the decision space K.,.
By the inequality In(1 + c) 2 c² for any |cl - +. have
max E [In (1 g0)l 2 max E [00g - g²]
OEK.
max 0(1 - Hwy) - 0² (Var(gl + (Az - 2y)))) OEK.
A
Var [g] + A2
A
where we note Big]] Bl((x) - <by))
nz Myy Var[g] Elgg! (B(g))², and Varlgl s d.
Combining (24) and (28), together with the expression of above, have
0 de do
B = 0
In
+
(29)
A
a
F. Proof of Proposition 3.2
Recall formulated one-sided hypotheses:
HD us - My 00 vs. H1 Ma By - 0,
and
H8 By 1A - e H. My 14 0.
Level-a Sequential Test. contrast to the betting interval employed in Chugg et al. (2023) for composite hypotheses test, which includes both positive and negative values, we specify nonpositive decision space 0, [-1/2d,, 0] order to
25

---

Online Detection of L.M-Geeerated Texts via Sequential Hypothesis Testing by Betting
Algorithm 3 Online Detection of LLMs via Online Optimization and Betting for the Composite Hypotheses Testing
Require: a score function d(.) Text + R.
1: Init: 0f: f 0, a "0 wealth Wo, W <- 1, step size y, difference parameter 6, and significance level parameter E (0,
2: for ( 1, do
3: # '' is the time budget, which can be oc if their is no time constraint.
4: Observe a text Y/ from an unknown source and compute Φ(ve).
5: Sample zt from a dataset human-written texts and compute ((x)).
6: Set of ((xx)) - (bye) c, It = <b(y)) ((T+)) - Update wealth W.A [-1 (1- ge 0 W w 1 -f)
7:
8:
if W,A 2 2/a W.P. 2 2/a then
9:
Declare that the source producing the sequence of texts yt is an LLM.
10: end if
11: Get a hint 1 and specify the decision space Kz+1 == 1- 1-==11------ 2d.41
Update //4 0 KL+1 via ONS on the loss function e, == In 9) and (B((0) In(1 g, g,00)
Compute =5- 10,(0
g. =B
de,(0")
13:
do
I 1-
d9
8,
14:
Compute o² at + (z at = 0
+ (zt
15:
Compute 0141 max min
0,1 1
0
2d,13
0ff = max
min (05 1=f.0) 2d
16: end for
17:
if the source has not been declared LLM then
18:
Sample Z ~ Unif(0, 1), declare the sequence of texts is from an LLM if W$ 2 27/a Wp 2 27//a.
19:
end if
ensure that the wealth process is supermartingale under Ho, while still enabling fast wealth growth under the alternative H1. The resulting wealth process is now given by:
W.A Wo II (1 0. -() II (1 0. c))
and
W Wo II (1 0,(-9. -)) II (1 0.( - -c))
where 9. - ((x.)) - (va)). Recall that e is a nonnegative constant satisfying e S d, for all Consequently, the range of (g -)) and P <). given by (d e) /2d, (d2 e)/2d)), always lies within the interval [-1, This guarantees that the wealth process remains nonnegative. Moreover, the supermartingale property of the wealth processes W.A and :. can still be preserved. Take W.A an example, under the corresponding null hypothesis Ho* Mr My e, i.e., Ep [Ø(x)) d(v))) < for P E H Then,
Ep
W 1 - Or (6(x)) - ((y)) -0) Fr-1
S SW
where the inequality is because Or is fully determined by Fi-1, and hence, given Fi-1, O is independent of (6(x,)-<66) - <), where the latter has conditional expectation less than or equal to zero under H Me My S e.
As for the other null hypothesis H : My us S e, i.c. Ep (o(y.) - ((xx)] S e, it can be shown that the wealth process
W.B also forms a nonnegative supermartingale.
Applying the randomized Ville's inequality gives us
P P W. 2 2/a) S 0/22
(30)
26

---

Online Detection of L.M-Generated Texts via Sequential Hypothesis Testing by Betting
and
1 w. 2//a) S 0/2.
(31)
Thus we can get the union bound of (30) and (31)
P( 2 :: (W, >
2 2/a)) sa,
which indicates that rejecting the null hypothesis when either W/ > 2/a W.P 2 2/a a is level-a sequential test. The detection process for the composite hypothesis is shown as Algorithm 3.
2. Asymptotic Power One. Now let us switch to analyze the test power of the proposed algorithm. We first recall the relation between regret and wealth in the
game (c.f. (18)), ic.,
In(Wa)) = In(WW, (0.) Regret,
(32)
where W.A is the learner's wealth round 6, 0, is any benchmark which lies in the decision space K2, 1 ___ d' = de +
- with
Denote / := E[In(1 - (g - <)0,)l the expected wealth of the benchmark in a single round. For the notation simplicity, in the following, we will drop the superseript A in w when it is clear to the context, Taking the expectation on both sides of (32), we have
E[ln(W:A]] = B[In (O,,(O.)))) (0.))) - E [Reegret((.))
= E
1n(1 - (g. -)0a)
- E Reggeet (0.)]
tw
E Regrett
where the last equality holds by assuming that the random variables (ga)x21 are i.i.d.
We now analyze the probability that the null has not been rejected by time t, which is when both the events { < 2 and WW < 2 hold. We have
W. < 2 = In(W;)) < In (2)
P In(W.)) - Wlnn(W, In (2) -F(n(W;((
= P In(w;A(a)) (O.)) Regret, (0.) E(In(W,A(0.)) <
2
1I (W, (W, (0.))]
α
2
= P (CIWA(O.)) B[In(W," < Regret, (0.) + lwo.
(33)
Now are going to show that Regret, (0.) + In (2) twi < - سس: when t is sufficiently large. Then, from (33), we will have P (l.)) ([In(W;((O.))) Regret,(0.)) + In (2) [w.] [In(W:((O)) E]n(WA((0))] < -$w.].
From the regret bound of ONS (15), it suffices to show that
1 2-
In
1
Σ
(g. - e)²
D
+ In
2 < w..
(34)
(gr < 0,,3
2
0
We will require the time f satisfy
t D7
(35)
W+
which leads 3D3 < two
Therefore, guarantee (34), it suffices find such that
4
two
2
1 ln
(g. - e)²
In
1 +
4
27
(gs e )0,))
27

---

Online Detection of L.MM-eeerated Texts via Sequential Hypothesis Testing by Betting
(g.-c)² We further note that (1=(9)-0)0,)² S (1-(dxte)/2d,)) I c(dd,),, for some constant e c2 sufficient condition:
(x,)²²
2d. (d-e))
which in turns implies a
tw
In
In (1 + ct(d))²).
27
The above condition together with (35) suggests that Regret, (0.) + In (2) lwa S -ww holds when
(d.)
~ w.
In
(36)
waa
Under (36), we hence have
2
W A_ < S ln(W:A(0.) - ([In(W;>(0))) < - w+ 2
(37)
Denote 1/ == In(1 - (gt - c)o.) - E[In(1 - (g - ] E [ln(1/2) - w., (332)) - -wll. We have that In(W,A(O.) - ([In(W((O.)) = - s/ which is sum of zero-mean i.i.d bounded random variables. Specifically, the a sub-Gaussian with parameter == In)).. By Hoeffding's s inequality, we have
(-
tc
P
3
S
c
S exp
-
2
(38)
for any constant 0. Then,
P [[(ewA(o)) - E[n,w; (0.)) < where we let c t
1
(-
(w.)²
2 = P
w/
S exp
1
2
2(n 3)2
wa in (38). Combining the above inequality and (37), we obtain 2
2
I <exp (
(w-)2 2(I 3)2²
TP W.
(39)
Similarly, we have
W.B < 2
(w
S exp
2((13)2'
(40)
where w := [nn (- c)0 for a benchmark 0..
Let He be the event that we stop at time f, i.e., H, = (W₁** 2 or W.B 2 2 Then, we deduce that
PP[ xo] = IP lim Nest
11000
lim P P[ Hog] 11000
lim P P[-Hi]
2 and W
2
lim P t-000
W
a
a
2
2
lim P 60000
W:
p
W.B
a
(W
(w y2 2 3)2
< lim exp
+ exp
1-000
0,
where the last equality is by (39) and (40). This shows that the test power is one.
28

---

Online Detection of LeneGreeeateed Texts via Sequential Hypothesis Testing by Betting
3. Expected Rejection Time т. Now we upper-bound the expected stopping time as follows. Denote 4, the time when (39) and (40) starts to hold at any t 2 1.. We have
B
[T > t]
TP P
IP [-H+]
1=1
2 +P
2
2
P
W.
W.B <
a
(w:1)2
+ exp (-
(w
exp
2(In 3|3
2(113)2
1=1
< 1, +
(w
exp
2(1n3)2
exp
21
S 1. + 2(I 3)2
(wg)) + 2)
(41)
(w!))²
where the last inequality uses exp(2) 2 + x for any = 220
To proceed, we recall the notation w : E [In(1 (g =e)o.]]- We now let the benchmark 0, to be the one that maximizes the expected wealth in one round, i.e., B [In (g e)) in the decision space K.,
By the inequality In(1 + c) 2 c c ²² for any |c| S 2. we have
W max E [n (1 (g c)0)] 2 OEK!
max E
[ 0ig c) - 0g -c)²]
BEK!
011 1by c) 0² (Varlg c) + (Ux Mby -)))
max BEK!
(A c)2
Varlg c| + (A E)²
(A o)²
(d.)² + (A - cc)2¹
where we note E[g] = B(((()) - (b))) = M2= - My, Varlg - E] = =El(g c)²] - (B((g - +)])², and Varlg - c] s (d'))2
Similarly, we have w [dt (A-
Combining (36) and (41), together with the expression of waa and wiB above, we have
E(r] = 0 ( (A c)²
(d, +e²² In
( (((A- (o(A-e) )
d, +e
(d. + +c)* A e)d
(42)
G. Experimental Results of Detecting 2024 Olympic News or Machine-generated News
Our generation process for fake news guided by Mitchell al. (2023); Bao al. (2023); Su al. (2023). Specifically, we use the T5 tokenizer to process each mumttewritten news article to retrieve the first 30 tokens {prefi)). Then, we initiate the generation process by sending the following messages to the model service, such as: "You are a News writer. Please write an article with about 1,50 words starting exactly with (prefix3."
Experimental Results for Two Scenarios. Figure 5 and Figure 6 show the results of detecting real Olympic news and news generated by Gemini-1.5 1.5-Flash, Gemini-1.5-Pru and PaLM 2, when designating human-written text from the XSum dataset as Fy in Scenario and Scenario 2, respectively. In Scenario 1, our algorithm consistently controls the FPRs below
29

---

Online Detection of LLM-Generated Texts via Sequential Hypothesis Testing by Betting
the significance level a for all source models, scoring models, and score functions. This is because we set the true value of A between two sequences of human texts to 6, which satisfies the null hypothesis condition, i.e., |/4 144--// Pyy S This ensures that the wealth remains a supermartingale. Texts generated by PaLM 2 are detected almost immediately by most score functions within 100 time steps, as illustrated in Figure Se and Figure 5f. Conversely, fake Olympic news generated by Gemini-1.5-Pre often fails to be identified as L.g--generteed within 500 time steps under certain score functions, as shown in Figure 5c and Figure 5d. Vertical lines in Figure 5a-5d are displayed because the the values for human texts and fake news, as shown Table 1, are smaller than the corresponding values. This means that the score discrepancies between fake news and XSum texts do not exceed the threshold necessary for rejecting Ho. Although under H1, the Á for Entropy when using Gemma-2B score texts generated by Gemini-1.5- is 0.2745, larger than the value of (0.2690), the discrepancy is too small to lead to rejection of Ho within 500 time steps.
According to Figure 6, Scenario 2 exhibits a similar trend to that observed in Scenario 1, where texts generated by PaLM 2 are quickly declared as originating from an LLM, while texts produced by Gemini-1.5-Pro are identified more slowly. In Scenario 2, DesttDetettGPT consistently outperforms all other score functions when using Neo-2.7 as scoring model, as evidenced by the results Figure 6a, 6c and 6e. Only the score funetions of supervised classifiers have FPRs slightly above the significance level a. Although their average estimated values of in Table are larger than the actual A value under Hg in Table high FPRs often occur because most values estimated in 1000 repeated tests smaller than the actual A values. When using Gemma-2B as the scoring model in Scenario 2, four score functions: Fast-DetectGPT, LRR, Likelihood, and DNA-GPT consistently maintain FPRs within the expected range a. Likelihood is the fastest to reject Ho. However, estimated values of DetectGPT, NPR, and Entropy are smaller than real A values under Ho- Then, even when Ho is true, the discrepancy between human texts exceed the estimated threshold e for rejecting Ho, which result in high FPRs, as shown in Figure 6b, 6d and 6f.
Table 1. Values of A. which calculated according to A Ei (zz:)/500 Σj (5s))/500 where o(x;) is score of the i-th text I, from XSum, ((y:) score the j-th text yj from source be detected. Every two columns starting from the third column represent the A values under Hi and Ho for each test scenario. For instance, in calculating A for the third column, Y/ represents the j-th fake news generated by Gemini-1.5-4 1.5-Flash based on pre-tokens of Olympic 2024 news. For the fourth column, Yj refers to the j-th 2024 Olympic news articles. Values in Column "Human, Human" are also used to set values for tests in Scenario 1.
XSum, Olympic
XSum, Olympic
XSum, Olympic
Scoring Model
Score Function
Human, 1.5-Flash
Human, Human
Human, 1.5-Pro
Human, Human
Human, PaLM
Human, Human
DPtt-DeeetGPT
2.4786
0.3634
1,2992
0,3660
3.6338
0.4232
DetectGPT
0.3917
0.0202
0.3101
0.0274
0.6050
0,0052
NPR
0.0232
0.0014
0.0155
0.0015
0.0398
0.0005
LRR
0.1042
0.0324
0.0289
0.0328
0.2606
0.0370
Neo-2.7
Logrank
0.2590
0.0543
0.1312
0.0561
0.4995
0.0743
Likelihood
0.3882
0.0618
0.2170
0.0652
0.7641
0.0948
Entropy
0.0481
0.0766
0.0067
0.0728
0.1878
0.0483
DNA-GPT
0.1937
0.0968
0.0957
0.1032
0.4086
0.1083
RoBERTa-hase
0.2263
0.0461
0.6287
0.0491
0.6343
0.0370
RoBERTa-large
0.0885
0.0240
0.0249
0.0250
0.4197
0.0281
Gast-DetecttPP
2.1412
0.5889
0.9321
0.5977
3.7314
0.6758
DeteetGPT
0.7146
0.3538
0.6193
0.3530
0.8403
0,3360
NPR
0.0632
0.0254
0.0477
0.0249
0.1005
0.0232
LRR
0.1604
0.0129
0.0825
0.0112
0.3810
0.0038
Gemma-2B
Logrank
0.3702
0.0973
0.2527
0,0932
0.5917
0.0687
Likelihood
0.6093
0.1832
0.4276
0.1761
0,9705
0.1358
Entropy
0.2668
0.2743
0.2745
0.2690
0.4543
0.2347
DNA-GPT
0.2279
0.0353
0.1144
0.0491
0.4072
0.0681
RoBERTa-base
0.2266
0.0461
0.0287
0.0491
0.6343
0.0370
RoBERTa-large
0.0885
0.0240
0.0249
0.0250
0,4197
0.0281
To summarize, the FPRs can be controlled below the significance level a if the preset is greater than or equal to the actual absolute difference in mean scores between two sequences of human texts. However, if is greater than or is nearly equal to the A value for human text x, and machine -eeerattd text ye, it would be challenging for our algorithm to declare the
30

---

Online Detection of LLMeGGeerateed Texts via Sequential Hypothesis Testing by Betting
Falso Positiv Rate tate (FPR)
False Posttive Rate (FPR)
0.10
False Positive Rate (FPR)
False Positive Rate (FPR)
0.10
0.04
0.04
0.03-
0.03
0.02-
0.04
0.02
0.04
0.01
0.02
0.01
0.02
0.00
0.00
0
100 200 300 400 500
0.00
0.02 0.04 0.06 0.08 0.10
0
100 200 300 400 500
0.00 0.02 0.04 0.06 0.08 0.10
Rejection (r)
Significance
Rejection (r)
Significance Level
Fast-Detect DetectGPT
NPR LAR
LogRank Likellhood
Entropy DNA-GPT
RoB-base RoB-large
Fast-Detect DetectGPT
NPR
LogRank Likellhood
Entropy DNA-GPT
RoB-base RoB-large
LRR
(a) Results for detecting the source of yı (2024 Olympic
(b) Results for detecting the source y (2024 Olympic
news generated by 1omini-1.5--lashh. with human- written text at
news generated by Gemini-1.5-Flashy
human- text as
sampled from XSum. The scoring model used is Neo-2.7.
sampled from XSum. The scoring model used is Gemma-2B.
False Pos-tive Rate (FRR)
False Pos Rate (FPR)
0.10
False Pos-tive Rate (FRR)
ralse Pos Rate (FRR)
0.10
0.04-
0.08
0.04
0.08
0.03-
0.03
0.02-
0.02
0.01
0.02
0.01
0.02
0.00
0.00
100 200 300 400 500 Rejection
0.00 0.02 0.04 0.06 0.08 0.10
0
100 200 300 400 500 Rejection
0.00 0.02 0.04 0.06 0.08 0.10
(r)
Significance
Significance
Fasl-Deleci DetectGPT
NPR LAR
LogRank Likelihaod
Entropy DNA-GPT
RoB BeB-bass
Fasl-Detecl DetectGPT
NPR LAR
LogRank Likelihaod
Entropy DNA-GPT
nuB-base RoB-large
RoB-large
(c) Results for detecting the source of y, (2024 Olympic
(d) Results for detecting the source of ye (2024 Olympic news generated by Gemini-1.5-Pro), with human-written text ay
news generated by Gemini-1.5-Pro), with numan-writted text æt
sampled from XSum. The scoring model used is Neo-2.7.
sampled from XSum. The scoring model used is Gemma-2B.
False Positve Rate (FPR)
ralsa Positive Rate (FP6)
0.10
False Posttive Rate (FPR)
ralsa Posttive Rate (FPR)
0.10
0.04-
0.08
0.04
0.08
0.03
0.06
0.03
0.06
0.01
0.01
0.00-
0.00
0.00
0.00
50 150 200 250
0.00 0.02 0.04 0.06 0.08 0.10
120 160
0.00 0.02 0.04 0.06 0.08 0.10
Rejection Time (r)
Significance Level (e)
Rejection Time (T)
Significance Level (e)
Fast-Detect
+ NPR LRR
LogRank Likellhood
Entropy DNA-GPT
RoB-base
Fast-Detect
* NPR LRR
LogRank Likellhood
Entropy DNA-GPT
RoB-base
DetectGPT
Ros-large
DetectGPT
RoB-large
(e) Results for detccting the source of y, (2024 Olympic news vs. news generated by PaLM 2), with numan-written text xe sampled from XSum. The scoring model used is Nco-2.7.
(f) Results for detecting the source of ye (2024 Olympic news vs. news generated by PaLM 2), with human- written text 2e sampled from XSum. The scoring model used is Gemma-2B.
Figure Results for detecting 2024 Olympic news and machine-generatee news with Algorithm Scenario We use source models: Gemini-1.5- 1.5-Flash, Gemini-1.5- 1.5-Po and PaLM generate fake news and scoring models: Nco-2.7, Gemma-2B. The left column displays results using the Nco-2.7 scoring model, while the right column presents results using the Gemma-2B scoring model. Score functions of supervised classifiers (RoB-base and RoB-large are independent of scoring models.
31

---

Online Detection of LLMeGGeerateed Texts via Sequential Hypothesis Testing by Betting
False Posiivve Rate (FPR)
0.12-
False positive Rate (FPR)
0.12
False Positive Rate (FPR)
1.0
Faloe Positive Rate (FPR)
1.0
0.8
0.8
0.6
0.6
0.06-
0.06
0.4
0.4
0.03
0.03
0.2
0.2
0.0
0.0
0
100 200 300 400 500
0.00 0.02 0.04 0.06 0.b8
0.10
100
200
300 400 500
0.00
0.04 0.06 0.08 0.10 Significance Level (a)
0.02
Rejection Time (r)
Significance Level (o)
Rejection Time (r)
Fast-Detect
Entropy
Fast-Detect DetectGPT
LogRank
Entropy
DetectGPT
Likellhood
RoB-large
RoB-large
(a) Results for detecting the source of y, (2024 Olympic news vs.
(b) Results for detecting the source of yı (2024 Olympic news vs. generated by Gemini-1.5-Flaat with numan-writte text at: sampled from XSum. The scoring model used Gemma-2B6
generated Semini-1.5--Faan with human-writter text are
sampled from XSum. The scoring model used Nco-2.7,
False Positive Rake (F9R)
0.12
glse Postive hake (FRR)
0.12
False Positive Rate (FPR)
1.0
Felse Positive Rate (FPR)
1.0
0.10
0.10
0.8
0.8
0.6-
0.6
0.06
0.06
0.4
0.4
0.04
0.04
0.2
0.00
0.00
0.0
100
200 300 400 s00
0.00 0.02 0.04 0.06 0.08 0.10 Significance Level (a)
100 400 500
0.00
0.02 0.04 0.06 0.08 0.10 Significance Level la)
Rejection Time (t)
Rejection Time (T)
Fast Detect DetectGPT
* NPR LRR
LogRank Likelihaod
Entropy
*- RoB bass-base
Fast-Detect * NPR DetectGPT
LooRank Likelihood
Entropy
RoB-base
DNA GPT
RoB large
* LRR
DNA-GPT RoB-large
(c) Results for detecting the source of ys (2024 Olympic news vs. news generated by Gemini-I.5-Pro), with buman-wrilten text We
(d) Results for detecting the souree of y (2024 Olympic news vs. news generated by Gemini-1.5-Proy, with buman-written text as
sampled from XSum. The scoring model used is Neo-2.7.
sampled from XSum. The scoring model used is Gemma-2B.
Fase Positvv Rate (FPR)
ase Rositvve Rtat (FPR)
False Positvve Rate (FPR)
0.8
Fabs Positive Rate (FPR)
0.8
0.08
0.6
0.6-
0.06
0.4
0.4
0.04
0.04
0.2
0.2-
0.02
0.02
0.00
0.00
0.0
0.0
0
100 200
300 400 500
0.00 0.02 0.04 0.06 0.08 0.10
0
40
80
120 160 200
0.00 0.02 0.04 0.06 0.08 0.10
Rejaction Time (T)
Significance Level (a)
Rejection Time
Significance Level ta)
Fast-Detect DesectGPT
LogRank Likellhood
Entropy DNA-GPT Rog-lard large
base
Fast-Debect DetectGPT
LogRank
Entropy
base RoB-large
(e) Results for detecting the source of y, (2024 Olympic news news generated by PaLM 2), with human written text 3t sampled from XSum. The scoring model used is Neo-2.7.
(f) Results for detecting the source of y (2024 Olympic news news generated by PaLM 2). with human written text æ; sampled from XSum. The scoring model used Gemma-2B.
Figure 6. Results for detecting 2024 Olympic news and muchine-generated news with algorithm for Scenario 2. We use models: Gemini-La 1.5-Flash, 1.5-nn and PaLM to generate fake news and scoring models: Neo-2.7, 7,7, Gemma-2B. The left column displays results using Neo-2.7 scoring model, while right column presents results using Gemma-28 scoring model. Score functions of supervised classifiers (RoB base and RoB- RoBglarge) are independent of scoring models.
32

---

Online Detection of LLM-GGnerated Texts via Sequential Hypothesis Testing by Betting
Table Values of de used in Scenario for which we assume that range [gel ( <(ve] ] known heforehand. Specifically, de is calculated max
((x))l where ((2;) is the score of i-th XSum text and ((v;) is the score of the text generated by Gemini-1. -.5-Fhahh, prompted by pre-tokens of 2024 Olympic news. This calculation ensures that di 2 [o(x;) ((y+)l for any time point 1 < f I 500. Every two columns starting from the third column represent the d, values for each used under H₃ and Ho in each test scenario. For instance, the values in the third column Similarly, the fourth column calculates the maximum difference between the scores of all XSum texts and texts sample of 2024 Olympic algorithm, where 0g E [-1/2d,, 0].
The derived de values is then used define the domain of in
XSum, Olympic
XSum, Olympic
XSum, Olympic
Scoring Model
Score Function
Human, 1.5-Flash
Human, Human
Human,
Human, Human
Human,
Human,
1.5-Pro
PaLM 2
Human
Gett-DeecCGPT
7.6444
5,9956
6,5104
6.1546
9.1603
5,8870
DetectGPT
2.3985
2.3102
2.1416
2.2683
2.6095
2.7447
NPR
0.1500
0.1436
0.1295
0.1465
0.1975
0.1353
LRR
0.8129
0.5877
0.6400
0.5875
0.9793
0.5421
Neo-2.7
Logrank
1.5861
1.6355
1.4065
1.6355
1.7298
1.6355
Likelihood
2.3004
2.4540
1.9491
2.4540
2.6607
2.5559
Entropy
1.6523
1.6630
1.5890
1.6702
1.9538
1.6265
DNA-GPT
1.5063
1.5425
1.3621
1.5649
1.5455
16348
base
0,9997
0.9995
0.9995
0.9995
0.9997
0.9996
RoBERTa-large
0.9983
0.9856
0.8945
0.8945
0.9992
0.8608
East-DetectGPP
7.7651
6,5619
7.3119
6,4343
8.6156
6,5640
DetectGPT
2.9905
2.5449
2.3846
2.4274
2,6807
2.7878
NPR
0.3357
0.2196
0.3552
0.2318
0,4118
0.2403
LRR
1.1189
0.7123
0.8780
0.7109
1.2897
0.7717
Gemma-2B
Logrank
1.5731
1.6467
1.4713
1.6468
1.7538
1.6397
Likelihood
2.4934
2.4229
2.3944
2.4228
2.8379
2.4694
Entropy
1.9791
1.9117
1.8572
1.8854
2.1359
1.9210
DNA-GPT
1.3214
1.4808
1,2891
1.4607
1.5014
1.6296
RoBERTa-base
0.9997
0.9995
0.9995
0.9995
0.9997
0.9996
RoBERTa-large
0.9983
0.9856
0.8945
0.8945
0.9992
0.8608
source of as an LLM within limited number of time steps under H1.
Moreover, we found that the rejection time is related to the relative magnitude of A -e and d, - e. According to the definition of nonnegative wealth W.A = W (1 (g. - -)) or W W. (1 0,(-9t - -)), large -0, within the range [0, 1/2d;] will result in large wealth which allows to quickly reach the threshold for wealth to correctly declare the unknown source an LLM. Based on the previous proposition of the expected time upper bound for composite hypothesis, we guess that the actual rejection time in our experiment is probably related to the relative magnitude of A and d, e, where d, is certain value for any t in each test as shown in Table 2. We define the relative magnitude as T,-c and sort the score functions by this ratio from largest to smallest for Scenario 1, as displayed in the Rank column in Table 3. This ranking roughly corresponds to the chronological order of rejection shown in Figure 5. The quick declaration of an LLM source when yr. is generated by PaLM and the slower rejection Ho when y: is generated by Gemini- 1.5-Pro in Figure could be attributed to the relatively larger and smaller values of A- respectively. The negative ratios arise because A e, leading to the vertical lines in the figures. For Scenario 2, the values of A-r J.-E are provided in Table 4, and the conclusions
from Scenario extend to Scenario 2.
Thus, we prefer a larger discrepancy between scores of human-written texts and machine machine-generated texts, which can increase A. Furthermore, a smaller variation among scores of texts from the same source can reduce e. These properties facilitate a shorter rejection time under H1.
Based on the results, when we know the actual value of de and c, algorithm is very effective. When estimate their values based on the previous samples that we get, the algorithm can still exhibit good performance for most score functions. It can be inferred that the rejection time and FPRs are effected by the score function Φ(-) and the scoring model that we select. If the configuration can further amplify the score discrepancy between human- written texts and machine- -enneratdd texts, the rejection time can be shortened. If the the score discrepancy between human texts is small, the value FPR will be low.
33

---

Online Detection of L.M.Generated Texts via Sequential Hypothesis Testing by Betting
Table 3. Values of the ratio A-c for Scenario 1, where A and are listed in Table 1, di are shown in Table 2. We sort the score function according the ratio from largest to smallest, as shown in Rank column. This ranking roughly corresponds the chronological order of rejection in in Figure 5.
Human, 1,5- L.5Fllash
Human, 1.5-Pro
Human, PaLM 2
Scoring Model
Score Function
Ratio
Rank
Ratio
Rank
Ratio
Rank
Dest-DeeecGPTT
0.2905
0.1519
0.3675
3
DetectGPT
0.1562
0.1337
2
0.2303
7
NPR 0.1467
4
0.1093
3
0.1995
9
LRR 0.0920
-0.0064
0.2373
6
Neo-2.7
Logrank 0.1336
0.0556
0.2568
5
Likelihood
0,1458
0.0806
0.2608
4
Entropy
-0.1111
-0.436 10
10
0,0732
10
DNA-GPT 0.0687
-0,0000
0.2089
8
RoBERTa-base
0.1892
00.0215
7.6600
RoBERTa-
0.0662
-0.0001
0.4032
2
Fast-DetectGPT
0.2163
0.0498
0.3848
3
DetectGPT
0.1368
0.1311
0.2151
8
NPR
0.1218
0.0690
5
0.1989
9
LRR
0.1334
0.0823
0.3104 4
0.2933
6
Gemma-2B
Logrank
0.1849
0.1157
Likelihood
0.1844
0.1134
0.3089
5
Entropy
-0.0044
10
0.0035
8
0.1155
10
DNA-GPT 0.1498
0.0527
0.2366
7
boBeRTa-base
0.1892
-0.0215
TO
7..600
RoBERTa-late
0.0662
-0.0001
9
0.4032
2
Table Values of the ratio A-e for Scenario 2, where A and are listed in Table and Table 5 respectively, de are shown in Table 6. We d:-c
the scoring function according the ratio from largest smallest, shown in the Rank column. This ranking roughly corresponds to the chronological order of rejection in in Figure
Human, 1.5Fllah
Human, 1.5-Pro
Human, PaLM
Scoring Model
Score Function
Ratio
Rank
Ratio
Rank
Ratio
Rank
Fast DetectGPT
0.1895
0.0874
0.2420
2
DetectGPT
0.0427
0.0103
0.1024
9
NPR 0,0592
2
0.0076
0.1268
8
LRR
0.0474
-0.0653 5
7
0.1568
7
Neo-2.7
Logrank 0.0576
-0.0218 4
0.1687
4
Likelihood
0.0582
-0.0132
0.1680
Entropy
-0.0833
-0.1162 10
10
-0.0051
10
DNA-GPT
0.0405
-0.0298 8
0.1636
6
RoBERTa-base
0.0461
-0.1042 6
0.2713
RoBERTa-large
0.0092
-0.0942
0.1814
3
Past-DetcetGPT
0.1562
0.0368
0.2444
2
DetectGPT
0.1351
0.1147 2
0.1552
8
NPR
0.1492
0.1167
0.1961
6
LRR
0.0867
6
0.0117
0.1959
5
Gemma-2B
Logrank
0.1205
0.0607
0.2086
4
Likelihood
0.1267
0.0702 9
0.2144
3
Entropy
0.0270
0.0293
0.1017
10
DNA-GPT
0,0713
7
-0.0144
8
0.1747
9
RoBERTa-base
0.0462
8
-0.1007
0.2737
RoBERTa- large
0.0060
10
-0.1013
0.1803
Comparisons with Baselines. Permutation test is a fixed-time test. The goal is to test whether the source of text yo the same as that the womatewritten text Ft, i.e., whether their scores are from the same distribution. we choose the mean
34

---

Online Detection of LLM-Generated Texts via Sequential Hypothesis Testing by Betting
Table Average values of used in Scenario 2 estimated by 20 texts in sequence of human-written zt. Every columns starting from the third column represent the values used for H1 and Ho for each test scenario. For instance, the third column calculates e tests between XSum text and Gemini-1.3 1.5-Flash-ger -raah-ggeeerated text sequences by scoring XSum texts, dividing them into two equal groups, and then doubling the average absolute mean difference between these groups across 1000 random shuffles. The fourth column follows the same method to determine the value tests between XSum texts and 2024 Olympic news. This calculated as
<= 2 1000 Ens E (( )/10 Ei / where ((x;")) denotes the score of the i-th text after the n-th random
shuffling of 20 text scores.
XSum, Olympic
XSum, Olympic
XSum, Olympic
Scoring Model
Score Function
Human, 1.5-Flash
Human.
Human,
Human,
Human,
Human,
Human
1.5-Pro
Human
PaLM 2
Human
Fast-DetectPT
0.6357
0.6371
0.6395
0.6417
0.6415
0.6426
DetectGPT
0.2781
0.2807
0.2840
0.2870
0.2851
0.2847
NPR
0.0141
0.0141
0.0145
0.0146
0.0140
0.0141
LRR
0.0665
0.0666
0.0674
0.0676
0.0652
0.0649
Neo-2.7
Logrank
0.1634
0.1632
0.1624
0.1624
0.1597
0.1590
Likelihood
0.2448
0.2450
0.2455
0.2426
0.2419
0.2416
Entropy
0.1994
0.1978
0.2022
0,2000
0.1973
0.2004
DNA-GPT
0.1404
0.1394
0.1321
0.1314
0.1345
0.1320
base
0.1438
0.1459
0.I463
0.1496
0.I261
0.I206
RoBERTa-large
0.0768
0.0787
0.0732
0.0740
0,0821
0.0853
Fast- DetectGPT
0.6806
0.6785
0.6749
0.6726
0.6781
0.6825
DetectGPT
0.2731
0.2725
0.2685
0.2664
0.2822
0.2840
NPR
0.0170
0.0168
0.0171
0.0171
0.0169
0.0169
LRR
0.0724
0.0728
0.0732
0.0735
0.0725
0.0719
Gemma-2B
Logrank
0.1547
0.1542
0.1567
0.1550
0.1556
0.1521
Likelihood
0.2438
0.2414
0.2472
0.2473
0.2427
0.2429
Entropy
0.2082
0.2088
0.2120
0.2092
0.2076
0.2078
DNA-GPT
0.1354
0.1355
0.1317
0.1325
0.1278
0.1286
base
0.1438
0.1433
0.1437
0.1448
0.T198
0.T208
RoBERTa-large
0.0807
0.0800
0.0750
0.0743
0.0831
0.0823
value the test statistic, the null hypothesis is that the means are equal (Ho Ne 14y) with batch size of k. Once we have generated & samples from these two sources, we conduct the test. When Hn is true, the samples are drawn from the same distribution, which means that the observed discrepancy between the two batches of scores is supposed to be minimal. The test determines whether this difference between the sample means large enough to reject Hu at a significance level. Specifically, we compare the p-value with the significance level a for each batch an uncorrected test, and a/2 for the j-th batch in a corrected test. The permutation test is conducted below:
(1) Calculate the observed mean difference A E b(z,)/K Σラ= o(2))/k of these two batches, and assume that
Ho is true;
(2) Combine these two sequences into one dataset, reshuffle the data, and divide it into two new groups This is the
permutation operation. Calculate the sampled absolute mean difference for the r-th permutation, Ā (m)) = |EE= ((z(")) )/kk
Σ (0g)") k/k]; )/kks
(5) Repeat step (2) for sufficient number of permutations (n 2, 000 in our test);
(6) Calculate the p-value, which is the proportion of permutations where Å > A, relative to the total number of permutations (2, 000). If the p-value is greater than the significance level we retain Ho: otherwise we reject Ho-
Proceed to the next batch if Ho retained in this batch test, and continue the above process until Ho is rejected or all data are tested.
In the experiment, we consider the composite hypothesis testing, which means the null hypothesis Hg laz Myl S e. we still use the above permutation test, it will become much easier for > Á (n) to hold, even when Hy is actually true. This would result in significantly higher FPRs. Thus, we only check p-values when the observed A exceeds the estimated c.
35

---

Online Detection of LLM-Generated Texts via Sequential Hypothesis Testing by Betting
Table Average values de used Scenario estimated by using the first texts each sequence. Every two columns starting from the third column represent the de values used for Hi and Ho for each test scenario. Specifically, for the third column, we get samples from XSum and the first 10 observed texts generated by Gemini-I 1.5-Flash
then calculate the maximum difference between 6(x;) and ((y;) for any 1 v i - 10, 1 2 j < 10. We double this maximum value to estimate de value, i.e., de = 2 (xax),suul((x)) - 6(3)]l The fourth column follows a similar calculation for detecting 2024 Olympic news with XSum texts, where (xxx) represents score of i-th text from XSum, ) denotes the score of the j-th text of 2024 Olympic news.
XSum, Olympic
XSum, Olympic
XSum, Olympic
Scoring Model
Score Function
Human, 1.5-Flash
Human, Human
Human, 1.5-Pro
Human, Human
Human, PaLM
Human. Human
Fast- DPtecttPT
10.3586
6.6788
8.1840
6.7517
13,0086
6.8456
DetectGPT
2.9396
2.6967
2.8232
2.7183
3,4076
2.7313
NPR
0.1680
0.1358
0.1464
0.1397
0.2178
0.1337
LRR
0.8620
0.6418
0.6572
0.6298
1.3115
0.6294
Neo-2.7
Logrank
1.8230
1.6746
1.5966
1.6676
2.1740
1.6807
Likelihood
2.7089
2.5370
2.4016
2.4975
3,3493
2.5529
Entropy
1.9123
1,9560
1.8840
1.9841
2.0712
1.9299
DNA-GPT 1.4570
1.5696
1.3530
1.5374
1.8097
1,642
RoBERTa-base
1,9004
T.I136
T.2745
1.1795
1.9993
1.0390
RoBERTa-larg
1.3443
0.6548
0.5857
0.5713
1.9435
0.6014
Fast- Gett-DeteCGPT
10.0337
7,4171
7.6691
7,4983
13,1693
7.6051
DetectGPT
3.5422
3.0998
3.3271
3,0398
3,8780
3.1375
NPR
0.3264
0.2326
0,2795
0.2346
0.4432
0.2270
LRR
1.0874
0.7253
0.8615
0.7283
1.6474
0.7234
Gemma-2B
Logrank
1.9435
1.6850
1.7377
1.6759
2.2468
1.6162
Likelihood
3.1277
2.6890
2.8156
2.6686
3,6366
2.5868
Entropy
2.3749
2.4105
2.3417
2.3869
2.6341
2.3166
DNA-GPT
1.4319
1.3889
1.3308
1.3798
1.7274
1.4199
RoBERTa-base
1,9343
T.I536
T.2867
T.T800
7.9993
0.9913
RoBERTa-large
1.3832
0.6636
0.5695
0.5501
1.9499
0.6488
The rejection time and FPRs that we plot are the average values across 1000 repeated runs under each significance level a. Permutation test is very time-consuming, because if we have samples for each group, then we will get m/k n/k batches, each batch need conduct the above steps.
As observed in Figure 7, the permutation tests without correction always have higher FPRs under Ho than that with corrected significance levels. This phenomenon aligns with the fact that without significance level adjustments, it is impossible to control the type-I error. Permutation tests with large batch sizes demonstrate relatively low FPRs, which are approximately equal to However, this seemingly excellent performance is due to the fact that the preset values are much larger than the actual absolute difference in mean scores between sequences of XSum texts and 2024 Olympic news. This discrepancy results in fewer or no v-value checks, thus sustaining Hg- Consequently, the FPRs are nearly identical across each significance level Permutation tests sensitive to the discrepancies between tends change more with smaller batch sizes due variation among scores of texts from the same source. Although permutation test can reject Ho after the first batch test, it consistently exhibit an FPR greater than a. Moreover, even if we set the value of based on a much larger sample size, rather than from estimates derived from a few points, there will still be variance between the A value calculated in batches and the preset c. As long as A calculated by a batch of samples is greater than the preset e, the permutation test is likely to have large FPR under Ho. Compared to fixed-time methods, our method can use parameter estimates based on just few points ensure faster rejection and lower FPRs. It can save time with high acuuracy especially when there is no prior knowledge of the threshold for composite hypotheses.
sequences, and the A value
H. Experimental Results of Detecting Texts from Three Domains
We also test on the dataset of Bao et al. (2023) to explore the influence of text domains on the detection result of our method. In this experiment, we only consider Scenario 1. We let x, be human-wriiii written text from three datasets following Mitchell et al. (2023), each chosen represent typical L.MMs application scenario. Specifically, we incorporate news articles sourced
36

---

Online Detection of L.Me-Gerrrated Texts via Sequential Hypothesis Testing by Betting
Roshive Rate (FPR)
False Rositive Rate (FPR)
False Positve Rate: (FPR)
0.8
0.8
0.8
0.8
0.6
0.6
Rositve
0.6
0.6
0.4
0.4-
0.4
0.0
0.0
0.0
0.0
0
100
200 300 400 s00
0.00
0.02
0.04
0.06 0.08 0.10
0
100
200
300 400 500
0.00
0.02 0.04 0.06 0.08 0.10 Significance Level (a)
Rejection Time (T)
Significance Level (o)
Rejaction Time (T)
Fast-DetectGPT
k=25 k=50
k=100
k=250
k=500
Fast-DetectGPT
k=25 k=50
k=100
k=250
k=500
(a) Comparisons between our method and the permutation test
(b) Comparisons between our method and the permutation test with correction, are 2024 Olympic news or news generated by Gemini- 1.5-Flash with human- auman-written scoring function used is Neo-2.7.
without correction, yı are 2024 Olympic news news generated by
Gemini-1.5-Flash with human human-written written text sampled from XSum.
my sampled from XSum. The
The scoring function used is Neo-2.7.
Positive Rate (FRR)
1.0
False Positvve Rate (FPg)
1.0
Positive Rate (FPR)
1.0
Falce Positve Rate (FPR)
1,0
0.8
0.8
0.8
0.8
0.6
0.6
0.6
0.2
0.2
0.2-
0.2
0.0
0.0
44069
0.0
0.0
100
200
300 400 500
0.00
0.02
0.04
0.08 0.08 0.10
100
200
300 400 500
0.00
0.02
0.04
0.06 0.08 0.10
Rejection Time (Y)
Significance Level (o)
Rejection Time (t)
Significance Level (a)
++ Fast DetectGPT
k=25 k=50
k=100
2-0
k-000
Fast DetecttPP
k=25 k=50
k=100
k=250
k-500
(c) Comparisons between our method and the permutation lest
(d) Comparisons between our method and the permutation test with correction, Y/ are 2024 Olympic news or news generated by Gemini-1 1.5--i-,,,PPr, with human-writter -writtn text sampled from XSum.
without correction, Y/ are 2024 Olympic news or news generated by
Gemini-1.3 1.5-Po, with human-wrtt written text sampled from XSum.
The scoring function used is Neo-2.7.
The scoring function used is Neo-2.7.
False Positvv Rate IFPA)
1.0
False Positive Rate (FPR)
1.0
False Positive Rate IFPA)
1.0
False Posttive Rate (FPR)
1.0
0.8
0.8
0.8
0.8
0.6-
0.4
0.4-
0.4
0.4
0.2
0.2
0.2-
0,2
0.0
0.0
0.0
0.0
200
300 400
500
0.00
0.04 0.08 08 0.10
100
200
400 500
0.04 0.06 0.b8 0.10 Significance Level (a)
0.02
Rejection Time (T)
Significance Level (o)
Rejection Time (T)
Fast Destecett
k=25 k-50
k=100
k-250
k=500
Fast Dest-DetectGPT
k=25 k=
k=100
k=250
k=500
(e) Comparisons between method and the permutation test without correction, yt are 2024 Olympic news or news generated by PaLM 2 with human-writh written text v, sampled from XSum. The scoring function used is Nco-2.7.
(I)) Comparisons between our method and the permutation test with correction, yı are 2024 Olympic news or news generated by PaLM with human-written text æ; sampled from XSum. The scoring function used is Nco-2.7.
Figure 7. Comparisons between our method and two baselines for detecting 2024 Olympic news and machine- machine-generated news. Fake generated by 3 source models: Gemini-1.5 1.55-Flshh, Gemini-1.5 and PaLM 2. The scoring model used is Neo-2.7 with the score function of Fast- DeteccGG.... We consider five batch sizes: 25, 50, 100, 250, 500. The left column displays results from the permutation test without correction, while the right column presents results of permutation test with corrected significance levels for each batch test.
37

---

Online Detection of L.M-Greeeatted Texts via Sequential Hypothesis Testing by Betting
from the XSum dataset (Narayan al., 2018), stories from Reddit WritingPrompts dataset (Fan al., 2018) and long-form answers written by human experts from the PubMedQA dataset (Jin al., 2019). Then, the capability of our algorithm is evaluated by detecting the source of texts y/ originated from the above source models or human datasets. Source models involved in this experiment are GPT-3 (Brown, 2020), ChatGPT (OpenAI, 2022), and GPT4 (Achiam et al., 2023) while the scoring model is Neo-2.7 (Black et al., 2021). The perturbation function for DetectGPT and NPR is T5-1 and the sampling model for Fast-DetectGPT is GPT-J-6B.
Figure 8a and Figure 8b present the averaged results when the two text streams are from the same domain and different domains, respectively- is, when both sequences are sampled from or prompted using the same dataset, or different datasets. Specifically, we compute the average rejection time and false positive rate (FPR) under each significance level a, aggregated over three LLMs (GPT-3, ChatGPT, GPT-4) and three domain configurations (three same-domain or three -imrrenn... A comparison of the two figures reveals that detection is faster when the prepared texts and the texts under evaluation originate from the same domain. For all score functions, our algorithm consistently controls FPRs below the target significance level a, and the corresponding average rejection times remain below 150. Among all functions, Fast-DetectGPT is the fastest to reject Ho, with average value of r being around 40 for same-domain texts and about 80 for different-domain texts.
False Positive Rate (PPR)
l alse Positive Rate (FPR)
0.10
False Positive Rate (FPR)
aass Positive Rate (FPR)
0.10
0.04
0.08
0.04
0.08
0.03
0.06
0.03
0.06-
0.02-
0.04
0.02
0.04
0.01-
0.02
0.01
0.02
0.00
0.00
30
60
90
120 150
0.00 0.02 0.04 0.06 0.08 0.10
30
60
90
120 150
0.00 0.02 0.04 0.06 0.08 0.10
Rejaction
Time (T)
Significance Level (a)
Rejaction
Time (T)
Significance Level (a)
Fast-Detect DDeeetGPP
NPR
LogRank Likellhood
Entropy DNA-GPT
RoB-base
FasttDetect DDeeecGPPT
NPR LRR
LogRank Likellhood
Entropy DNA-GPT RoB-large
mo8-base
LRR
RoB-large
(a) Averaged test results with texts x, and yı from the same domain across three source models and three text domains. The scoring model used is Neo-2.7.
(b) Averaged test results with texts and y from different domains across three source models and three text domains. The scoring model used is Neo-2.7.
Figure& 8. Average results of Scenario There are three source models: GPT-3, ChatGPT, and GPT-4, and three domains. For (a), two text sequences are both of XSum, Writing or PubMed datasct. For (b), two sequences are of XSum and Writing, XSum and PubMed, Writing and PubMed. Each sequence has 150 samples, which means the time budget is T 150. The left subfigure (a) and (b) shows the average rejection time under Hi versus. the averaged FPRs under Ho under each significance level a. Thus, plots closer to the left bottom corner are preferred, which indicate correct detection an L.M with shorter rejection times and lower FPRs. In the right subfigure of each panel, the black dashed line along with the shaded area illustrates the expected FPR, consistently maintained below the significance level a..
Specifically, according to Figure 10, when texts are both sampled from PubMedQA datasets, the behaviour of most score functions are worse than that for texts from XSum and WritingPrompts. Specifically, it costs more time for them to reject Ho. There are more vertical lines in Figure 9c, 10i and 9i, which means texts generated by GPT-4 are more challenging for our algorithm when using certain score functions such as RoBERTa- RaB/RTa-baseelarge to detect before T 150.
Figure 8b illustrates the performance of our algorithm with different score functions when detecting two streams of different- domain texts. All functions can guarantee FPRs below the a-significance level. When T, is from XSum and y, is generated by GPT-4 based on Writing, only Fast- DesttDetectGPT can declare the source of ye as an LLM before T = 150. Another interesting phenomenon is that when y/ is generated by GPT-3 based on PubMed, tests with most score functions fail to successfully identify its source as an LLM before the time budget expires, regardless of the domain of the human text used as x, for detection. Only the score functions of two supervised classifiers consistently reject Ho before 150, which shown as Figure 9g. 10d and 10g.
The parameter 9, is chosen from the range |1/2,,, 0j. The value of de for any t is equal to the maximum absolute difference between two sequences of scores for each test, as can be seen in Table 9 for the same-domain texts and Table 9 for the different-domaie texts. We get the absolute difference A between the scores Φ( and o(v)) for texts from the same domain and different domains involved in our experiments, as shown in the Table 11 and Table 12 respectively. Averaged values of across three source models (GPT-3, ChatGPT and GPT-4) when texts Fe and ye are from the same domain and different domains are presented in Table 7 and 8, respectively. In practice, we can also select the value of de and e based on the hint
38

---

Online Detection of L.M-Generated Texts via Sequential Hypothesis Testing by Betting
False Positive Rate (PPR)
0.10
Felr Posttive Rate (PPR)
False Positve Rate (FP
0.10
8.0
palze Positiv Pate
0.08
0.04
0.03
0.06
0.03
0.06
6.02-
0.01
0.02
0.01
0.02
0.00
0.00
0.00
0.00
DO
90
0.02 0.04 0.06 0.10
120
150
0.02 0.04 0.08 0.08 0.10 0.00
Rejection
Significance Level (e)
Rejoction [t]
Significance
LogRank
Entropy
-Rstt
LogRank
Entropy
DetectGPT
RoB-targe
DetectGPT
RoB-large
(a) Same domain: Iu is sampled from XSum, yt is from XSum generated by GPT-3 based on XSum.
(b) Same domain: 3y is sampled from XSum, is from XSum
Or generated by ChatGPT based on XSum.
Faise Positive Pate (PPR)
palte Positive Pate (PPR)
0.10
Febe Positive Rate (1PR)
False Positve Rate (FP1)
0.10
0.04-
0.08
0.04
0.03-
0.06
0.03
0.06
0.02
0.01
0.02
0.01
0.02
0.00
0.00
0.00
0.00
DO
90
0.02 0.04 0.56 0.10
120 150
0.00
0.02 0.04 0.68 0.08 0.10 Significance
Rejection
Significance Level (o) Entropy
Rejoction Itl
Fast-Detect
LogRank
FRstt
LogRank
Entropy
base RoB-large
RoB-targe
DetectGPT
(c) Same domain: J. is sampled from XSum, Yr is from XSum generated by GPT-4 hased on XSum.
(d) Same domain: Te is sampled from Writing, is from Writing generated by GPT-3 hased on Writing.
Felge Positive bete (698)
0.05-
palse Positve gato (aal
0.10-
Talge Posiive Rate (898)
Falce vositiv Bate (FPR)
0.10
0.04
0.08
0.04
0.03
0.06
0.03
0.06
0.02-
0.04
0.01
0.02
0.01
0.00
0.on
0.00
0.00
6O
90
0.0z 0.04 0.56 0.08 0.10 Significance Level (o)
60
120
0.00 0. 0.04 0.68 0.b8 o.0 Significance
Rejection
Rejection Time ITl
Fast-Detact
LogRank
Entropy
RoB-base RoB-large
Dotect
LogRank
Entropy
RaB-base
RoB-large
(c) Same domain: 3e is sampled from Writing, Yt is from Writing generated by ChatGPT hased on Writing.
(f) Same domain: I, is sampled from Writing, is from Writing generated by GPT-4 hased on Writing.
Falge Posilvve bate (198)
0.05-
Falos Pesélve aate (ppe)
0o110
Faloe Roselvv Bete (198)
Filse Pocitiv Rate (FRR
0.10
0.04
0.04
0.03
0.06
0.03
0.06
0.04
0.01
0.02
0.01
0.00
0.00
0.00
0.00
30
50 120
0.02 0.04 0.06 0:08 0.10 Significance Level (al
60
120
150
0.00
0.04 0.06 0.10
Rejection Time
Rejection
Time [t)
Significance Level (a)
Fast-Detect
NPR
LogRanik Likelioood
Entropy
RuB-bese lerge
Fest- DesttDetect
NPR
LogRank
Entropy
RoB- base RoB-large
(g) Same domain: is sampled from PubMed, is from PubMed generated GPT-3 based PubMed.
(h) Same domain: H, is sampled from PubMed, is from PubMed generated by ChatGPT based PubMed.
False Posiivv bate (P9R)
0.05
Pate Positive Pate (FPR)
0.100
0.04
0.08
0.03
0.06
6.02-
0.04
0.01
0.02
0.00
0.00
30
120 Rejection Time (T)
90
0.02 0.04 0.06 0.00 0.10 Significance Level (at
Dest-Detect
APP
LogRankc Likelhoodd
Entropy
RoO bes-- RoB-lorge
DetactGPT
LAR
(i) Same domain: Fe is sampled from PubMed. ye is from PubMed generated GPT-4 based PubMed.
Figure 9. Test results: mean rejection times (under Hz) and FPRs (under Hu) under each significance level a using 10 score functions, with texts z; and y, from the same domain. There are 3 source models: GPT-3, ChatGPT and GPT-4, Scoring model: Neo-2.7.
39

---

Online Detection of L.MMGenerated Texts via Sequential Hypothesis Testing by Betting
False Positive Rate (19R)
0.06
0.10
0.06
Ealce Rositve Date (PP
0.10
0.04
0.05-
kate Posibie Pate
0.06
0.08
Fabe Postblee Rgte
0.04
0.03
0.06
0.03
0.03
6.02-
0.01
0.02
0.02
0.00
0.00
0.00
90
0.02 0.04 0.06 Significance Level (e)
0.10
120
0.00
0.02 0.04 0.08 0.08 0.10 Significance
Rejection
Rejection itl
LogRankc
Entropy
Fast
LogRank
Entropy
RaB-base RoB-large
DetectGPT
RoB-aarge
DetectGPT
(a) Different domains: 31 is sampled from XSum, is from
(b) Different domains: my is sampled from XSum, y is from
Writing or generated by GPT-3 based on Writing.
Writing generated by ChatGPT based on Writing.
False Positive Rate (1PR)
0.06
0.10
Febe Posttive Rate (IPR)
0.06
Ealse Positve Date (FP1)
0.10
0.05-
Jelzo Positive Pate
0.08
0.05
0.04
0.06
0.04
0.06
0.03
0.03
6.02-
0.07
0.01
0.02
0.02
0.00
0.00
0.00
0.02 0.04 0.56 0.10
120
0.02 0.04 0.08 0.08 0.10 0.00
Rejection
Signiftcance Level (o)
Rejoction Itl
Significance
Fast-Detect
LogRank
Entropy
RoB-base RoB-targe
Fast
LogRank
Entropy
base RoB-large
(c) Different domains: 3y is sampled from XSum, is from
(d) Different domains: xy is sampled from XSum, Y1 is from PubMed generated by GPT-3 based on PubMed.
Writing generated by GPT-4 based on Writing.
Felge Positive Rete (1P8)
0.06-
False Positve sato gaa9
0.10-
0.06
Falce Positiv Bate (FPR
0.10
0.05-
0.08
0.05
0.04
0.06
0.04
0.06
0.03
0.03
0.02
0.04
0.02
0.01
0.02
0.00
0.on
0.00
0.00
90
0.02 0.04 0.56 0.08 0.10 Significance Level (o)
120 150
0.00 0.b2 0.04 0.68 0.b8 0.10 Significance
Rejection
Rejection Time ITl
Fast-Detact
LogRank
Entropy
RoB-base RoB-large
Fast
LogRank
Entropy
RaB-base
RoB-large
(c) Different domains: 3a is sampled from XSum, is from PubMed or generated by ChatGPT based on PubMed.
(f) Different domains: x, is sampled from XSum, Y1 is from PubMed generated by GPT-4 based on PubMed.
False Posilive Rate (1P8)
0.08
Falos Posélve aate (pp6)
0.10-
False Posiiive Rate (FPR)
0.08
Filse Pocitive Rate (FRR
0.10
0.00
0.06-
0.06
0.04
0.04
0.04
0.04
0.02
0.02
0.00
0.00
0.00
0.00
30
60
120
150
0.02 0.04 0.06 0:08 0.10 Significance Level (al
60
120
150
0.00 0.bz 0.04 0.06 0.08 o.10 Significance Level (a)
Rejection
Time
Rejection
Time [t)
Fast-Detect
NPR
LogRanik Likelihood
Entropy
RuB-bese lerge
Destt-etectt
NPR
LogRank
Entropy
RoB- base
RoB-large
(g) Different domains: Iv is sampled from Writing, is from PubMed generated GPT-3 based on PubMed.
(h) Different domains: is sampled from Writing, is from
PubMed generated by ChatGPT based PubMed.
False Positive Rate (FPR)
0.08
Falte Positive Pate (FPR)
0.110
0.06
0.08
0.06
0.04
0.04
0.02
0.02-
8.00
0.00
30
90
120 150
0.02 0.64 0.06 0.00 0.10 Significance Level (at
Rejection Time (T)
fast: Dest-Detect
APPR LRR
LogRanic Likelloood
Entropy
RoO bee--
DetactGPT
RoB RoB-lorge
(i) Different domains: is sampled from Writing, is from PubMed generated GPT-4 based on PubMed.
Figure 10. Test results: mean rejection times (under Hi) and FPRs (under Ho) under each significance level a using 10 score functions, with texts za and y, from different domains. There are source models: GPT-3, ChatGPT GPT-4. Scoring model: Neo-2.7.
40

---

Online Detection of LLM-Geeerateed Texts via Sequential Hypothesis Testing by Betting
of the bound d, by the previous observed samples.
Since we let equal to the actual A value of two sequences of human texts, which ensures FPRs of all score functions
for each significance level a remain below a. As we have mentioned previously, our algorithm can ensure a nonnegative supermartingale wealth under Ho and thus control the type-I error. Besides, the relative magnitude of A and de e would influence the rejection time under H1-
Table Average Values derived by using Neo-2.7 as the scoring model and various score functions texts from same domain across three source models (GPT-3, ChatGPT and GPT-4). Every two columns starting from the third column represent the average A values across three LLMs under H, and Hu in cach test scenario. For instance, the third column presents the average absolute difference in mean scores between 150 Xsum texts and 150 texts generated by LLMs based on XSum texts. The fourth column illustrates the average value between 150 XSum texts and 150 XSum texts. Values in Column "Human, Human" are also used to set value for tests.
XSum, XSum
Writing, Writing
PubMed, PubMed
Scoring Model
Score Function
Human, LLMs
Human, Human
Human, LLMs
Human,
Human, LLMs
Human, Human
Human
Past-DetectGPP
2.2235
0.0513
2.5690
0.0138
1.0891
0.0179
DetectGPT
0,4048
0.0562
0.4985
0.0121
0.2206
0.0061
NPR
0.0200
0.0025
0.0276
0.0001
0.0135
0.0018
LRR
0.0919
0.0106
0.1002
0.0047
0.0714
0.0075
Neo-2.7
Logrank
0,2710
0.0303
0.4082
0.0037
0.2707
0.0257
Likelihood
0,4384
0.0420
0.6518
0.0145
0.4604
0.0354
Entropy
0.1017
0.0344
0,2393
0.0086
0.2135
0.0243
DNA-GPT
0.1917
0.0317
0,2852
0.0232
0.5531
0.1456
RoBERTa-base
0,4585
"00012
0.2825
0.0186
0.1330
0.0143
RoBERTa-large
0,2165
0.0090
0.1387
0.0057
0.1088
0.0072
Table 8. Average Values of A derived by using different score functions to texts from the defferent domains across three source models (GPT-3, CharGPT and GPT-4). Neo-2.7 is the scoring model used for the first eight score functions. Every two columns starting from the third column represent the average A values across three LLMs under H1 and Hg in each test scenario. For instance, the third column presents the average absolute difference mean scores between 150 Xsum texts and 150 texts generated by LLMs based on Writing texts. The fourth column illustrates the average A value between 150 XSum texts and 150 Writing texts.
XSum, Writing
XSum, PubMed
Writing, PubMed
Scoring Model
Score Function
Human, LLMs
Human, Human
Human, LLMs
Human,
Human, LLMs
Human,
Human
Human
Fast-DetectGPT
2.2848
0.2841
0.7267
0.3624
1.0108
0.0783
DetectGPT
0.4828
0.0342
0.0786
0.2992
0.0630
0.2835
NPR
0.0246
0.0030
0.0115
0.0031
0.0145
0.0012
LRR
0.0428
0.0665
0.0572
0.0219
0.1094
0.0447
Neo-2.7
Logrank
0.1050
0,3149
0,2429
0,0278
0.5579
0.2872
Likelihood
0.1603
0.5025
0.4548
0.0081
0.9573
0.4969
Entropy
0.2071
0.4464
0.2778
0.1154
0.7242
0.5617
DNA-GPT
0.1301
0.1551
3.0447
2.4916
3.1997
2.6467
RoBERTa-base
0.3284
0.0459
0.3304
0.1974
0.2844
0.I514
RoBERTa-larg
0.1351
0.0073
0.1962
0.0874
0.1889
0.0801

---

Online Detection of LLM-Geeeratedd Texts via Sequential Hypothesis Testing by Betting
Table Values d, with assumption that we know range of Igcl = (<b(=)) - ( beforehand. Every two columns starting
from the third column represent the de values for each used under H1 and Ho in each test scenario.
Test1
Test2
Test3
Text Domain
Score Functions
Human, GPT-3
Human, Human
Human, ChatGPT
Human, Human
Human, GPT-4
Human, Human
Cast-DetectGPT
7,4812
6,0978
7.9321
5,8211
7.1870
5.9146
DetectGPT
2.7997
2.3369
2,4485
2,4753
2.0792
2.1686
NPR
0,2025
0,1469
0.1418
0.1449
0.1184
0.1248
LRR
0.5798
0.5798
0.6088
0.4623
0.5913
0.5412
Logrank
1.3841
1.4601
1.1775
1.0302
1,4912
1.4399
XSum, Xsum
Likelihood
2.1195
2.1755
1.8610
1.5937
2.2947
2.1624
Entropy
2.0369
1,9155
1.4522
L.3976
1.9421
1.8986
DNA-GPT
1.0847
0,9399
0.9219
0.8006
1.0386
0,9783
RoBERTa-base
0.9997
0,9977
0.9997
0.9977
0,9997
0.9977
RoBERTa-large
0.9970
0.8191
0.9944
0.3471
0.9862
0.8191
DeTeeeePTGTT
8.0598
6,5768
8,4128
5,9812
6.6396
6.5999
DetectGPT
3.5032
3.1609
3.2897
3.1502
2.7128
3.3487
NPR
0.2103
0.1500
0,1920
0.1356
0.1286
0.1502
LRR
0.6487
0,5483
0.7428
0,5115
0.5417
0,4754
Logrank
2.1419
1,4950
1,5963
1.4885
1.6212
1.4024
Writing, Writing
Likelihood
3,3565
2.0134
2.4280
1.8810
2.3618
2.0046
Entropy
2.8300
1.6976
1.6677
1.4732
2.0601
1.5292
DNA-GPT
1.2223
1,2879
1.1989
1.0239
1.3639
1.2719
RoBERTa base
0.9997
0,9997
0,9997
0,9997
0,9996
0,9997
RoBERTa-large
0.9992
0,9172
0.9456
0.5028
0.9172
0.9172
DetecttGTTTT
5.6200
4.7132
5,8150
4.8065
4.6136
4.6400
DetectGPT
2.0692
1.8394
2.1635
2.3610
1.5185
2.0677
NPR
0.1888
0.2020
0.1867
0.2185
0.2028
0.2173
LRR
0.6811
0.7180
0.7433
0.6000
0.8885
0.7180
Logrank
1.8434
2.4121
1.9131
1.7634
2.6000
2.3949
Pubmed, Pubmed
Likelihood
2.8480
3,4780
3.1426
2.8756
3,8419
3,5055
Entropy
2.0549
2.2002
2.0570
1.8376
2.4877
2.3156
DNA-GPT 5.1276
4.7710
4.3683
4.7319
4,9899
4.8031
RoBERTa-basee
0.3982
0.9962
0.9984
0.9933
0.5995
0.996.
RoBERTa- large
0.9688
0.8863
0.8860
0.8863
0.8702
0.8863
42

---

Online Detection of LLM-Generatedd Texts via Sequential Hypothesis Testing by Betting
Table Values de with assumption we know range [ge] = [Ø(())) - ( beforehand. Every two columns starting
from the third column represent the de values for each used under H1 and Ho in each test scenario.
Test1
Test2
Test3
Text Domain
Score Function
Human, GPT-3
Human, Human
Human, ChatGPT
Human, Human
Human, GPT-4
Human,
Human
Fast- DetectGPT
7.7260
6,2430
8.0558
6.0578
7.3268
6,1780
DetectGPT
3.1711
2.8288
2.8371
2.6976
2.5561
2.6884
NPR
0,2001
0.1398
0.1736
0.1571
0.1249
0.1352
LRR
0.5817
0.5355
0.6395
0.6469
0.4837
0.5354
Logrank
1.8576
1.6371
1.4423
1.7444
1.0620
1.5484
XSum, Writing
Likelihood
3.1515
2.2793
2.0097
2.3806
1.5890
2.3587
Entropy
2.6910
2,0533
18358
2,0545
1.5863
1.7798
DNA-GPT
1.0351
1.3620
0.8960
1.1271
0.7700
1.1035
RoBERTa-base
0.9997
0.9920
0.9997
0.9997
0.9996
0.9997
RoBERTa-large
0.9992
0,9172
0.9455
0.3471
0,2952
0.5028
Fast-DetectGPT
5.7796
5.2418
6.0477
5.5643
5.2864
5.5884
DetectGPT
1.9280
2.1249
2.2991
2.7521
2.0123
2.8121
NPR
0.1529
0.1813
0.1607
0.1745
0.1608
0.1609
LRR
0.7144
0.6422
0.6765
0.5376
0.6952
0.5247
Logrank
1.6718
2.2405
1.3408
1.6702
1.5756
XSum, Pubmed
Likelihood
2.4632
3.1031
2.3094
2.5604
2.2453
2.3964
Entropy
2.2528
2.3452
1.9397
1.6543
1.9994
1.8273
DNA-GPT
6.6172
6.2606
5.6482
6.0117
6,3382
6.0307
RoBERTa- base
0,9983
0.9964
0.9984
0.9976
0.9995
0.9976
RoBERTa-large
0.9688
0.8191
0.8610
0.8863
0,8703
0.8863
Fast-DelectGPT
4.9091
5,3870
6.3816
5.4647
5.8265
5.3231
DetectGPT
2.7326
2.9296
2.5926
3.0456
2.3285
3.1283
NPR
0.1546
0.1829
0.1609
0.1927
0.1733
0.1588
LRR
0.7289
0.6567
0.8526
0.7093
0,8070
0.6364
Logrank
1.7772
2.0524
2.0617
1.8547
1.8669
1.6617
Writing, Pubmed
Likelihood
2.6541
2,8230
3.1136
2.6684
3.0234
2.6871
Entropy
2.4478
2.5402
2.6542
2.3688
2.3984
2.2263
DNA-GPT
7.0524
6.6958
6.0130
6.3766
6.6221
6,3146
RoBERTa-base
0.9983
0.9964
0.9999
0.9955
0.9996
0.3996
RoBERTa-large
0.9688
0.9172
0.8610
0.8863
0.8703
0.8863
43

---

Online Detection of L.M-Generated Texts via Sequential Hypothesis Testing by Betting
Table 11. Values of A derived by using Neo-2.7 the scoring model for the first eight score functions score from the domain. There are three source models: GPT-3, ChatGPT and GPT-4. Every two columns starting from the third column represent the A values under H1 and Hu each test scenario.
Testl
Test2
Test3
Text Domain
Score Function
Human, GPT-3
Human, Human
Human, ChatGPT
Human, Human
Human, GPT-4
Human,
Human
Fast- East-DetectGPT
1.9598
0,0770
2.9471
0,0106
1.7638
0.0664
DetectGPT
0.5656
0.0729
0,4816
0.0114
0.1672
0.0844
NPR
0.0305
0.0031
0.0248
0.0006
0.0047
0.0037
LRR
0.0349
0.0159
0.1568
0.0032
0.0839
0.0127
Logrank
0. 0.1938
0.0455
0.3794
0.0077
0.2397
0.0378
XSum, Xsum
Likelihood
0.3488
0.0630
0.5960
0.0109
0.3704
0.0521
Entropy
0.0586
0.0517
0.1456
0.0117
0.1009
0.0400
DNA-GPT
0.1579
0.0476
0.2742
0.0287
0.1432
0.0188
RoBERTa-base
0.5939
0.0210
0.5946
0.0002
0.I871
0.0212
RoBERTa-large
0,3941
0.0133
0.2037
0.0001
0.0516
0.0135
PPst-DetectGPT
2.3432
0.0204
3.1805
0.0206
2.1831
0.0002
DetectGPT
0.5752
0.0181
0.6980
0.0093
0.2223
0,0088
NPR
0,0330
0.0001
0.0415
0.0001
0.0083
0.0001
LRR
0.0979
0.0062
0.1512
0.0009
0.0516
0.0070
Logrank
0.3737
0.0055
0.5407
0.0026
0.3103
0.0028
Writing, Writing
Likelihood
0.5915
0.0217
0.8511
0.0046
0.5126
0.0171
Entropy
0.2260
0.0130
0.3428
0.0036
0.1490
0.0093
DNA-GPT
0.2441
0.0348
0.3745
0.0155
0,2370
0.0193
RoBERTa-baal
0.6070
0.0279
0.2184
0.0093
0.0220
0.0186'
RoBERTa-large
0,3845
0.0085
0.0143
0.0033
0.0175
0.0052
Fast-DetectGPT
0.7397
0.0025
1.3683
0.0243
1.1594
0.0268
DetectGPT
0,2417
0.0092
0.2470
0.0051
0.1729
0.0041
NPR
0,0146
0,0015
0.0175
0,0012
0,0084
0.0028
LRR
0.0101
0.0070
0.1099
0.0042
0.0944
0.0112
Logrank
0.0282
0.0293
0.4115
0.0092
0.3725
0.0385
Pubmed, Pubmed
Likelihood
0.0771
0.0411
0.6922
0.0121
0.6119
0.0532
Entropy
0.0766
0.0316
0,2939
0.0048
0.2701
0.0364
DNA-GPT
0.3518
0.1309
0.8721
0.0875
0.4354
0.2184
RoBlRTa--aase
0.T839
0.0078
T..166
0.0137
0.0466
0.0275
la-
0.1318
0.0077
0.1294
0.0031
0.0652
0.0108
44

---

Online Detection of L.M-Generated Texts via Sequential Hypothesis Testing by Betting
Table 12. Values of A derived by using Neo-2.7 the scoring model for the first eight functions score from different domains. There are three source models: GPT-3, ChatGPT and GPT-4. Every two columns starting from the third column represent the A values under Hi Ho in each test scenario.
Testl
Test2
Test3
Text Domain
Score Functions
Human, GPT-3
Human, Human
Human, ChatGPT
Human, Human
Human, GPT-4
Human, Human
Fast- DeteDetetTGT
2.1206
0.2430
2.8602
0.2996
1.8737
0.3096
DetectGPT
0.6211
0.0278
0.6617
0.027
0.1657
0,0478
NPR
0.0323
0.0008
0.0377
0.0038
0.0039
0.0044
LRR
0.0391
0.0526
0.0757
0.0747
0.0137
0.0723
Logrank
0.0893
0.2899
0.2081
0.33
0.0175
0.3249
XSum, Writing
Likelihood
0.1362
0.4771
0.3281
0.5184
0.0166
0.5121
Entropy
0.1843
0.4233
0.1228
0,462
0.3142
0.4539
DNA-GPT
0.1279
0.1509
0.1953
0.1638
0.0672
0.1505
RoBERTa-base
0.6513
0.0063
0.2744
0.0633
0.0596
0.0562
RoBERTa-large
0.3789
0.0029
0.0253
0.0078
0.0011
0.0112
Fast-DetectGPT
0.4179
0.3244
0.9937
0.3989
0.7686
0.3639
DetectGPT
0.0098
0.2424
0.0723
0.3245
0.1538
0.3308
NPR
0.0148
0.0017
0.0133
0.0029
0.0064
0.0048
LRR
0.0214
0.0184
0.0868
0.0273
0.0633
0.0199
Logrank
0.0349
0.0226
0,3819
0.0388
0.312
0.0219
XSum, Pubmed
Likelihood
0.1195
0.0014
0.6838
0.0205
0,5612
0.0024
Entropy
0.0782
0.1232
0.4019
0.1032
0.3533
0.1197
DNA-GPT
2.8511
2.6302
3,2363
2,4517
3.0468
2.3930
RoBERTa-base
1.37
"..993
0.3591
0.2063
0.2609
0.I928
RoBERTa-large
0,2087
0.0846
0,2165
0.0902
0.1633
0.0873
Pa
0.6609
0.0813
1.2933
0.0993
1.0782
0.0543
DetectGPT
0.0376
0,2702
0.0453
0.2974
0.1060
0.2830
NPR
0.0156
0.0025
0.0170
0.0008
0.0108
0.0004
LRR
0.0312
0.0342
0.1614
0.0474
0.1356
0.0525
Logrank 0.3248
0.2673
0.7119
0.2912
0.6369
0,3030
Writing, Pubmed
Likelihood
0,5966
0,4785
1.2021
0.4978
1.0733
0,5145
Entropy
0,5015
0.5465
0.8638
0.5651
0.8072
0.5735
DNA-GPT
3.002
2.7812
3,4000
2.6155
3.1972
2.5435
RoBeRTa-basee
0.3548
0.1767
0.29.99
0.T4TO
0.2046
O.T366
RaBaRTa-large
0.2057
0.0817
0.2088
0.0825
0.1521
0.0761
I. Addressing Parameter Estimation Challenges and Mixed LLMs or Sources Tasks
Estimate Parameters Based on More Samples. Better parameter estimation can enhance the performance of our algorithm.
For instance, the previous experiments, we used the first 10 samples of each sequence estimate parameters d, and Here, give example show that using larger sample size for estimation could possibly yield better results. Specifically, using 20 samples from each sequence for estimation, with test begins the 21-th time step, could lead to improved algorithm performance.
Figure 11 demonstrates that when parameters are estimated with more samples from the initial time steps, all score functions maintain False Positive Rates (FPRs) below the specified significance levels. Additionally, almost all functions identify the LLM source more quickly compared to when fewer samples are used for estimation. This example indicates potential benefits of using more extensive data for parameter estimation in enhancing the effectiveness of our approach.
Tasks for Mixture of LLMs Sources. Our method be extended to additional tasks. Its fundamental goal in sequential hypothesis testing is to determine whether texts from an unknown source originate from the same distribution as those in a prepared human text dataset, where we consider mean value as the statistical metric.
Even if texts from the LLM source are produced by various LLMs, they still satisfy the alternative hypothesis, which means that our statistical guarantees remain valid and the algorithm could continue to perform effectively. The results are illustrated Figure 12a.
45

---

Online Detection of LLM-Generated Texts via Sequential Hypothesis Testing by Betting
Falso Postive Rate (FPR)
0.10
False Posttive Rote (FRR)
0.10
False False Posttive Rate (FPR)
0.10
False Posttive Rate (FRR)
0.10
0.08
0.08
0.08
0.08
0.06
0.06
0.06
0.06
0.04-
0.04
0.04
0.04
100 200 300 400 500 Rejection Time (t)
0.00 0.02 0.04 0.06 0.08 0.10
100 200 300 400 500 Rejection Time (1)
0.00 0.02 0.04 0.06 0.08 0.10
Significance Level (o)
Significance Level (o)
Fast-Detect -- NPR LRR
LogRank Likellhood
Entropy DNA-GPT
RoB-base
Fast-Detect =- NPR LRR
LogRank Likellhood
Entropy DNA-GPT
RoB-base
DetectGPT
RoB-large
DetectGPT
RoB-large
(a) Averaged test results estimating parameters based on first 10 samples.
(b) Averaged test results estimating parameters based on first 20 samples.
Figure Comparison different durations in the initial stage parameter estimation Scenario Here, text as sampled from XSum and di is from 2024 Olympic news machine-generated news, across three source models. The scoring model is Neo-2.7.
Subfigure (b) suggests that longer duration for parameter estimation may lead improved results. We emphasize that the begins t 21.
False Positive Rake (FPR)
False Positive Rate (FPR]
0.10
False Positive Rate (FPR)
False ivsitive Rate (FPR]
0.10
0.06
0.04-
0.02-
0.02
0.02
0.02
0.00
0.00
0.00
0.00
100 200 300 400 500 Rejection
0,00 0.02 0.04 0.06 0.08 0.10
0 100 200 300 400 500
0.00 0.02 0.04 0.06 0.08 0.10 Significance
(t)
Significance
Rejection (r)
Fast-Detect
NPR LAR
LogRenk Likelihaod
Entropy DNA-GPT
RoB-bese
Fast:Detect
NPR
LogRenk Likelihaod
Entropy DNA-GPT RoB laB-larrge
RoB-bese
DesectGPT
RoB-larn RoB-large
DelectGPT
LRR
(a) Task When the L.M LLMs (under 1)). Specifically, the sequence consists of 100 texts generated by Gemini-1.5-Pra 200 texts generated by Gemini-1.5. Flash, and 200 texts generated by PaLM 2.
posts texts generated by different
(h) Task 2: When the unknown source posts mixture of human- written texts and L.M--eeerated texts (under H1). Specifically, the sequence consists of 200 texts written by human, and 300 texts generated by PaLM 2.
Figure 12. (Extension to other scttings) (a) Results when the sequence of texts are produced by various LLMs instead of a single one. (b) Results under the setting that the null hypothesis corresponds to the that all the texts from the unknown source human-written. while the alternatvv hypothesis H, corresponds to the one that not all ys are human-written.
When the unknown source publishes both human-written and LLM-generated texts, our method can effectively address this scenario. Here, the null hypothesis assumes that all texts from the unknown source are human-written In contrast, the alternative hypothesis posits that not all texts are bumtn-written,, which indicates the presence of texts generated by LLMs. Figure 12b demonstrates that our algorithm, equipped with nearly all score functions, consistently performs well in this new context.
46


Reviews:
[{"review_id": "DTZzxUXOU9", "reviewer": "Reviewer_bKsA", "summary": "This work has studied an online detection method for AI-generated texts, and it can identify texts from unknown source models. The proposed method mainly makes use of sequential hypothesis testing and has an advantage of non-parametric property. Comparision experiments with several baseline methods (e.g. DetectGPT, Fast-Detect, LRR) shows the merits of the proposed method.\n\n## update after rebuttal. I appreciate the authors for the rebuttal response which partially addressed my concerns. However, I still feel it is not essential to the motivation/necessity of such sequential detection scenarios. Besides, I am not convinced by the comparison with some offline detectors, e.g. Binoculars, which has already performed well in terms of detection speed and accuracy. I would therefore maintain my rating.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for your careful reading and useful comments. Here are our responses. \n\n**(Necessity.)** Our emphasis is not on the frequency of adjustments, but rather on the fact that **most** offline detectors detect by comparing the text score with a **pre-determined** threshold, chosen via training and directly impacting accuracy. This applies to detectors like RoBERTa-Base/Large and ReMoDetect, which require supervised training and threshold-based classification (e.g., probability). While our method only uses an offline detector to score texts, **the threshold (i.e., the significance level $\\alpha$) is specified according to the user’s needs (the type-1 error allowed)**. \n\n**(Novelty.)** While prior work such as Raidar leverages hypothesis testing with log-probability scores in an offline setting, our approach is, to our knowledge, **the first to provide a general framework for adapting existing score-based detectors to the online setting**. Our key contribution is introducing an **any-time valid, non-parametric sequential testing framework** with **statistical guarantees** and **bounded expected detection time**, which is not available in existing offline detectors.\n\n**(Attack scenarios.)** Our work does not propose a new score function, which requires to be evaluated under attacks. Rather, we focus on how to use existing detectors in an online framework. Some have already addressed attacks: DetectGPT and Fast-DetectGPT consider paraphrasing attacks (Mitchell et al., 2023, Bao et al., 2023), and DNA-GPT studies revision attacks (Yang et al., 2023). While such attacks may reduce the gap between human and LLM scores, they typically do not eliminate the difference between them. Therefore, our basic hypothesis, that their population means are different, is reasonable.\n\n**(Distribution assumption.)** Lines 104–108: *\"... our approach is non-parametric, and hence ...\"*. That is, we make no assumption on the specific form of underlying distributions (e.g., Gaussian), which is standard for non-parametric methods (Balsubramani et al., 2015). Like most hypothesis testing methods, including non-parametric ones, we assume samples are drawn i.i.d. from unknown distributions (lines 141–148), without assuming their specific forms. \n\n**(Online vs. Real-time)** Scenarios are different. Some detectors predict **individual** samples efficiently, but they are still applied offline: each sample is scored and classified independently. In contrast, our online framework leverages sequential hypothesis testing, where samples are observed in a **streaming** fashion. Our goal is to obtain an anytime-valid level-$\\alpha$ test (i.e., controlling type-1 error when the source is human) while controlling the time to reject $H_0$ when the alternative is true (i.e., controlling the time to correctly identify the source as an LLM). To the best of our knowledge, this is **a novel online detection scenario of detecting LLMs, and therefore existing works for offline classification cannot be directly applied to our online scenario.** For example, a naive way to adapt the offline detector in our scenario is that if at a certain point, the offline detector predicts that a sample is written by LLM, then it declares the source is LLM (and hence stops). However, this naive method will have a false positive rate of 1 when the number of rounds becomes sufficiently large, unless the offline detector never makes an error in recognizing human text. \n\n**(Human samples.)** The reference human texts need not match the domain of the observed samples, because the hypothesis testing framework does not impose any assumptions on their distribution. As shown in Appendix H, our method works well even when the domains of the reference and target texts differ (XSum, WritingPrompts, PubMedQA).\n\n**(vs. Binoculars.)** Our method is not an offline detector, but rather a general online detection framework with strong theoretical guarantees. While offline detectors optimize metrics like AUROC, we focus on making sequential decisions with any-time valid statistical guarantees. In principle, our method could operate on top of Binoculars’ score function in an online manner, providing early stopping and statistical guarantees that Binoculars alone does not offer. Therefore, the goals of our method are complementary to existing detectors.\n\n**(Computational time.)** Our method is designed to offer an online framework to offline detectors. **The computational time will be that of the underlying offline detector plus a 1D online Newton update, which incurs very light overhead.**  The number of detection steps reflects the statistical notion of sample complexity, a key metric in sequential hypothesis testing. Our focus is on the statistical efficiency (i.e., how much texts are needed before making a decision with statistical guarantees), rather than raw computational time, which mainly depends on the score function used.", "claims_and_evidence": "Yes.", "methods_and_evaluation_criteria": "Yes, the idea of using hypothesis testing by betting makes sense for the online detection scenario.", "theoretical_claims": "Yes, I've checked two propositions (3.1 and 3.2).", "experimental_designs_or_analyses": "Yes, I have checked the experimental part which includes baselines and comparison results.", "relation_to_broader_scientific_literature": "Yes, this work proposes a relatively novel way to conduct online detection of AI-generated texts. It enjoys faster detection speed, and might be more suitable for detecting streaming data(e.g. from social media).", "essential_references_not_discussed": "NA.", "other_strengths_and_weaknesses": "Strengths:\n\nThe authors propose an online LLM-generated text detection method based on sequential hypothesis testing, capable of identifying texts generated by unknown-source LLMs. Additionally, this method is non-parametric and does not require assuming different prior distributions for human and LLM-generated texts.\n\nThe proposed algorithm is rigorously justified through extensive theoretical and experimental analyses. The content is comprehensive (though carefully reading nearly 50 pages is unrealistic, such thoroughness is necessary). The writing is well-structured, notation usage is correct, figures are visually appealing, and the construction process is easy to understand.\n\nFrom the perspective of evaluating the effectiveness of different detection algorithms, this paper is undoubtedly a well-founded and rigorous innovation. It parallels the Performance Profiles framework used in optimization research (https://arxiv.org/abs/cs/0102001), but currently, no theoretically guaranteed and objective evaluation methodology exists in AI-generated text detection. For example, evaluating detection algorithms based on rejection counts is an interesting idea.\n\nWeaknesses:\n\nIn the Introduction, the authors emphasize that existing score-based detection methods are sensitive to threshold selection (lines 036-038, right column). However, considering representative real-time methods like Binoculars and Fast-DetectGPT, threshold adjustments do not seem to be frequently required for achieving robust detection across different domains and source models. This is even more evident for trained zero-shot methods like RoBERTa-Base/Large and ReMoDetect (NeurIPS 2024). Therefore, this may not be a significant weakness of prior detection methods, and the authors might need to further justify the necessity of an online detection approach.\n\nThe paper appears to primarily discuss sequential hypothesis testing in the context of existing detection methods rather than developing a fundamentally new detection approach (i.e., the adaptation seems too direct). Many of the claimed advantages are inherent to sequential hypothesis testing (e.g., error control and ONS strategies). Considering that some existing detection methods have also applied hypothesis testing concepts (such as Raidar (ICLR 2024) and certain watermarking techniques), albeit with differences (e.g., sequential testing does not require a fixed sample size in advance), the novelty of this work seems somewhat limited.\n\nThe study lacks evaluation in traditional attack scenarios, such as text rewriting, paraphrasing, style transfer, and multilingual settings—an essential step for validating a new detector.\n\nIn the Introduction, the authors state that their method does not assume any underlying distribution for human or machine-generated texts (lines 104-108, left column). However, later in the paper (lines 141-148, left column), human and LLM-generated texts are assumed to originate from some distribution. This inconsistency may lead to misunderstandings.", "other_comments_or_suggestions": "No.", "questions_for_authors": "I have some questions regarding this work:\n\ni) The authors categorize detection methods as either offline or online, grouping all real-time, training-free detection methods under offline detection. Could the authors clarify the distinction between real-time detection methods (e.g., likelihood-based methods and Fast-DetectGPT) and their concept of online detection? Based on the authors’ analysis of the null hypothesis, the performance of online detection heavily relies on offline detection. If the score function from offline detection fails to effectively distinguish between human and LLM-generated texts, the effectiveness of hypothesis testing will also degrade. Given that offline detection methods can already be highly effective, why is online detection necessary? This issue is particularly relevant to the attack scenarios mentioned in Weaknesses point 3.\n\nii) What is the strategy for selecting the human reference sample x_t? The choice of reference samples directly affects the hypothesis testing results. The paper mainly considers news texts, but in real-world applications, how can we locate or curate suitable human text datasets to ensure that the test remains consistently effective?\n\niii) Binoculars (ICML 2024) achieves high AUROC while maintaining a low false positive rate (FPR), which is also one of the goals of the proposed method. How does this method compare against Binoculars to further highlight its advantages?\n\niv) The paper refers to \\textit{time}, which seems to indicate the number of detection steps. Could the authors also report the actual computational time required for detection?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "L1MSvPCvWk", "reviewer": "Reviewer_2DZ6", "summary": "The paper studies the problem of detecting whether a series of texts is LLM/machine-generated sequentially. To this end, they build on existing work on offline detection of LLM-generated text which proposes a variety of scoare functions, as well on recent work in sequential hypothesis testing. More concretely, they frame the problem as an sequential hypothesis testing problem where the null is that the expected score over the source’s distribution is equal to that over a human-written distribution (which they assume access to). They particularize the algorithm and analysis from Chugg et. al. 2023 to their setting, and subsequently enjoy a bound on the probability of false rejection (over all time) as well as a an upper bound on the expected stopping time of the method. They run experiments with a variety of score function specifications from prior literature and three source models that show promising empirical results in terms of detection speed and false discovery control for the proposed method, showing that testing by betting can be useful for this application of recent interest and relevance.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We first thank the reviewer for your positive feedback and helpful suggestions. Below are our responses.\n\n**(Assumption on the score function.)**\nWe agree that the assumption of the existence of a score function that produces distinguishable means for human-written texts and LLM-generated texts is critical. We describe this assumption as “mild” based on empirical observations. As discussed in Appendix G (Table 1) and Appendix H (Table11-12), we provide evidence showing that the empirical mean score differences produced by most of the adopted score functions are significant across multiple human-written text datasets and LLMs. Additionally, to address the case where the mean difference under $H_0$ is small but nonzero (i.e., when both sequences are human-written by different individuals), we have extended our theoretical analysis to the composite hypothesis setting (see Proposition 3.2). This formulation allows for a tolerance parameter $\\epsilon$ and leads to a more realistic criterion. Our experiments are all conducted in this setting and show good performance, which validates the effectiveness of our method under our assumption.\n\n**(Expected stopping time bound)** Regarding the upper bound on the expected stopping time, we acknowledge the reviewer’s concern that the bound might be loose when directly plugging in actual parameter values. We emphasize that this bound serves primarily as a worst-case guarantee. Nevertheless, our empirical results demonstrate that the algorithm often detects LLM-generated texts much earlier than the bound suggests in practice. Additionally, the bound also provides valuable insights into the stopping time by revealing which factors affect it and in what way, such as the mean score gap $\\Delta$, the bound estimate $d_*$, and the user-specified significance level $\\alpha$. \n\n**(Real world setting.)**\nWe would like to clarify that we have more experimental results considering the real-world applications in Appendix H for additional datasets such as WritingPrompts and PubMed. Furthermore, we have tested scenarios where the target sequence of texts is of a different domain/topic from that of the prepared human-written texts (e.g., Figure 8). Besides, We extend our method to scenarios where texts from the unknown source are produced by various LLMs (see Figure12(a)), and when the unknown source posts a mixture of human-written texts and LLM-generated texts (see Figure12(b)). The experimental results of the above scenarios show the effectiveness of our method.\n\n**(Overwhelming quantity.)** Thanks for your helpful suggestion! we will revise and streamline the appendix in future versions. \n\n**(One-sided hypotheses tests.)**\nWe thank the reviewer for this insightful comment. We think it is relatively straightforward to modify the underlying method for \nthe one-sided test that the reviewer kindly points out. For example, if the null hypothesis is $H_0: \\mu_x < \\mu_y$,  then we can specify the wealth dynamic as $W_t = W_{t-1} \\left( 1 - \\theta_t (\\phi(y_t) - \\phi(x_t)) \\right).$ Under the null $H_0$, we have\n$E[ W_t | F_{t-1} ] = W_{t-1} \\left( 1 - \\theta_t E[ \\phi(y_t) - \\phi(x_t)] \\right) = W_{t-1} \\left( 1 + \\theta_t (\\mu_x - \\mu_y ) \\right) \\leq W_{t-1},$ where we used the fact that $\\theta_t$ is $F_{t-1}$-measurable. This shows that the wealth process can be a non-negative supermartingale for this case. Therefore, we can apply Ville's inequality to show that this test is a valid-$\\alpha$ test. Furthermore, when the alternative $H_1$ is true, we can apply the online Newton method can help increase the wealth.\n\nOn the other hand, we would like to point out that using this one-sided test requires the user knows that the score of human-written text is higher (or lower) than the score of machine-generated text beforehand. The two-sided test does not require this assumption. Specifically, if the score function $\\phi(\\cdot)$ is not necessarily an affinity score (e.g., perplexity [1]), then the formulation in our paper might be more suitable.\n\nWe agree that different means for different sub-populations of humans are more realistic. While we are primarily concerned with the sequential testing scenario of humans vs. LLMs, we believe that enabling different sub-populations of humans and exploring potential application scenario is a valuable future direction.\n\n[1] Yongqiang Ma, Jiawei Liu, Fan Yi, Qikai Cheng, Yong Huang, Wei Lu, and Xiaozhong Liu. Ai vs. human–differentiation analysis of scientific content generation. arXiv preprint arXiv:2301.10416, 2023.\n\n**(Typos and other comments.)**\nWe thank the reviewer for carefully pointing out these typos and the figure issue. We have corrected these. Figure 1 will be replaced with a higher-resolution version to improve its visual quality. We appreciate your attention to detail.", "claims_and_evidence": "Yes the claims are generally supported by evidence. One note is that the assumption on the score function is repeatedly referred to as “mild” yet in my opinion it is not really obviously that mild, so maybe this point could be discussed more at length or be given more nuance. Additionally, maybe more care could be taken to analyze (or at least discuss) the predictiveness/usefulness of the upper bound on stopping time. For example, if one uses the actual, true constants of the bound and plugs in reasonable values for the parameters, I am worried the resulting bound would be quite large.", "methods_and_evaluation_criteria": "I think the experiments conducted are a good first step and useful for illustrating the abilities of testing by betting approaches on the application at hand. The authors consider relevant baselines, as well as multiple score specifications and different LLM models generating the text. Of course by only using two particular datasets of human text we only get very preliminary insight and much additional work would be needed to properly evaluate machine-generated detection methods in a real world, live setting.", "theoretical_claims": "The theoretical claims follow almost directly from prior work in sequential hypothesis testing by betting.", "experimental_designs_or_analyses": "I feel like the experimental design and analyses are intuitive and fitting for the application at hand. I appreciate the inclusion of both the alpha-spending baseline as well as the invalid baseline.", "relation_to_broader_scientific_literature": "The paper employs recent theoretical advancements in sequential hypothesis testing, most notably that of Chugg et. al. 2023 (but this is part of a flourishing line of work, for which the authors give a good list of references). The paper also ties to existing work on detecting machine-generating texts, by proposing a sequential approach, as well as by employing different scores proposed in various prior works.", "essential_references_not_discussed": "Not to my knowledge.", "other_strengths_and_weaknesses": "The core strength of this paper is that it applies the testing by betting framework to the very suitable and relevant application (detecting machine-generated text). This application in my opinion makes much more sense as an online decision problem rather than offline and therefore I think the paper is a good addition to the existing literature on detecting llm-generated text.\n\nThe assumption that the machine and human generated texts come i.i.d. from their respective distributions is quite significant and potentially too unrealistic for the scenario at hand. One other possible weakness is that it may make more sense to consider one-sided hypotheses tests and test for ‘>’ or ‘<‘ rather than $H_0: \\mu_x = \\mu_y$.", "other_comments_or_suggestions": "One suggestion I have is considering one-sided hypotheses tests. To me, it would make sense to consider scores for which the higher they are, the more likely it is for the source to be an LLM. In that case, considering the null hypothesis be $H_0: \\mu_y > \\mu_x + \\epsilon$ would be more fitting. I find the testing for equality or near equality to be a bit unrealistic, as, whatever the score, I would expect different means for different subpopulations of humans, and it may be easier to hope for a score that maintains ordering (i.e. is able to cluster humans to one side and machines to the other). It would also possibly require a bit of additional modification of the techniques from Chugg et. al. 2023 and the analysis.\n\nTypos and other small comments: \n- “is” should be “are” on line 73 right column I think\n- “guarantees” should be “guarantee” on line 229 left column\n- “perform” should be “performs” on line 369 left column\n- Figure 1 appears as too low quality for me so maybe that can be fixed?", "questions_for_authors": "My main question is related to the suggestion above — why would we have a nulll test for ‘=‘ rather than ‘>’/‘<‘? What are your thoughts on doing one-sided tests given the nature of this application?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cYIHzXBeld", "reviewer": "Reviewer_DHzx", "summary": "The paper focuses on developing an algorithm for online detection of texts generated by\nlarge language models (LLMs). The main contribution is an algorithm based on sequential\nhypothesis testing techniques, which allows for quick and accurate identification of LLM\u0002generated texts in a streaming setting. The algorithm leverages score functions from existing\noffline detection methods and uses a betting framework to accumulate evidence for or\nagainst the null hypothesis that the text source is human-written. The authors conduct\ncomprehensive experiments using various score functions and datasets to demonstrate the\neffectiveness of their method.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We first would like to thank the reviewer for the positive feedback and good suggestions. Here are our responses.\n\n**(Supplementary material.)**\nWe would like to clarify that the supplementary material includes a detailed README.md file, which provides a clear overview of the codebase, experiment organization, score functions used, and testing scenarios. \n\n**(Computational complexity.)**\nThank you for raising this point. It is unclear which specific “traditional methods” the reviewer is referring to, so we address both possible interpretations below.\n- If the reviewer is referring to offline detection methods (e.g., Fast-DetectGPT, LRR, etc.). We would like to clarify that our method is built on top of existing offline detectors, and is designed to complement rather than compete with them.\nOur focus is to build an online framework that adapts existing offline detectors to an online setting where texts are observed in a streaming fashion. In this case, the only additional computation introduced by our method is a 1D Online Newton Step (ONS) update. For example, if we use the score function of Fast-DetectGPT to compute the text score, **the computational time will be that of the underlying offline detector plus a 1D online Newton update, which incurs only light overhead.**\n- On the other hand, if the reviewer is referring to the fixed-time permutation test baseline that we include for comparison, we note that permutation-based methods typically involve multiple resampling and recomputation of the test statistic (details in Appendix G), which is often computationally heavier than our lightweight online updates. In our case, each round of detection only involves computing a single score and performing a 1D update, making it much more efficient in streaming settings.\n\n**(Partitions.)**\nWe would like to clarify that our method is built on top of existing offline detectors, and is designed to complement rather than compete with them. Thus, our method does not involve base-to-new partitioning, nor does it assume or require any particular structure in the score function.\nOur contribution lies in providing a general online detection framework that takes text scores evaluated by a chosen score function of an existing offline detector as input, and performs sequential hypothesis testing with rigorous statistical guarantees. Specifically, our method ensures type-I error control at level-$\\alpha$ and power of 1.\nWhile some of the offline detectors we incorporate, such as Fast-DetectGPT, may internally use base-to-new partitioning as part of their score computation, this is entirely orthogonal to our approach, which operates solely on the text scores.\nTherefore, the choice of how the score function is computed, including the number of partitions or any use of base-to-new structure, is not within the scope of our framework.\n\n**(FPR control for noisy score inputs.)**\nOur theoretical guarantee on controlling the type-I error (false positive rate) relies on **Ville's inequality** applied to a **non-negative supermartingale** wealth process. This guarantee holds **regardless of the variance or noisiness** of the score function outputs, as long as the null hypothesis holds, i.e., the expected difference in scores satisfies $\\mu_x=\\mu_y$ or $|\\mu_x - \\mu_y| \\leq \\epsilon$ under the composite null.\n\nSpecifically, as shown in Appendix E of our paper, the wealth process \n$W_t = \\prod_{i=1}^t (1 - g_i \\theta_i)$\nis a supermartingale under $H_0$ where the score dufference $ g_t = \\phi(x_t) - \\phi(y_t) $ and $ \\theta_t $ is the adaptive betting fraction. Applying randomized Ville’s inequality [1, 2] to this process ensures that \n$\\mathbb{P}(\\exists t: W_t \\geq 1/\\alpha \\text{ or } W_T \\geq Z/\\alpha) \\leq \\alpha,$\nwhere $Z\\sim\\text{Unif}(0,1).$ \n\nThe above property of wealth process guarantees control of type-I error at level $\\alpha$. While noisy scores may increase the rejection time under $H_1$, they do not compromise our theoretical type-I error guarantee under $H_0$.\n\n[1] Ville, J. Etude critique de la notion de collectif. Gauthier- Villars Paris, 1939.\n\n[2] Ramdas, A. and Manole, T. Randomized and exchangeable improvements of markov’s, chebyshev’s and chernoff’s inequalities. arXiv preprint arXiv:2304.02611, 2023.\n\n **(Generalization to other types of texts or different LLMs.)**\nWe would like to clarify that we also have more experimental results for additional datasets such as WritingPrompts (stories) and PubMed (long-form answers written by human experts) in Appendix H. Furthermore, we have tested scenarios where the target sequence of texts is of a different domain/topic from that of the prepared human-written texts (e.g., Figure 8). Besides, We extend our method to scenarios where texts from the unknown source are produced by various LLMs (see Figure12(a)), and when the unknown source posts a mixture of human-written texts and LLM-generated texts (see Figure12(b)). Our method consistently perform well in the above scenarios.", "claims_and_evidence": "The claims made in the paper, such as the ability to control the false positive rate and\nprovide an upper bound on the expected detection time, are supported by clear and\nconvincing evidence. The authors present theoretical propositions and empirical results from\nexperiments that validate these claims.", "methods_and_evaluation_criteria": "Methods: the use of sequential hypothesis testing and betting techniques is a novel\napproach to the online detection of LLM-generated texts.\nThe evaluation criteria, including the false positive rate and rejection time, are relevant\nmetrics for assessing the performance of the algorithm in a streaming setting. The authors\nalso consider composite hypotheses and provide a detailed analysis of the algorithm's\nperformance under different scenarios.", "theoretical_claims": "The paper includes several theoretical claims, including the control of the false positive rate\nand the upper bound on the expected detection time. The proofs for these claims are\nprovided in the appendix. The proofs appear to be correct and well-reasoned, with clear\nexplanations of the assumptions and the logical steps involved", "experimental_designs_or_analyses": "The authors use a variety of score functions and datasets to test the algorithm's\nperformance, which provides a comprehensive evaluation of its effectiveness. The\nexperiments are repeated multiple times to account for randomness, and the results are\naveraged to provide reliable estimates of the algorithm's performance. The analysis of the\nresults is thorough, and the authors provide clear explanations for the observed trends.", "relation_to_broader_scientific_literature": "None.", "essential_references_not_discussed": "None", "other_strengths_and_weaknesses": "Strengths:\nThe proposed method has practical applications in areas such as content moderation,\nacademic integrity, and social media analysis.\nWeaknesses:\nThe online optimization and betting framework may have higher computational complexity\ncompared to traditional methods, especially when dealing with large-scale datasets.", "other_comments_or_suggestions": "The paper discusses using pseudo base-to-new partitions to train the detector, but it's not\nclear whether the performance of the model is sensitive to the number of partitions. While\nthe ablation study shows performance increases with more partitions, there might be an\noptimal number beyond which performance plateaus or even deteriorates. A more detailed\nanalysis of this aspect could provide better guidance on how to choose the number of\npartitions (K) for practical applications.", "questions_for_authors": "1. How does the algorithm handle cases where the score function outputs are highly variable or noisy? Would this affect the algorithm's ability to control the false positive rate?\n2. The experiments in the paper are conducted on specific datasets and LLMs. How well does the method generalize to other types of texts or different LLMs? \n3. The paper suggests that the online optimization and betting framework may have higher computational complexity compared to traditional methods, particularly when handling large-scale datasets. Could the authors provide a more detailed discussion on the computational complexity of the proposed framework? \n4. Specifically, how does it scale with larger datasets, and are there any strategies for mitigating this complexity in practical applications?\n5. The paper mentions that performance improves with an increasing number of pseudo base-to-new partitions in the detector training. However, it remains unclear whether there exists an optimal number of partitions. Specifically, is there a point beyond which adding more partitions leads to diminishing returns, or could the performance even start to deteriorate? A more thorough analysis of this relationship would be helpful in understanding the balance between the number of partitions and the model’s effectiveness.\n6. Additionally, could the authors provide more detailed guidance on how to select the number of partitions (K) for practical applications? Understanding how to choose K in real-world scenarios would be beneficial, particularly in optimizing performance while avoiding unnecessary complexity or overfitting.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The authors present a method for online detection of LLM-generated text using a sequential hypothesis testing framework based on betting. The authors build offline score functions and apply a nonparametric sequential testing procedure to provide statistical guarantees. The authors also conduct extensive theoretical analysis and empirical evaluation across multiple datasets, score functions, and source models.

The methodology is interesting and the application to the streaming detection settings is compelling. The reviewers and I generally agree that the authors provide a rigorous theoretical framing, and a comprehensive empirical setup and experiments. Reviewers also appreciated the novelty of the online detection scenario and the robustness of the statistical guarantees. 

However, reviews and I share concerns about the overall novelty relative to existing sequential testing frameworks, as well as the limited evaluation against recent offline detectors like Binoculars. One reviewer also questioned the necessity of the proposed framework given the effectiveness of existing real-time detectors, though the rebuttal by the authors clarified that the proposed method offers anytime-valid statistical guarantees not present in those baselines.

Overall, while not all concerns were fully resolved, the paper presents a sound and well-executed framework that extends existing offline detectors into a meaningful online setting with rigorous guarantees. On this basis I recommend a weak accept for this paper, however I do recommend the authors address the issues raised by the reviewers and noted above in their revised manuscript.
Decision: Accept (poster)