Title: Zero-Shot Detection of LLM-Generated Text via Implicit Reward Model
Venue: NeurIPS 2025
Accepted: yes
Keywords: Large Language Model

LLM Generated Text Detection

Reward Model

Zero-Shot Detection
Abstract: Large language models (LLMs) have demonstrated remarkable capabilities across various tasks. However, their ability to generate human-like text has raised concerns about potential misuse. This underscores the need for reliable and effective methods to detect LLM-generated text. 
In this paper, we propose IRM, a novel zero-shot approach that leverages Implicit Reward Models for LLM-generated text detection. Such implicit reward models can be derived from publicly available instruction-tuned and base models. Previous reward-based method relies on preference construction and task-specific fine-tuning. In comparison, IRM requires neither preference collection nor additional training.
We evaluate IRM on the DetectRL benchmark and demonstrate that IRM can achieve superior detection performance, outperforms existing zero-shot and supervised methods in LLM-generated text detection.

Zero-Shot Detection of LLM-Generated Text via Implicit Reward Model

Abstract

Large language models (LLMs) have demonstrated remarkable capabilities across various tasks. However, their ability to generate human-like text has raised concerns about potential misuse. This underscores the need for reliable and effective methods to detect LLM-generated text. In this paper, we propose IRM, a novel zero-shot approach that leverages Implicit Reward Models for LLM-generated text detection. Such implicit reward models can be derived from publicly available instruction-tuned and base models. Previous reward-based method relies on preference construction and task-specific fine-tuning. In comparison, IRM requires neither preference collection nor additional training. We evaluate IRM on the DetectRL benchmark and demonstrate that IRM can achieve superior detection performance, outperforms existing zero-shot and supervised methods in LLM-generated text detection.

1 Introduction

Large Language Models (LLMs) have demonstrated remarkable capabilities across various tasks [1, 2]. In particular, their ability to follow instructions and generate human-like content has made them essential tools in various real-world applications. However, this trend raises concerns about the misuse of LLMs, such as generating fake news [3] and plagiarism [4]. Since LLM-generated text can be indistinguishable from human-written text [5], it is necessary to develop effective methods for detecting such content.

Detection methods can generally be categorized into two paradigms: supervised and zero-shot approaches. Supervised methods typically train a classifier using both human-written and LLMgenerated texts. However, such classifiers often fail to generalize well to texts generated by unseen LLMs [6, 7]. In contrast, zero-shot methods provide an alternative with more generalization ability and without the need for training [8]. These methods leverage various LLMs and exploit statistical classification metrics to assess how likely a text is to be generated by an LLM, such as the average log likelihood or the log-rank of each tokens within the detected text [9, 10, 5]. However, in real-world the source LLM of the detected text is often unknown, thus these zero-shot methods have to rely on a proxy LLM and suffer from a performance drop. This highlights the need to identify model-agnostic metrics for detection.

ReMoDetect [11] has recently proposed using the reward score-the output of a reward model that measures the extent to which a text aligns with human values-as a detection metric. The rationale is that powerful LLMs are typically trained to obtain high reward score, making this signal potentially model-agnostic. Specifically, ReMoDetect constructs a preference dataset from human and ChatGPT responses to the same prompt, and fine-tunes a pre-trained reward model for adaptation to the detection task. However, this fine-tuning inherits the drawbacks of supervised methods. As

5

0

5

0

50

100

150

200

250

300

350

Frequency

(a) RM-deberta-v3-large-v2

7.5

5.0

2.5

0.0

2.5

5.0

Reward Score

0

50

100

150

200

250

(b) ReMoDetect

600

400

200

0

200

Reward Score

0

200

400

600

800

1000

(c) IRM: Llama-3.2-1B family

Human-written

LLM-generated

shown in Figure 1 (b), when ReMoDetect is evaluated on texts generated by unseen LLMs, there remains substantial overlap between the two score distributions of human-written and LLM-generated texts, suggesting the limited generalization ability.

In this paper, we propose IRM, a novel zero-shot method that leverages implicit reward models for LLM-generated text detection. Such implicit reward models can be derived from publicly available instruction-tuned and base models. Given a detected text y , an instruction-tuned model π θ and a base model π ref, the implicit reward score of y is formulated as r ( y ) = log π θ ( y ) π ref ( y ) . This score serves as the detection metric in our method, where a higher score indicates that the input text is more likely to be generated by LLMs. This formulation is built on Direct Preference Optimization (DPO) [12], which provides theoretical support for parameterizing implicit reward models. Compared to ReMoDetect, IRM requires neither preference collection nor additional training. As shown in Figure 1 (c), the two distributions under IRM are more separable than those in (b), suggesting that our zero-shot metric transfers well to unseen LLMs.

We evaluate IRM on the DetectRL benchmark [13] and demonstrate that IRM is an effective zero-shot method for LLM-generated text detection. Notably, with Llama-3.2-1B family, IRM achieves an average score of 91.77%, surpassing previous zero-shot baselines such as Log-Likelihood (77.46%) and Log-Rank (77.42%), as well as Binoculars (87.67%), a method which also utilizes two LLMs during detection. Furthermore, IRM even outperforms the supervised reward-based method, ReMoDetect (85.86%). These results underscore IRM's potential as a effective zero-shot detector for real-world applications.

2 Preliminaries

The mainstream alignment approach for LLMs is Reinforcement Learning from Human Feedback (RLHF) [14, 15], which relies on a reward model as a proxy for human preferences. In this section, we first introduce the concept of reward modeling, followed by an overview of the RLHF process.

Reward modeling. Typically, the reward model is initialized from a pre-trained language model with a additional linear head that outputs a scalar value. It is trained on a preference dataset, where human annotators compare multiple responses to the same prompt and rank them based on quality such as helpfulness or harmlessness. Formally, the preference is modeled using the Bradley-Terry (BT) model [16]:

"This article explores the

domain of learning from

compressed..."

base model

instruction-tuned model

reward score

where r denotes the reward model, x is the prompt, and y 1 , y 2 are two candidate responses. Given a prompt-response pair ( x, y ) , the reward model outputs a scalar score r ( x, y ) , where a higher score indicates stronger alignment with human values.

RLHF. During RLHF, the learned reward model is used to provide feedback for optimizing the language model. The LLM is framed as a policy model π θ , which generates responses conditioned on input prompts. The policy is optimized to maximize its expected reward while remaining close to a reference model π ref, typically the original base LLM prior to alignment. This leads to the following optimization objective:

where x denotes the prompt, y the response, r the reward model, and β a hyper-parameter controlling the strength of KL regularization.

3 Method

We introduce IRM, a zero-shot detection method for LLM-generated text detection. We formalize the approach and show how it can be instantiated using publicly available instruction-tuned and base models.

Implicit reward model for detection. Direct Preference Optimization (DPO) [12] offers a closedform solution to the RLHF objective in Eq. 2:

where Z ( x ) = ∑ y π ref ( y | x ) exp ( 1 β r ( x, y ) ) is the partition function that normalizes the distribution. By rearranging this equation, we can express the reward as a function of the policy and reference models, yielding an implicit reward model:

For simplicity, we let x = ∅ and y be the detected text. In this case, the term β log Z ( x ) becomes a constant across all detected texts. Importantly, applying a linear transformation to the reward scores does not affect commonly used classification metrics such as AUROC and F 1 , as long as the threshold is adjusted accordingly. As a result, we define the detection metric as:

where L is the length of y and y i denotes the i -th token. DPO also provides a theoretical justification for the effectiveness of the implicit reward model. As shown in Eq. 4, given a reward model r ( x, y )

and a reference model π ref, one can analytically derive the optimal policy model π ∗ θ under the RLHF objective. Conversely, given a learned policy model π θ and a reference model π ref, one can recover a corresponding reward model such that π θ is optimal with respect to that reward under the same reference model. This bidirectional relationship ensures that the implicit reward model is consistent with the behavior of the policy model relative to the reference model. Therefore, if the policy model aligns well with human preferences, the derived implicit reward model is also expected to exhibit strong performance in tasks such as LLM-generated text detection.

Constructing IRM using instruction-tuned and base models. Typically, publicly available LLMs release both instruction-tuned model and its corresponding base model. We leverage the instructiontuned model as the policy model and the base model as the reference model. In this way, without any fine-tuning, we can obtain an implicit reward model for LLM-generated text detection. Given a detected text y , its reward score can be computed via Eq. 5. Moreover, the same formulation naturally extends to LLMs trained via iterative alignment. Suppose instruction model π T is obtained from the base model π 0 through T rounds of optimization: π 0 → π 1 → ... → π T -1 → π T . While the intermediate models π 1 , ..., π T -1 are typically not publicly released, we can still conceptually define the implicit reward at each step i by applying Eq. 4:

By summing the rewards over all iterations, we obtain the total reward:

Since x = ∅ , the term β ∑ T i =1 Z i ( x ) becomes a constant. As linear transformation does not affect the ranking of detected texts, the overall reward is determined by log π T ( y ) π 0 ( y ) . Therefore, given only the instruction-tuned and base models, we can construct an implicit reward model for detection without requiring access to intermediate checkpoints.

4 Experiments

4.1 Settings

Datasets. We conduct evaluations on a large benchmark, DetectRL [13], which covers various domains, multiple LLMs and diverse attacks scenarios in real-world settings. We follow the test setting of DetectRL in the evaluation of detection methods. The statistics of the benchmark are reported in Table 6 in Appendix A.1. Notably, the texts in DetectRL are mostly complete, unlike previous works where the texts are often truncated to prefixes of complete texts. This allows for a more realistic evaluation of detection methods, as texts are typically not truncated in real-world scenarios.

Models. We employ Gemma-2 [17], Llama-3.2, Gemma [18], Qwen-2 [19], and Qwen-2.5 [20] model families for implementation. These model families provide both publicly available base models and instruction-tuned models, which are essential for deriving the implicit reward model. Within these families, we focus on lightweight LLMs, such as Gemma-2-2B-it, and Llama-3.2-1B-Instruct, to ensure computational efficiency while maintaining competitive performance.

Baselines. We compare IRM with recent zero-shot methods, including Binoculars [21] and Lastde [22], as well as typical zero-shot baselines such as Log-Likelihood [9, 10], Rank [5], Log-Rank [5], LRR [23] and Fast-DetectGPT [24]. Since we employs reward score as detection metrics, we also include ReMoDetect [11] and its initialization, RM-Deberta-v3-large-v2, as well as reward models from RewardBench [25], such as GRM [26] as baselines. For practical consideration, we exclude methods that require additional LLMs to perturb the original text for metric computation, as the extra cost is too high in real-world settings.

Metrics. Follow the settings of DetectRL, we report AUROC and F 1 Score as the main evaluation metrics. AUROC is widely used for assessing zero-shot detection methods and F 1 Score provides a comprehensive evaluation of detector capabilities by balancing the Precision and Recall.

Implementation Details. All experiments are conducted on two NVIDIA RTX 4090 GPUs (24GB each). All model and datasets used in this paper are fully detailed and referenced in Table 7 in Appendix A.1.

4.2 Main Results

4.2.1 Zero-shot LLM-generated Text Detection

In Table 1, we present the overall performance of IRM and other zero-shot detection methods on the DetectRL benchmark. Detailed results for each sub-task are reported in Table 8, 9, 10 and 11 in Appendix A.2. With the Llama-3.2-1B family, IRM achieves the highest average score, improving the best result by 4.1%, from 87.67% to 91.77%. Using the Gemma-2-2B family, IRM also attains a competitive average score of 77.05%, slightly below the best result of 77.81%. All methods, except Fast-DetectGPT, perform better when using the Llama-3.2-1B family than with the Gemma-2-2B family, indicating that Llama-3.2-1B family is more suitable for these zero-shot detection approaches.

4.2.2 Reward-based LLM-generated Text Detection

In Table 2, we present the overall performance of IRM and other reward models on the DetectRL benchmark. With the Llama-3.2-1B family, IRM achieves the highest average score, improving the best result by 5.8%, from 85.86% to 91.77%. Notably, IRM outperforms ReMoDetect, a supervised reward model that fine-tunes a pre-trained reward model (RM-deberta-v3-large-v2) using task-specific data. In addition, IRM also outperforms competitive reward models from the RewardBench benchmark, such as GRM-Llama3.2-3B and GRM-gemma2-2B, when evaluated on DetectRL benchmark. This suggests a potential mismatch between reward models optimized for preference classification and the requirements of LLM-generated text detection, indicating that directly leveraging such models for detection may be suboptimal.

4.3 Analysis

Robustness to domains. As shown in Table 8, IRM outperforms other detection methods on 6 out of 8 evaluation metrics and achieves the best average performance across all domains. While its performance varies slightly across domains-for instance, achieving particularly strong results on the arXiv dataset and relatively lower scores on the Review dataset-it maintains overall superiority throughout. For example, using the Llama-3.2-1B family, IRM achieves an AUROC of 98.56% and

an F 1 Score of 95.02% on the arXiv dataset, and an AUROC of 97.03% and an F 1 Score of 91.65% on the Review dataset. One possible explanation is that positive reviews are tend to be more neutral or benign compared to negative reviews, making them more likely to receive higher reward scores. This subtle property of the data may slightly reduce the separation margin between human-written and LLM-generated texts. Still, IRM maintains strong performance overall, underscoring its effectiveness across diverse domains.

Robustness to LLMs. As shown in Table 9, IRM outperforms other detection methods on 7 out of 8 evaluation metrics and achieves the best average performance across all LLMs. While IRM demonstrates strong generalization across a wide range of generation models, its performance can be further enhanced when the instruction-tuned and base models used to construct the implicit reward model share a similar model-family background with the generation model. For example, when using the Gemma-2-2B family, IRM performs particularly well in detecting texts generated by PaLM-2-bison, achieving an AUROC of 88.65% and an F 1 Score of 79.41%. Similarly, with the Llama-3.2-1B family, IRM demonstrates outstanding performance in detecting Llama-2-70B's outputs, achieving an AUROC of 99.35% and an F 1 Score of 96.58%.

Robustness to attacks. As shown in Table 10, IRM outperforms other detection methods on 8 out of 10 evaluation metrics and achieves the highest average performance across all attacks applied to LLM-generated texts. For example, when using Llama-3.2-1B family, IRM achieves an average AUROC of 97.19% and an average F 1 Score of 92.01%. While IRM demonstrates strong robustness against attacks on LLM-generated texts, its performance is relatively lower but still competitive when attacks are applied to human-written texts, as shown in Table 11. For example, with the Llama-3.2-1B family, IRM achieves an average AUROC of 94.48% and an average F 1 Score of 90.78%, which are slightly below than the best results: 94.63% for AUROC and 91.82% for F 1 Score. We further investigate which attack type has the greatest impact on IRM's performance. As shown in Table 3, with the Llama-3.2-1B family, IRM is particularly affected by the polishing attack, where LLMs are prompted to improve human-written texts. This result is intuitive: since the polished texts are generated by LLMs and typically exhibit improved fluency or coherence, they tend to receive higher reward scores than their original, unpolished counterparts. As a result, the distinction between human-written and LLM-generated texts become less clear, leading to a moderate drop in detection performance for IRM.

Generalization of IRM. In the generalization task, following the DetectRL benchmark settings, we evaluate each sub-task using the optimal threshold from other sub-tasks and calculate the average F 1 Score across all sub-tasks. As shown in Table 1, IRM exhibits strong generalization capability

100

200

300

Text Length

0.2

0.3

0.4

0.5

0.6

0.7

(a) Gemma-2-2B family Train

100

200

300

Text Length

0.2

0.4

0.6

0.8

(b) Gemma-2-2B family Test

100

200

300

Text Length

0.2

0.4

0.6

0.8

(c) Llama-3.2-1B family Train

100

200

300

Text Length

0.2

0.4

0.6

0.8

(d) Llama-3.2-1B family T est

Log-Likelihood

Rank

Log-Rank

LRR

Fast-DetectGPT

Binoculars

IRM

Lastde

across different domains, LLMs and attacks. For instance, when using Llama-3.2-1B family, IRM achieves F 1 Score of 90.23%, 90.72% and 90.87% on domain, LLM and attack generalization tasks. These results suggest that the decision boundary of the implicit reward score is stable across various conditions, highlighting IRM's superior generalization ability.

Effectiveness of instruction-tuned models for detection methods. As prior works often utilize base models when implementing detection methods, we follow this practice and implement baseline methods using both Llama-3.2-1B and Gemma-2-2B. As shown in Figure 4, using instructiontuned models consistently improves the performance of 5 out of 6 baseline methods. For instance, when using Llama-3.2-1B-Instruct, the performance of Log-Likelihood improves from 66.44% to 77.46%. Similarly, with Gemma-2-2B-it, the performance increases from 65.74% to 72.53%. These results suggest that instruction-tuned models are generally more effective for implementing detection methods.

Impact of text length. In the length task, following the DetectRL benchmark settings, we analyze the impact of text length on IRM's performance. For the train sub-task, we use the optimal threshold derived from the dataset with a length interval of 160-180 and apply it to datasets with other length intervals. Conversely, for the test sub-task, the optimal threshold of each dataset is evaluated on the dataset with a 160-180 length interval. We refer to the dataset used to derive the threshold as the training dataset, and the dataset used to evaluate the threshold as the test dataset. As shown in Table 1, when using Llama-3.2-1B family, IRM achieves the highest F 1 Score of 82.34% and 83.34% on the train and test sub-tasks, respectively. However, when using Gemma-2-2B family, IRM's performance drops significantly compared to other methods. To further investigate the influence of text length, we present the detailed results of train and test sub-tasks in Figure 3. IRM exhibits a distinct performance pattern trend compared to the other methods. Specifically, IRM tends to perform better than the length interval of the training dataset is similar to that of the test dataset. For example, using Gemma-2-2B family, thresholds derived from texts with lengths of 180-260 words perform better on 160-180-word texts (Figure 3 (a)), while the threshold from texts with 160-180-word is more effective for texts of 80-140 words (Figure 3 (b)). Similarly, using Llama-3.2-1B family, thresholds derived from texts of 140-260 words perform better on 160-180-word texts (Figure 3 (c)), whereas the threshold from 160-180-word texts performs better on texts of 140-200 words (Figure 3 (d)). This phenomenon may

100

200

300

Text Length

80

60

40

20

0

(a) Gemma-2-2B family

100

200

300

Text Length

60

40

20

0

(b) Llama-3.2-1B family

IRM

be attributed to the way the implicit reward score is computed. As shown in Eq. 5, the reward is accumulated over all tokens, so the total reward varies with text length. Consequently, the reward distribution shifts across different length intervals, which affects the optimal threshold value. This trend is illustrated in Figure 4, where the optimal threshold of IRM changes with text length, showing an approximately linear relationship. Notably, for the Gemma-2-2B family, the optimal threshold fluctuates and deviates from the fitted red dashed line within the 160-220-word intervals. Besides, we can also observe that, with Gemma-2-2B family, the optimal threshold of IRM fluctuates and deviates from the red dash within 160-220-words intervals, which can explain the performance drop of IRM when using Gemma-2-2B family. Since this length range corresponds to the train and test dataset in the length task, such instability in threshold values may explains the observed performance drop of IRM when using the Gemma-2-2B family in this task.

Impact of model version and size. We further investigate the impact of model version and size on the performance of IRM by implementing it across various model families, as presented in Table 5. When using the Llama-3.2-1B family, IRM achieves the best average score of 91.77%, slightly higher than the 91.08% achieved with the larger Llama-3.2-3B family. A similar trend is observed in the Qwen2 and Qwen2.5 families: for example, IRM attains 73.24% with Qwen2.5-0.5B family, outperforming Qwen2.5-1.5B family (69.69%). Besides, IRM achieves 83.72% with the Gemma-2B family, compared to 77.05% with Gemma-2-2B family. These results suggest that larger or newer versions of a model do not always lead to better detection performance.

5 Related Works

LLM-generated text detection. Since generative language models have shown remarkable capability in text generation, detecting AI-generated texts has attracted significant research attention. Current research primarily focus on supervised methods and zero-shot methods for LLMs. Supervised methods involve collecting labeled data and training classifiers for distinguishing human-written and machine-generated texts [27-29, 11]. However, it has been shown that many supervised methods often suffer from overfitting to the training set [6, 7], limiting their generalization ability. In contrast, zero-shot methods have shown better generalization ability, making them a promising alternative. Current zero-shot methods are based on pretrained LLMs and utilize heuristic metrics to classify

LLM-generated texts and human-written texts. These metrics include likelihood [9, 10], entropy, rank, log-rank [5], probability curvature [8, 24, 30], etc. Additionally, perturbing texts and measuring the variation of metrics is a commonly used augmentation method for constructing better metrics [22, 31-33], but it incurs extra computational costs. Our method is a zero-shot method that leverages the characteristics of aligned LLMs and derives a metric from LLMs in an off-the-shelf manner, without the need of additional training. Additionally, our method does not require perturbing texts to compute the variation of metrics, avoiding extra computational costs.

Implicit reward model. In LLM alignment, the primary focus is on obtaining a well-aligned LLM, rather than a high-quality reward model. Thus, although DPO and other methods use implicit reward models to implement LLM alignment, the implicit reward models obtained after training are typically not reused. Recently, some research has leveraged implicit reward models to label process rewards for reasoning tasks [34]. Since our method uses the reward score as a metric for distinguishing human-written and LLM-generated texts, we propose leveraging implicit reward models to compute the reward score for texts.

6 Discussion and Conclusion

In this paper, we propose IRM, a novel method for zero-shot detection of LLM-generated text. IRM utilizes implicit reward models to compute reward scores as detection metrics. Experiments on the DetectRL benchmark demonstrate that IRM is an effective zero-shot method for LLM-generated text detection. Notably, with Llama-3.2-1B family, IRM achieves an average score of 91.77%, surpassing previous zero-shot baselines such as Log-Likelihood, Log-Rank and Binoculars. Furthermore, IRM even outperforms the supervised baseline ReMoDetect, showing that IRM remains highly effective without any task-specific adaptation. These results underscore IRM's potential as a effective zero-shot detector for real-world applications.

Limitations and future works. In our experiments, we focus exclusively on lightweight LLMs, excluding larger LLMs due to computational constraints. Similarly, we do not consider generative reward models as baselines, as they typically require larger LLMs and more computation resources. Besides, we restrict our analysis to deriving IRM from LLMs within the same model family. For cases where only instruction-tuned models are available, deriving corresponding IRM is not feasible. We leave how to effectively leverage LLMs from different model families for future work.

Societal impact. IRM contributes to the responsible use of LLMs by providing a zero-shot method for detecting LLM-generated text. This capability can enhance transparency and help mitigate misuse in high-stakes domains such as education and media. On the other hand, misclassifications may inadvertently suppress human-generated text, highlighting that need for continuous development more powerful detection methods.

Acknowledgments and Disclosure of Funding

We thank all the anonymous reviewers for their insightful and valuable feedback on this paper. This work is supported by the National Natural Science Foundation of China (Grant No.U21B2009).

References

Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners, 2020. URL https://arxiv.org/abs/2005.14165 .

OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao,

Mohammad Bavarian, Jeff Belgum, Irwan Bello, Jake Berdine, Gabriel Bernadett-Shapiro, Christopher Berner, Lenny Bogdonoff, Oleg Boiko, Madelaine Boyd, Anna-Luisa Brakman, Greg Brockman, Tim Brooks, Miles Brundage, Kevin Button, Trevor Cai, Rosie Campbell, Andrew Cann, Brittany Carey, Chelsea Carlson, Rory Carmichael, Brooke Chan, Che Chang, Fotis Chantzis, Derek Chen, Sully Chen, Ruby Chen, Jason Chen, Mark Chen, Ben Chess, Chester Cho, Casey Chu, Hyung Won Chung, Dave Cummings, Jeremiah Currier, Yunxing Dai, Cory Decareaux, Thomas Degry, Noah Deutsch, Damien Deville, Arka Dhar, David Dohan, Steve Dowling, Sheila Dunning, Adrien Ecoffet, Atty Eleti, Tyna Eloundou, David Farhi, Liam Fedus, Niko Felix, Simón Posada Fishman, Juston Forte, Isabella Fulford, Leo Gao, Elie Georges, Christian Gibson, Vik Goel, Tarun Gogineni, Gabriel Goh, Rapha Gontijo-Lopes, Jonathan Gordon, Morgan Grafstein, Scott Gray, Ryan Greene, Joshua Gross, Shixiang Shane Gu, Yufei Guo, Chris Hallacy, Jesse Han, Jeff Harris, Yuchen He, Mike Heaton, Johannes Heidecke, Chris Hesse, Alan Hickey, Wade Hickey, Peter Hoeschele, Brandon Houghton, Kenny Hsu, Shengli Hu, Xin Hu, Joost Huizinga, Shantanu Jain, Shawn Jain, Joanne Jang, Angela Jiang, Roger Jiang, Haozhun Jin, Denny Jin, Shino Jomoto, Billie Jonn, Heewoo Jun, Tomer Kaftan, Łukasz Kaiser, Ali Kamali, Ingmar Kanitscheider, Nitish Shirish Keskar, Tabarak Khan, Logan Kilpatrick, Jong Wook Kim, Christina Kim, Yongjik Kim, Jan Hendrik Kirchner, Jamie Kiros, Matt Knight, Daniel Kokotajlo, Łukasz Kondraciuk, Andrew Kondrich, Aris Konstantinidis, Kyle Kosic, Gretchen Krueger, Vishal Kuo, Michael Lampe, Ikai Lan, Teddy Lee, Jan Leike, Jade Leung, Daniel Levy, Chak Ming Li, Rachel Lim, Molly Lin, Stephanie Lin, Mateusz Litwin, Theresa Lopez, Ryan Lowe, Patricia Lue, Anna Makanju, Kim Malfacini, Sam Manning, Todor Markov, Yaniv Markovski, Bianca Martin, Katie Mayer, Andrew Mayne, Bob McGrew, Scott Mayer McKinney, Christine McLeavey, Paul McMillan, Jake McNeil, David Medina, Aalok Mehta, Jacob Menick, Luke Metz, Andrey Mishchenko, Pamela Mishkin, Vinnie Monaco, Evan Morikawa, Daniel Mossing, Tong Mu, Mira Murati, Oleg Murk, David Mély, Ashvin Nair, Reiichiro Nakano, Rajeev Nayak, Arvind Neelakantan, Richard Ngo, Hyeonwoo Noh, Long Ouyang, Cullen O'Keefe, Jakub Pachocki, Alex Paino, Joe Palermo, Ashley Pantuliano, Giambattista Parascandolo, Joel Parish, Emy Parparita, Alex Passos, Mikhail Pavlov, Andrew Peng, Adam Perelman, Filipe de Avila Belbute Peres, Michael Petrov, Henrique Ponde de Oliveira Pinto, Michael, Pokorny, Michelle Pokrass, Vitchyr H. Pong, Tolly Powell, Alethea Power, Boris Power, Elizabeth Proehl, Raul Puri, Alec Radford, Jack Rae, Aditya Ramesh, Cameron Raymond, Francis Real, Kendra Rimbach, Carl Ross, Bob Rotsted, Henri Roussez, Nick Ryder, Mario Saltarelli, Ted Sanders, Shibani Santurkar, Girish Sastry, Heather Schmidt, David Schnurr, John Schulman, Daniel Selsam, Kyla Sheppard, Toki Sherbakov, Jessica Shieh, Sarah Shoker, Pranav Shyam, Szymon Sidor, Eric Sigler, Maddie Simens, Jordan Sitkin, Katarina Slama, Ian Sohl, Benjamin Sokolowsky, Yang Song, Natalie Staudacher, Felipe Petroski Such, Natalie Summers, Ilya Sutskever, Jie Tang, Nikolas Tezak, Madeleine B. Thompson, Phil Tillet, Amin Tootoonchian, Elizabeth Tseng, Preston Tuggle, Nick Turley, Jerry Tworek, Juan Felipe Cerón Uribe, Andrea Vallone, Arun Vijayvergiya, Chelsea Voss, Carroll Wainwright, Justin Jay Wang, Alvin Wang, Ben Wang, Jonathan Ward, Jason Wei, CJ Weinmann, Akila Welihinda, Peter Welinder, Jiayi Weng, Lilian Weng, Matt Wiethoff, Dave Willner, Clemens Winter, Samuel Wolrich, Hannah Wong, Lauren Workman, Sherwin Wu, Jeff Wu, Michael Wu, Kai Xiao, Tao Xu, Sarah Yoo, Kevin Yu, Qiming Yuan, Wojciech Zaremba, Rowan Zellers, Chong Zhang, Marvin Zhang, Shengjia Zhao, Tianhao Zheng, Juntang Zhuang, William Zhuk, and Barret Zoph. Gpt-4 technical report, 2024. URL https://arxiv.org/abs/2303.08774 .

Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. Defending against neural fake news. Advances in neural information processing systems, 32, 2019.

Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S. Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, Erik Brynjolfsson, Shyamal Buch, Dallas Card, Rodrigo Castellon, Niladri Chatterji, Annie Chen, Kathleen Creel, Jared Quincy Davis, Dora Demszky, Chris Donahue, Moussa Doumbouya, Esin Durmus, Stefano Ermon, John Etchemendy, Kawin Ethayarajh, Li Fei-Fei, Chelsea Finn, Trevor Gale, Lauren Gillespie, Karan Goel, Noah Goodman, Shelby Grossman, Neel Guha, Tatsunori Hashimoto, Peter Henderson, John Hewitt, Daniel E. Ho, Jenny Hong, Kyle Hsu, Jing Huang, Thomas Icard, Saahil Jain, Dan Jurafsky, Pratyusha Kalluri, Siddharth Karamcheti, Geoff Keeling, Fereshte Khani, Omar Khattab, Pang Wei Koh, Mark Krass, Ranjay Krishna, Rohith Kuditipudi, Ananya Kumar, Faisal Ladhak, Mina Lee, Tony Lee, Jure Leskovec, Isabelle Levent, Xiang Lisa

Li, Xuechen Li, Tengyu Ma, Ali Malik, Christopher D. Manning, Suvir Mirchandani, Eric Mitchell, Zanele Munyikwa, Suraj Nair, Avanika Narayan, Deepak Narayanan, Ben Newman, Allen Nie, Juan Carlos Niebles, Hamed Nilforoshan, Julian Nyarko, Giray Ogut, Laurel Orr, Isabel Papadimitriou, Joon Sung Park, Chris Piech, Eva Portelance, Christopher Potts, Aditi Raghunathan, Rob Reich, Hongyu Ren, Frieda Rong, Yusuf Roohani, Camilo Ruiz, Jack Ryan, Christopher Ré, Dorsa Sadigh, Shiori Sagawa, Keshav Santhanam, Andy Shih, Krishnan Srinivasan, Alex Tamkin, Rohan Taori, Armin W. Thomas, Florian Tramèr, Rose E. Wang, William Wang, Bohan Wu, Jiajun Wu, Yuhuai Wu, Sang Michael Xie, Michihiro Yasunaga, Jiaxuan You, Matei Zaharia, Michael Zhang, Tianyi Zhang, Xikun Zhang, Yuhui Zhang, Lucia Zheng, Kaitlyn Zhou, and Percy Liang. On the opportunities and risks of foundation models, 2022. URL https://arxiv.org/abs/2108.07258 .

Sebastian Gehrmann, Hendrik Strobelt, and Alexander Rush. GLTR: Statistical detection and visualization of generated text. In Marta R. Costa-jussà and Enrique Alfonseca, editors, Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 111-116, Florence, Italy, July 2019. Association for Computational Linguistics. doi: 10.18653/v1/P19-3019. URL https://aclanthology.org/ P19-3019/ .

Anton Bakhtin, Sam Gross, Myle Ott, Yuntian Deng, Marc'Aurelio Ranzato, and Arthur Szlam. Real or fake? learning to discriminate machine from human generated text, 2019. URL https://arxiv.org/abs/1906.03351 .

Adaku Uchendu, Thai Le, Kai Shu, and Dongwon Lee. Authorship attribution for neural text generation. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu, editors, Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8384-8395, Online, November 2020. Association for Computational Linguistics. doi: 10.18653/ v1/2020.emnlp-main.673. URL https://aclanthology.org/2020.emnlp-main.673/ .

Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D. Manning, and Chelsea Finn. Detectgpt: zero-shot machine-generated text detection using probability curvature. In Proceedings of the 40th International Conference on Machine Learning, ICML'23. JMLR.org, 2023.

Tatsunori B. Hashimoto, Hugh Zhang, and Percy Liang. Unifying human and statistical evaluation for natural language generation. In Jill Burstein, Christy Doran, and Thamar Solorio, editors, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 1689-1701, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics. doi: 10.18653/v1/N19-1169. URL https://aclanthology.org/N19-1169/ .

Irene Solaiman, Miles Brundage, Jack Clark, Amanda Askell, Ariel Herbert-Voss, Jeff Wu, Alec Radford, Gretchen Krueger, Jong Wook Kim, Sarah Kreps, Miles McCain, Alex Newhouse, Jason Blazakis, Kris McGuffie, and Jasmine Wang. Release strategies and the social impacts of language models, 2019. URL https://arxiv.org/abs/1908.09203 .

Hyunseok Lee, Jihoon Tack, and Jinwoo Shin. Remodetect: Reward models recognize aligned LLM's generations. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https://openreview.net/forum?id=pW9Jwim918 .

Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https://openreview.net/forum?id=HPuSIXJaa9 .

Junchao Wu, Runzhe Zhan, Derek F. Wong, Shu Yang, Xinyi Yang, Yulin Yuan, and Lidia S. Chao. DetectRL: Benchmarking LLM-generated text detection in real-world scenarios. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024. URL https://openreview.net/forum?id=ZGMkOikEyv .

Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano. Learning to summarize with human feedback. Advances in neural information processing systems, 33:3008-3021, 2020.

Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730-27744, 2022.

Ralph Allan Bradley and Milton E. Terry. Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika, 39:324, 1952.

Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, Johan Ferret, Peter Liu, Pouya Tafti, Abe Friesen, Michelle Casbon, Sabela Ramos, Ravin Kumar, Charline Le Lan, Sammy Jerome, Anton Tsitsulin, Nino Vieillard, Piotr Stanczyk, Sertan Girgin, Nikola Momchev, Matt Hoffman, Shantanu Thakoor, Jean-Bastien Grill, Behnam Neyshabur, Olivier Bachem, Alanna Walton, Aliaksei Severyn, Alicia Parrish, Aliya Ahmad, Allen Hutchison, Alvin Abdagic, Amanda Carl, Amy Shen, Andy Brock, Andy Coenen, Anthony Laforge, Antonia Paterson, Ben Bastian, Bilal Piot, Bo Wu, Brandon Royal, Charlie Chen, Chintu Kumar, Chris Perry, Chris Welty, Christopher A. Choquette-Choo, Danila Sinopalnikov, David Weinberger, Dimple Vijaykumar, Dominika Rogozi´ nska, Dustin Herbison, Elisa Bandy, Emma Wang, Eric Noland, Erica Moreira, Evan Senter, Evgenii Eltyshev, Francesco Visin, Gabriel Rasskin, Gary Wei, Glenn Cameron, Gus Martins, Hadi Hashemi, Hanna Klimczak-Pluci´ nska, Harleen Batra, Harsh Dhand, Ivan Nardini, Jacinda Mein, Jack Zhou, James Svensson, Jeff Stanway, Jetha Chan, Jin Peng Zhou, Joana Carrasqueira, Joana Iljazi, Jocelyn Becker, Joe Fernandez, Joost van Amersfoort, Josh Gordon, Josh Lipschultz, Josh Newlan, Ju yeong Ji, Kareem Mohamed, Kartikeya Badola, Kat Black, Katie Millican, Keelin McDonell, Kelvin Nguyen, Kiranbir Sodhia, Kish Greene, Lars Lowe Sjoesund, Lauren Usui, Laurent Sifre, Lena Heuermann, Leticia Lago, Lilly McNealus, Livio Baldini Soares, Logan Kilpatrick, Lucas Dixon, Luciano Martins, Machel Reid, Manvinder Singh, Mark Iverson, Martin Görner, Mat Velloso, Mateo Wirth, Matt Davidow, Matt Miller, Matthew Rahtz, Matthew Watson, Meg Risdal, Mehran Kazemi, Michael Moynihan, Ming Zhang, Minsuk Kahng, Minwoo Park, Mofi Rahman, Mohit Khatwani, Natalie Dao, Nenshad Bardoliwalla, Nesh Devanathan, Neta Dumai, Nilay Chauhan, Oscar Wahltinez, Pankil Botarda, Parker Barnes, Paul Barham, Paul Michel, Pengchong Jin, Petko Georgiev, Phil Culliton, Pradeep Kuppala, Ramona Comanescu, Ramona Merhej, Reena Jana, Reza Ardeshir Rokni, Rishabh Agarwal, Ryan Mullins, Samaneh Saadat, Sara Mc Carthy, Sarah Cogan, Sarah Perrin, Sébastien M. R. Arnold, Sebastian Krause, Shengyang Dai, Shruti Garg, Shruti Sheth, Sue Ronstrom, Susan Chan, Timothy Jordan, Ting Yu, Tom Eccles, Tom Hennigan, Tomas Kocisky, Tulsee Doshi, Vihan Jain, Vikas Yadav, Vilobh Meshram, Vishal Dharmadhikari, Warren Barkley, Wei Wei, Wenming Ye, Woohyun Han, Woosuk Kwon, Xiang Xu, Zhe Shen, Zhitao Gong, Zichuan Wei, Victor Cotruta, Phoebe Kirk, Anand Rao, Minh Giang, Ludovic Peran, Tris Warkentin, Eli Collins, Joelle Barral, Zoubin Ghahramani, Raia Hadsell, D. Sculley, Jeanine Banks, Anca Dragan, Slav Petrov, Oriol Vinyals, Jeff Dean, Demis Hassabis, Koray Kavukcuoglu, Clement Farabet, Elena Buchatskaya, Sebastian Borgeaud, Noah Fiedel, Armand Joulin, Kathleen Kenealy, Robert Dadashi, and Alek Andreev. Gemma 2: Improving open language models at a practical size, 2024. URL https://arxiv.org/abs/2408.00118 .

Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, Pouya Tafti, Léonard Hussenot, Pier Giuseppe Sessa, Aakanksha Chowdhery, Adam Roberts, Aditya Barua, Alex Botev, Alex Castro-Ros, Ambrose Slone, Amélie Héliou, Andrea Tacchetti, Anna Bulanova, Antonia Paterson, Beth Tsai, Bobak Shahriari, Charline Le Lan, Christopher A. Choquette-Choo, Clément Crepy, Daniel Cer, Daphne Ippolito, David Reid, Elena Buchatskaya, Eric Ni, Eric Noland, Geng Yan, George Tucker, George-Christian Muraru, Grigory Rozhdestvenskiy, Henryk Michalewski, Ian Tenney, Ivan Grishchenko, Jacob Austin, James Keeling, Jane Labanowski, Jean-Baptiste Lespiau, Jeff Stanway, Jenny Brennan, Jeremy Chen, Johan Ferret, Justin Chiu, Justin Mao-Jones, Katherine Lee, Kathy Yu, Katie Millican, Lars Lowe Sjoesund, Lisa Lee, Lucas Dixon, Machel Reid, Maciej Mikuła, Mateo Wirth, Michael Sharman, Nikolai Chinaev, Nithum Thain, Olivier Bachem, Oscar Chang, Oscar Wahltinez, Paige Bailey, Paul Michel, Petko Yotov, Rahma Chaabouni, Ramona Comanescu, Reena Jana, Rohan Anil, Ross McIlroy, Ruibo Liu, Ryan Mullins, Samuel L Smith, Sebastian Borgeaud, Sertan Girgin, Sholto Douglas, Shree Pandya, Siamak Shakeri, Soham De, Ted Klimenko, Tom Hennigan, Vlad Feinberg, Wojciech

Stokowiec, Yu hui Chen, Zafarali Ahmed, Zhitao Gong, Tris Warkentin, Ludovic Peran, Minh Giang, Clément Farabet, Oriol Vinyals, Jeff Dean, Koray Kavukcuoglu, Demis Hassabis, Zoubin Ghahramani, Douglas Eck, Joelle Barral, Fernando Pereira, Eli Collins, Armand Joulin, Noah Fiedel, Evan Senter, Alek Andreev, and Kathleen Kenealy. Gemma: Open models based on gemini research and technology, 2024. URL https://arxiv.org/abs/2403.08295 .

An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jianxin Yang, Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng He, Junyang Lin, Kai Dang, Keming Lu, Keqin Chen, Kexin Yang, Mei Li, Mingfeng Xue, Na Ni, Pei Zhang, Peng Wang, Ru Peng, Rui Men, Ruize Gao, Runji Lin, Shijie Wang, Shuai Bai, Sinan Tan, Tianhang Zhu, Tianhao Li, Tianyu Liu, Wenbin Ge, Xiaodong Deng, Xiaohuan Zhou, Xingzhang Ren, Xinyu Zhang, Xipin Wei, Xuancheng Ren, Xuejing Liu, Yang Fan, Yang Yao, Yichang Zhang, Yu Wan, Yunfei Chu, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, Zhifang Guo, and Zhihao Fan. Qwen2 technical report, 2024. URL https://arxiv.org/abs/2407.10671 .

Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tianyi Tang, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report, 2025. URL https://arxiv.org/abs/2412.15115 .

Abhimanyu Hans, Avi Schwarzschild, Valeriia Cherepanova, Hamid Kazemi, Aniruddha Saha, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Spotting LLMs with binoculars: Zeroshot detection of machine-generated text. In Forty-first International Conference on Machine Learning, 2024. URL https://openreview.net/forum?id=axl3FAkpik .

Yihuai Xu, Yongwei Wang, Yifei Bi, Huangsen Cao, Zhouhan Lin, Yu Zhao, and Fei Wu. Training-free LLM-generated text detection by mining token probability sequences. In The Thirteenth International Conference on Learning Representations, 2025. URL https: //openreview.net/forum?id=vo4AHjowKi .

Jinyan Su, Terry Zhuo, Di Wang, and Preslav Nakov. DetectLLM: Leveraging log rank information for zero-shot detection of machine-generated text. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Findings of the Association for Computational Linguistics: EMNLP 2023, pages 12395-12412, Singapore, December 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023.findings-emnlp.827. URL https://aclanthology.org/ 2023.findings-emnlp.827/ .

Guangsheng Bao, Yanbin Zhao, Zhiyang Teng, Linyi Yang, and Yue Zhang. Fast-detectGPT: Efficient zero-shot detection of machine-generated text via conditional probability curvature. In The Twelfth International Conference on Learning Representations, 2024. URL https: //openreview.net/forum?id=Bpcgcr8E8Z .

Nathan Lambert, Valentina Pyatkin, Jacob Morrison, LJ Miranda, Bill Yuchen Lin, Khyathi Chandu, Nouha Dziri, Sachin Kumar, Tom Zick, Yejin Choi, Noah A. Smith, and Hannaneh Hajishirzi. Rewardbench: Evaluating reward models for language modeling, 2024. URL https://arxiv.org/abs/2403.13787 .

Rui Yang, Ruomeng Ding, Yong Lin, Huan Zhang, and Tong Zhang. Regularizing hidden states enables learning generalizable reward model for LLMs. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https://openreview. net/forum?id=jwh9MHEfmY .

Xiaomeng Hu, Pin-Yu Chen, and Tsung-Yi Ho. RADAR: Robust AI-text detection via adversarial learning. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https://openreview.net/forum?id=QGrkbaan79 .

Vivek Verma, Eve Fleisig, Nicholas Tomlin, and Dan Klein. Ghostbuster: Detecting text ghostwritten by large language models. In Kevin Duh, Helena Gomez, and Steven Bethard, editors, Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 1702-1717, Mexico City, Mexico, June 2024. Association for Computational Linguistics. doi: 10.18653/v1/2024.naacl-long.95. URL https://aclanthology.org/2024.naacl-long. 95/ .

Yang Xu, Yu Wang, Hao An, Zhichen Liu, and Yongyuan Li. Detecting subtle differences between human and model languages using spectrum of relative likelihood. In Yaser AlOnaizan, Mohit Bansal, and Yun-Nung Chen, editors, Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 10108-10121, Miami, Florida, USA, November 2024. Association for Computational Linguistics. doi: 10.18653/v1/2024. emnlp-main.564. URL https://aclanthology.org/2024.emnlp-main.564/ .

Guangsheng Bao, Yanbin Zhao, Juncai He, and Yue Zhang. Glimpse: Enabling whitebox methods to use proprietary models for zero-shot LLM-generated text detection. In The Thirteenth International Conference on Learning Representations, 2025. URL https: //openreview.net/forum?id=an3fugFA23 .

Xianjun Yang, Wei Cheng, Yue Wu, Linda Ruth Petzold, William Yang Wang, and Haifeng Chen. DNA-GPT: Divergent n-gram analysis for training-free detection of GPT-generated text. In The Twelfth International Conference on Learning Representations, 2024. URL https: //openreview.net/forum?id=Xlayxj2fWp .

Chengzhi Mao, Carl Vondrick, Hao Wang, and Junfeng Yang. Raidar: generative AI detection via rewriting. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=bQWE2UqXmf .

Shixuan Ma and Quan Wang. Zero-shot detection of LLM-generated text using token cohesiveness. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 17538-17553, Miami, Florida, USA, November 2024. Association for Computational Linguistics. doi: 10.18653/ v1/2024.emnlp-main.971. URL https://aclanthology.org/2024.emnlp-main.971/ .

Lifan Yuan, Wendi Li, Huayu Chen, Ganqu Cui, Ning Ding, Kaiyan Zhang, Bowen Zhou, Zhiyuan Liu, and Hao Peng. Free process rewards without process labels, 2024. URL https: //arxiv.org/abs/2412.01981 .

Liam Dugan, Alyssa Hwang, Filip Trhlík, Andrew Zhu, Josh Magnus Ludan, Hainiu Xu, Daphne Ippolito, and Chris Callison-Burch. RAID: A shared benchmark for robust evaluation of machine-generated text detectors. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12463-12492, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10.18653/v1/2024.acl-long.674. URL https://aclanthology.org/2024.acl-long.674/ .

Zhihui Chen, Kai He, Yucheng Huang, Yunxiao Zhu, and Mengling Feng. Divscore: Zero-shot detection of llm-generated text in specialized domains, 2025. URL https://arxiv.org/ abs/2506.06705 .

Chris Yuhao Liu, Liang Zeng, Yuzhen Xiao, Jujie He, Jiacai Liu, Chaojie Wang, Rui Yan, Wei Shen, Fuxiang Zhang, Jiacheng Xu, Yang Liu, and Yahui Zhou. Skywork-reward-v2: Scaling preference data curation via human-ai synergy, 2025. URL https://arxiv.org/abs/2507. 01352 .

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: See Abstract and section 1.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: See Section 6.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [NA]

Justification:

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: See section 4.1.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: We provide codes and data with instruction files in supplemental materials. Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: See appendix A.1.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [No]

Justification: Our method is deterministic and does not involve stochastic components such as random initialization, sampling, or data splits. Therefore, statistical significance measures or error bars are not applicable.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: See section 4.1.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We confirm our paper with the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: See section 6.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: We do not release any data or models in this paper, and thus no specific safeguards are required.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We have clearly cite all the existing assets we used.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: We do not release new asset.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: None.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: None.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

16. Declaration of LLM usage

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.

Answer: [Yes]

Justification: Our proposed method IRM leverages publicly available LLMs, including instruction-tuned and base models, to construct an implicit reward model for LLM-generated text detection. These models are integral to the core methodology and are clearly described in the main text.

Guidelines:

The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.

Please refer to our LLM policy ( https://neurips.cc/Conferences/2025/LLM ) for what should or should not be described.

A Details of Main Experiments

A.1 Details of Models and Datasets

A.2 Detailed Results of Main Results

B Additional Experiments

B.1 Additional Benchmarks

Settings. We further evaluate our method on the RAID [35] and DivScore [36] benchmarks. We adopt LLaMA-3.2-1B as the backbone detector, which demonstrated strong performance on the DetectRL benchmark. For the RAID benchmark, due to the large scale, we randomly sample 512 annotated pairs from the training split for each source model. We report AUROC as the main evaluation metric. For the DivScore benchmark, we use the official test split for evaluation and report both AUROC and F 1 Score. Since the lengths of test samples vary considerably, we compute the mean of implicit rewards across all tokens, instead of the sum (Eq. 5), to normalize the decision threshold, which empirically shows an approximately linear relationship with text length (see Figure 4).

Results. As shown in Table 12 and Table 13, IRM performs competitively across a wide range of models, supporting the generalizability of our approach.

B.2 Additional Baselines

Settings. We further include Skywork-Reward-V2 [37] models as additional baselines. These reward models are trained on large-scale preference data and use backbones up to 8B parameters. They achieve SOTA on the RewardBench benchmark, providing a strong reference to evaluate whether reward model scaling enhances generalization in LLM-generated text detection.

Results. As shown in Table 14, IRM consistently outperforms the reward-model-based methods, even when using the same or smaller backbones. This further supports our claim that standard preference-trained reward models, despite their size and strong performance on preference classification, do not generalize well to the LLM-generated text detection task. In contrast, IRM's implicit reward modeling is inherently aligned with this detection objective, leading to robust performance across different settings.

Algorithm 1 Inference pipeline of IRM

Inputs: policy model π θ , reference model π ref, detected text x , decision threshold t . Output: authenticity label y ∈ { human-written , LLM-generated }

1: p ref ← π ref ( x )

2: p θ ← π θ ( x )

3: r ← log p θ -log p ref

4: if r < t then

5: y ← human-written

6: else

7: y ← LLM-generated

8: end if

9: return y

▷ Compute implicit reward score

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel, lightweight approach to detect AI-generated texts. The proposed method leverages the implicit reward model constructed from the relation between internal representations of base and instruction-tuned versions of an LLM theoretically backed by the properties of RLHF finetuning. Using it, authors develop a detector of artificial texts. Conducted experiments show its high performance against various generating models and adversarial attacks on different text domains (styles/genres).\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n- An interesting approach leveraging the RLHF-nature of the common model instruction fine-tuning scheme. Proposed method achieves noticeable improvements over competitive solutions and is relatively easy to deploy.\n\n- Experiments are quite extensive, showing the robustness of the proposed AI-text detector to domain shift and various adversarial attacks. Evaluation also covers some less common attacks (e.g., translation of a text generated in another language).\n\n- The paper is well-structured and processing in a logically coherent manner.\n\n\nWeaknesses:\n\n- My main concerns with this paper are particular limitations of the evaluation of the proposed method. First of all, only one dataset is used - DetectRL - and it covers models primarily from the first half/mid-2023. Some more modern models (e.g., GPT4-o or something else) would be appreciated.\nSecondly, evaluation metrics. In practice, type I and type II errors (false positive and false negative) have different weights. None of the metrics provided for the experiments (AUROC and $F_1$) cover this aspect; I would suggest providing FPR/FNR (or any other appropriate metric). \nAlso, I would like to ask if it is possible to include metrics for baselines in Table 3?\n\n- On another note, a clearly outlined pipeline for application of the proposed method to a text (possibly, an example) in the paper (appendices) would be much appreciated.\n\nQUESTIONS:\nSee Weaknesses.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

comments: SUMMARY:
This paper introduces a new framework Implicit Reward Model (IRM), a training-free LLM-generated text (LGT) detector which is motivated by prior work ReMoDetect. The method utilized implicit reward implied by Direct Preference Optimization (DPO). Experiments conducted on detection benchmark (DetectRL) show that IRM significantly outperforms other baselines. Also, the paper shows the robustness of IRM against the domain, LLM, attack, and text-length shifts.

STRENGTHS AND WEAKNESSES:
[S1] Great detection performance and generalization: IRM consistently outperform baselines. 

[S2] No further training needed: The detector don’t need further training which significantly 

[S3] Extensive experiments: Contains extensive evaluation across various benchmarks and baselines. 

[S4] Insightful Analysis: The paper quantifies threshold drift with length, explores model version and size effects.

[W1] Didn't analyzed about detecting reasoning models: DetectRL doesn’t contains recent reasoning models’ responses such as openai o3 or deepseek-r1.

QUESTIONS:
[Q1] Detecting recent reasoning models: Have you evaluated IRM and other baselines with outputs from reasoning models such as OpenAI o3 or Deepseek-R1?

[Q2] Can you provide evaluation results on detecting randomly sampled generations by adjusting temperatures?

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 4
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-02

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces IRM, a zero-shot text detector. It scores a text via the DPO reward model parameterization, i.e. the log-ratio between an instruction-tuned model and its corresponding base model.\n\nSTRENGTHS AND WEAKNESSES:\n- The method has a simple implementation and yields significant gains. It requires no extra data, fine-tuning, or text perturbation. The motivation is clear: machine-generated text typically overoptimizes reward during RLHF, so reward can be a criterion for distinguishing btw human and machine text.\n- The paper provides thorough ablations on text length and robustness.\n- The motivation for specifically focusing on implicit reward models is unclear to me. Aside from DPO-style reward models, you could use any available reward model for this purpose (sequence classifier or even generative reward models). There are a bunch in [1]. Do you have a reason for focusing on implicit reward models specifically?\n\n[1] https://huggingface.co/spaces/allenai/reward-bench\n\nQUESTIONS:\n- On Table 5, as the paper states, \"larger or newer versions of a model do not always lead to better detection performance.\" Do you have any interpretation of why this is the case? Intuitively, it seems that the implicit reward model from a larger backbone should be better at estimating reward, which should, in turn, improve its ability to recognize \"overoptimized text.\"\n- Have you tried larger models, such as 7b models or beyond? I'd be interested to see whether the scaling trends continue to hold.\n- Could you try the same method with other types of reward models based on the same backbones (without further fine-tuning)? For example, https://huggingface.co/Ray2333/GRM-gemma2-2B-rewardmodel-ft this model uses the gemma2-2b backbone.\n\nLIMITATIONS:\n- Tests on quite small models.\n- Motivation for considering implicit reward models is unclear.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes IRM, a zero-shot LLM text detector based on implicit reward models. It uses instruction-tuned and base model pairs to compute their log likelihood ratio, which serves as the detector signal. These implicit reward models have the benefit of not needing to train an explicit reward model for the task of LLM detection, and instead directly use pre-trained instruction-tuned and base model pairs, which are generally readily available. Experiments show improved performance compared to other zero-shot LLM detection methods, and directly supervised methods, like the ones that train a direct reward model.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n- the paper is well-written and easy to follow \n- the method is novel for the text detection scenario. I do not believe any other method has used the log likelihood ratio used in RLHF as a signal for LLM text detection\n- Experiments are fairly extensive, covering multiple domains, models, attacks on detectors (such as rewriting). \n- A study on the impact of text length is also provided, which has not been extensively reported in prior works, and is helpful to know the limitations of the method\n\nWeaknesses\n- Intuitively, if text comes from the preference dataset, shouldn't they have high implicit reward, even if written by a human, since the preference dataset should align more with what humans would write? This seems to contradict the intuition provided in the paper (Figure 1, right side). I am curious how this figure would look if we compared the implicit reward distribution between preference data, non-preference data, for both human and LLM versions. While any detector signal (such as implicit reward) can in theory be used, it would be great to clarify the intuition behind the implicit reward. \n- An additional benchmark dataset would help strengthen the paper, such as RAID (https://arxiv.org/abs/2405.07940). While DetectRL covers many scenarios, it would help support the papers claims if at least one other standard benchmark dataset suite is included. \n- It would be helpful to understand the effect of LLM size on the detector performance, similar to https://arxiv.org/pdf/2305.09859 for DetectGPT. This would help users understand which instruction-tuned/base model policy pairs and their sizes are the most \"universal\" across LLMs and datasets..\n- Is there a fully-supervised classifier baseline, such as RoBERTa? The fully supervised models seem trained for reward modeling, but I wonder how a baseline that directly tries to classify human/LLM text would compare.\n\nQUESTIONS:\nSee weaknesses.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}