Fine-Grained Machine-Generated Text Detection

## Abstract

Abstract \setcode utf8 * * footnotetext: Equal contribution.

## 1 Introduction

The development of advanced large language models (LLMs), such as GPT-4, Claude-3.5, Gemini-1.5, Llama-70b (OpenAI, ; Anthropic, ; Gemini, ; Llama, ), improved the prevalence and the coherence of machine-generated content. This trend makes it increasingly difficult to differentiate between texts produced by machines from such written by humans Macko et al. (); Wang et al. (, ). As a result, there have been growing concerns about the authenticity and integrity of textual content Crothers et al. (); Tang et al. ().
While many detectors have been developed to address this new challenge Mitchell et al. (); Wang et al. (), they often struggle to keep up with the rapid development of LLMs. Generations produced by new models are hard to detect as they become more coherent and represent out-of-distribution instances, compared to what detecting systems saw during training Macko et al. (); Koike et al. ().
Moreover, the use of prompting to generate more human-like texts or applying LLMs to refine or change the tone of human writings further complicates detection.

Figure: Figure 1: LLM-DetectAIve interface: automatic text detection (top) and human detector playground (bottom).
Refer to caption: x1.png

Most prior work on detecting machine-generated text focused on binary detection, i.e., predicting whether the text is generated by a machine or written by a human. This dichotomy leaves no space for mixed categories of human-machine collaboration. However, we argue for the need for additional categories, as machine-polishing of human-written text is acceptable in certain cases (e.g., for academic papers), but not in other (e.g., in education).
In education, using LLMs to complete entire assignments or even to polish human-written essays is typically prohibited (Susnjak, ).
Therefore, it is important to perform fine-grained text classification.
For example, detecting the use of LLMs in text humanization and refinement becomes critical to ensure the fair assessment of students’ genuine knowledge and abilities. Fine-grained human/machine identification is also important for authorship detection in digital forensics.
To address this problem, we propose a new formulation of problem, as multi-way classification with the following labels:

- I.
Human-Written: the ext is created solely by a human author without GenAI assistance.
- II.
Machine-Generated: the text is entirely produced by a machine based on input prompts without any human intervention.
- III.
Machine-Written Machine-Humanized: the text is initially generated by a machine and then subtly modified to appear more human-like. This involves automatically tweaking the LLM to make the output appear more human.
- IV.
Human-Written Machine-Polished: the text is written by a human and then is refined or polished by a machine, e.g., to correct grammar, improve style, and/or optimize readability while trying to preserve the meaning of the original human text.

**Table 1: Statistics about our datasets across LLMs over the four classes: I. Human-Written, II. Machine-Generated, III. Machine-Written Machine-Humanized and IV. Human-Written Machine-Polished. For row II + III + IV, the data is approximately uniformly distributed across the three classes.**
| Text Class | Generator | OUTFOX | Wikipedia | Wikihow | Reddit ELI5 | arXiv abstract | PeerRead |
| --- | --- | --- | --- | --- | --- | --- | --- |
| M4GT-Bench |  |  |  |  |  |  |  |
| I | Human | 14,043 | 14,333 | 15,999 | 16,000 | 15,998 | 2,847 |
| II | davinci-003 | 3,000 | 3,000 | 3,000 | 3,000 | 3,000 | 2,340 |
| gpt-3.5-turbo | 3,000 | 2,995 | 3,000 | 3,000 | 3,000 | 2,340 |  |
| cohere | 3,000 | 2,336 | 3,000 | 3,000 | 3,000 | 2,342 |  |
| dolly-v2 | 3,000 | 2,702 | 3,000 | 3,000 | 3,000 | 2,344 |  |
| BLOOMz | 3,000 | 2,999 | 3,000 | 2,999 | 3,000 | 2,334 |  |
| gpt4 | 3,000 | 3,000 | 3,000 | 3,000 | 3,000 | 2,344 |  |
| New Generations |  |  |  |  |  |  |  |
| II + III + IV | gpt-4o | 8,966 | 8,995 | 9,000 | 9,000 | 9,000 | 7,527 |
| gemma-7b | 8,280 | 8,985 | 9,000 | 9,000 | 9,000 | 0 |  |
| llama3-8b | 8,271 | 8,985 | 9,000 | 9,000 | 9,000 | 0 |  |
| llama3-70b | 8,577 | 8,985 | 9,000 | 9,000 | 9,000 | 0 |  |
| mixtral-8x7b | 17,001 | 8,985 | 9,000 | 9,000 | 9,000 | 0 |  |
| gemma2-9b | 0 | 8,985 | 9,000 | 9,000 | 9,000 | 0 |  |
| III | gemini1.5 | 0 | 1,652 | 1,601 | 904 | 0 | 0 |
| mistral-7b | 0 | 2,993 | 3,000 | 0 | 0 | 2,344 |  |
| IV | gemini1.5 | 0 | 1,652 | 1,601 | 904 | 2,994 | 586 |
| mistral-7b | 0 | 2,993 | 3,000 | 0 | 0 | 2,344 |  |

We further develop LLM-DetectAIve, a system that accurately distinguishes between different types of text generation and editing. With this, we aim to uphold academic integrity and ensure a fair evaluation process for both students and researchers.
Our contributions are as follows:

- •
We reformulate the task as fine-grained multi-way classification.
- •
We collect a dataset for this reformulation using generations from a variety of LLMs.
- •
We build, evaluate, and compare several machine-generated text detectors on our new fine-grained dataset.
- •
We develop a Web-based demo that (*i*) allows users to input text and to obtain fine-grained classification prediction, and (*ii*) offers a playground for users to test their
ability to detect texts with varying degrees of LLM involvement, according to the above 4-way fine-grained schema.

## 2 Dataset

To collect the dataset for our multi-way fine-grained detector, we first gathered datasets that were curated for binary machine-generated text detection from previous work, and then we extended the data into our four labels by introducing new corresponding generations. Sections 2.2 and 2.3 discuss the prompts we used for generation and data cleaning, respectively.

### 2.1 Data Overview

We build the new dataset by extending the M4GT-Bench Wang et al. (), which is an benchmark dataset for evaluating machine-generation text detectors that encompasses multiple generators and domains, including arXiv, Wikihow, Wikipedia, Reddit, student essays (OUTFOX), and peer reviews (PeerRead). From these sources, we sampled a subset comprising 79,220 human-written texts and 103,075 machine-generated texts.
Next, we expanded this dataset by (*i*) collecting additional machine-generated texts produced by new LLMs (e.g., GPT-4o), (*ii*) generating machine-written then machine-humanized texts, and (*iii*) polishing human-written texts using various LLMs.
This resulted in 91,358 fully-MGTs, 103,852 machine-written then machine-humanized texts, and 107,900 human-written then machine-polished texts. Table 1 gives detailed statistics about the dataset.
For data generation, we used a variety of LLMs, including Llama3-8b, Llama3-70b Llama (), Mixtral 8x7b Jiang et al. (), Gemma-7b, Gemma2-9b Team (), GPT-4o (OpenAI, ), Gemini-1.5-pro (Gemini, ), and Mistral-7b Jiang et al. (). By incorporating a diverse array of LLMs and domains, we aim to enhance the detection accuracy within actual domains and generators, as well as improve generalization.

### 2.2 Generation Prompts

For the *Machine-Written Machine-Humanized* class, examples of prompts include Rewrite this text to make it sound more natural and human-written or “Rephrase this text to be easy to understand and personable.” For the *Human-Written Machine-Polished* class, we used prompts such as “Paraphrase the provided text.” or “Rewrite this text so that it is grammatically correct and flows nicely.” Additionally, we introduced a trailing prompt appended to each randomly selected prompt to prevent undesirable text that the LLM may prepend to its output, e.g., “Only output the text in double quotes with no text before or after it. Text: {} Your response:”.
We used 5-6 prompts per domain to generate data for the *Machine-Written Machine-Humanized* and *Human-Written Machine-Polished* classes. In addition to the *Machine-Generated* class, we used the original prompts from the M4GT-Bench dataset.

### 2.3 API Tools & Data Cleaning

For data generation, we used multiple APIs from OpenAI, Gemini, Groq, and DeepInfra, to generate a total of 303,110 texts for the three LLM-dependent classes. For each of the two new class generations, we limited the text length to 1,500 words in order to accommodate the context length restrictions of some smaller LLMs and to efficiently manage time and costs.
The output of the LLMs occasionally included formatting such as “Here is the paraphrased text:” and “Sure!” despite instructions in the trailing prompt to exclude any additional output. We removed these phrases in the post-processing with two considerations. On the one hand, this naturally occurs in real-world applications, i.e., humans will remove these irrelevant phrases when they use the target content.
Moreover, the presence of these indicative artifacts could impact the detectors’ generalization and the quality of the dataset, given that they are potentially unique for a specific text class.

## 3 Detection Models

We trained three detectors by fine-tuning RoBERTa (Liu et al., ), DeBERTa (He et al., ), and DistilBERT (Sanh et al., ).
DeBERTa is built upon BERT and RoBERTa by incorporating disentangled attention mechanisms and an enhanced mask decoder, which improves word representation.
Eventually, in the demo, we used DistilBERT, which is a compact and fast variant of BERT: 60% faster and 40% smaller, while retaining 97% of BERT’s language understanding capabilities.
Table 2 shows the values of the hyper-parameters for each model. We used RoBERTa and DistilBERT in our domain-specific experiments. However, due to the inferior performance of DistilBERT to RoBERTa in our preliminary trials, we substituted DistilBERT with DeBERTa in the following experiments (DeBERTa is superior to RoBERTa).

**Table 2: Hyper-parameter values across the models.**
| Dataset | Detector | Learning rate | Weight Decay | Epochs | Batch Size |
| --- | --- | --- | --- | --- | --- |
| arXiv | RoBERTa | 2e-5 | 0.01 | 10 | 16 |
| DistilBERT | 2e-5 | 0.01 | 10 | 16 |  |
| OUTFOX | RoBERTa | 2e-5 | 0.01 | 10 | 16 |
| DistilBERT | 2e-5 | 0.01 | 10 | 16 |  |
| Full Dataset | RoBERTa | 5e-5 | 0.01 | 10 | 32 |
| DeBERTa | 5e-5 | 0.01 | 10 | 32 |  |

## 4 Experiments and Evaluation

The previous studies have shown that the accuracy of detectors drops substantially when testing on out-of-domain examples (Wang et al., ).
To alleviate this, we propose three strategies: (*i*) train multiple domain-specific detectors, each specifically responsible for detecting inputs from one domain, (*ii*) train one universal detector using more training data across various domains, and (*iii*) leverage domain-adversarial neural network (DANN) for domain adaption.

### 4.1 Domain-Specific Detectors

We fine-tuned RoBERTa and DistilBERT using the data from arXiv and OUTFOX, using a ratio of training, validation, and test sets of 70%:15%:15%.
The results are shown in Table 3. We can see that both RoBERTa and DistillBERT performed well on OUTFOX. Overall, RoBERTa is more robust over diverse domains, with accuracy greater than 95% on both domains, with a small number of mis-classifications occurring between classes with overlapping features, such as Machine-Generated vs. Human-Written, vs. Machine-Polished classes, as the confusion matrices in Figure 2 show.
However, in this setup, the users need to first specify the domain of the input text, which is an extra effort. To mitigate this, we further trained a universal model that does not require the user to select the domain.

### 4.2 Universal Detectors

We fine-tuned RoBERTa and DeBERTa using the full dataset; the data distribution for this is shown in Table 4.
To reduce data imbalance and prevent the detector from favoring any particular class, we excluded some of the original data. The evaluation results in Table 5 indicate that DeBERTa consistently outperforms RoBERTa across all evaluation measures we use. Therefore, we deployed the fine-tuned DeBERTa as the backend detection model for our demo.

**Table 3: Domain-specific performance for RoBERTa and DistilBERT on arXiv and OUTFOX.**
| Detector | Test Domain | Prec | Recall | F1-macro | Acc |
| --- | --- | --- | --- | --- | --- |
| RoBERTa | arXiv | 95.82 | 95.79 | 95.79 | 95.79 |
| OUTFOX | 95.67 | 95.43 | 95.53 | 95.65 |  |
| DistilBERT | arXiv | 88.98 | 87.97 | 87.93 | 87.79 |
| OUTFOX | 96.66 | 96.65 | 96.65 | 96.65 |  |

Figure: Figure 2: Domain-specific confusion matrix for RoBERTa on arXiv (top) and on OUTFOX (bottom).
Refer to caption: x2.png

**Table 4: Distribution of the data used for fine-tuning universal detectors based on RoBERTa and DeBERTa.**
|  |  | Machine- | Machine- | Machine- |
| --- | --- | --- | --- | --- |
| Domain | Human | Generated | Polished | Humanized |
| arXiv | 15,998 | 18,000 | 18,000 | 18,000 |
| Reddit | 16,000 | 18,904 | 18,904 | 18,904 |
| wikiHow | 15,999 | 22,601 | 22,601 | 22,601 |
| Wikipedia | 14,333 | 22,615 | 22,615 | 22,615 |
| PeerRead | 2,847 | 4,684 | 4,684 | 4,684 |
| Outfox | 14,043 | 17,000 | 17,000 | 17,000 |

**Table 5: Detector performance on the full dataset.**
| Detector | Prec | Recall | F1-Macro | Acc |
| --- | --- | --- | --- | --- |
| RoBERTa | 94.79 | 94.63 | 94.65 | 94.62 |
| DeBERTa | 95.71 | 95.78 | 95.72 | 95.71 |

### 4.3 DANN-Based Detector

In our domain-specific experiments above, we achieved strong performance when the domain of the text was provided. However, in cross-domain evaluation, the performance is sub-optimal as previous work has suggested (Wang et al., , ). In real-world scenarios, the domain would not always be specified, and thus we need a classifier that is as domain-independent as possible.. Thus, we investigated the use of domain adversarial neural networks Ganin et al. () to train a domain-robust detector.
DANN was initially designed to achieve domain adaptation by aligning representations across different domains with three major components:

- •
Representation Extractor: which builds a representation of the input data; here, we use RoBERTa.
- •
Label Predictor: to predict the class labels based on the representation; it is trained using labeled data from the source domain.
- •
Domain Classifier: connected to the representation via a gradient reversal layer (GRL), it distinguishes between the source and the target domains. It multiplies the gradient by a negative constant during back-propagation, promoting domain-invariant representation.

The network is trained using standard back-propagation and stochastic gradient descent, optimizing the label classification loss while intentionally confusing the model regarding the domain by reversing the gradient from the domain classifier. This reduces the label classification loss while increasing the domain classification one.
As a result, the Domain-Adversarial Neural Network (DANN) yields a representation that is independent of the domain. In our experiments, we trained the DANN to predict our four classes and to be as confused as possible when predicting the six sources/domains.
The results are shown in Table 6. We can see that using domain adversarial training on top of RoBERTa-enhances the overall performance compared to just fine-tuning RoBERTa as in Section 4.2. This suggests that decoupling the model from domain-specific representation leads to an improvement in its overall performance.

**Table 6: Comparing domain-specific RoBERTa vs. DANN+RoBERTa. The latter outperforms the former across all measures, indicating that decoupling the model from domain-specific representation is beneficial.**
| Detector | Prec | Recall | F1-macro | Acc |
| --- | --- | --- | --- | --- |
| RoBERTa | 94.79 | 94.63 | 94.65 | 94.62 |
| DANN+RoBERTa | 96.30 | 95.54 | 96.06 | 95.24 |

### 4.4 Comparison to Existing Systems

There are several previously proposed systems for detecting machine-generated text, such as GPTZero,(^2^22[https://gptzero.me/](https://gptzero.me/)) ZeroGPT,(^3^33[https://www.zerogpt.com/](https://www.zerogpt.com/)) and Sapling AI detector,(^4^44[https://sapling.ai/ai-content-detector](https://sapling.ai/ai-content-detector)) but none of them supports four classes. GPTZero is the only one that goes beyond binary classification: it adds a *mixed text*; however, it limits users to only 40 free runs per day or 10,000 words per month for registered accounts. Thus, we could not perform comparison on our entire test dataset. Instead, we randomly sampled 60 machine-generated texts and 60 human texts (10 per source) per source. In this binary classification setting, LLM-DetectAIve achieved 97.50% accuracy, outperforming GPTZero, ZeroGPT, and Sapling AI, with 87.50%, 69.17%, and 88.33%, respectively.

### 4.5 Generalization Evaluation

To evaluate the generalization ability of our detector on unseen domains and generators, we experimented with testing on two additional datasets: MixSet (Ji et al., ) and IELTS essays written by individuals for whom English is a second language.(^5^55[https://huggingface.co/datasets/chillies/IELTS_essay_human_feedback](https://huggingface.co/datasets/chillies/IELTS_essay_human_feedback))
For the IELTS essays, after deduplication, we randomly sampled 300 (essay problem statement, human-written essay) pairs, and then we produced the corresponding machine-written essays using the problem statements based on Llama3.1-70B. We further generated Machine-Written Machine-Humanized and Human-Written Machine-Polished.
For MixSet, the original dataset contains a total of 3,600 examples, with 300, 300, 600, and 2,400 examples for Human-Written, Machine-Generated, Machine-Written Machine-Humanized and Human-Written Machine-Polished, respectively.
It involves models such as Llama2-70B and GPT-4, and text covering domains of email content, news, game reviews, and so on.
The results are shown in Table 7, where we can see that the detector performs much worse on unseen domains and generators, compared to in-domain and in-generator cases. The performance on IELTS is better than on MixSet.
This can be attributed to the inclusion of the OUTFOX data (English native-speaker student essays) in the training data, while the domains and the generators in MixSet are not in the training set.
The low generalization performance suggests challenges in adapting black-box detectors to the diverse domains and generators in real-world applications.

**Table 7: Cross-domain evaluation of our detector on unseen domains and generators: IELTS and MixSet.**
| Dataset | Prec | Recall | F1-macro | Acc |
| --- | --- | --- | --- | --- |
| IELTS | 63.74 | 66.91 | 66.55 | 66.91 |
| MixSet | 59.18 | 64.25 | 54.95 | 60.08 |

## 5 Demo Web Application

Our demo web application has two interfaces: (*i*) an interface for fine-grained MGT detection, and (*ii*) a playground for users.

### 5.1 Automatic Detection

The automatic detection interface is shown in Figure 1 (top). It allows users to input a text, and then the system responds with the class that the text belongs to. To ensure the prediction accuracy, the length of the submitted text is constrained to 50-500 words since the performance of our detectors drops significantly for shorter texts.
Longer texts will be truncated, as we are limited by the context window size of the BERT-like transformers we use.

### 5.2 Human Detector Playground

The demo further offers a human detector playground as an interactive interface, which allows users to test their capability to distinguish between the four text categories. Figure 1 (bottom) shows a snapshot of the playground interface where the users can try the system, gaining insights into the subtle differences between various types of human-written and machine-generated texts.

### 5.3 Deployment and Implementation

Our demo is deployed on Hugging Face Spaces, which allows seamless integration with transformer models, ease of use, and robust support for hosting machine learning applications. For implementing the user interface, we used Gradio. The code is publicly available under an MIT license.

## 6 Conclusion and Future Work

In an era of advanced large language models, maintaining the integrity of text poses significant challenges. We presented a system that aims to identify the use of machine-generated text, accurately differentiating human-written text from various types of automatically generated text.
Unlike previous work, we use a fine-grained classification schema (Human-Written, Machine-Generated, Machine-Written Machine-Humanized, and Human-Written Machine-Polished), which offers insights into the origins of the text, thus enabling trustworthiness.
In future work, we plan to improve the Domain Adversarial Neural Network (DANN) to improve the results even further. We further plan to explore the possibility of using a DANN on the text’s generator instead of the text’s domain to generalize detection across different text generators. Using a DANN on both the domain and the generator could potentially lead to a truly universal detector. We also aim to expand the classification to include a fifth category: machine-written and human-edited text, enhancing detection capabilities and providing a more comprehensive analysis of text origins. To further improve the system, we also plan to address potential biases in the dataset caused by formatting styles linked to specific domains, such as Wikihow and PeerRead, to ensure better robustness across a broader range of human-written content. Last but not least, we want to expand the dataset to encompass a diverse set of languages, enabling the development of a robust multilingual detection model.

## Limitations

We acknowledge certain limitations of our work, which we plan to address in future work. First, although our work has explored more fine-grained machine-generated text scenarios beyond conventional binary classification, we did not consider a complex scenario where the text is first generated by a machine and then is manually edited by humans to suit their personal needs. This is primarily due to the high costs associated with collecting data that requires human editing.
Moreover, we identified some issues with the dataset. Specifically, some LLMs associate specific domains with particular formatting styles, such as markdown for lists, bullet points, and headers. This issue was particularly noticeable in the Wikihow and PeerRead domains, where the LLMs frequently applied these formatting styles, potentially skewing the data and impacting the accuracy of our classifications. It also remains uncertain whether our system can generalize to detecting models or languages not included in our English-only dataset.

## Ethical Statement and Broad Impact

#### Data License

A primary ethical consideration is the data license. We reused pre-existing corpora, such as OUTFOX and Wikipedia, which have been publicly released and approved for research purposes. Moreover, we generated new data on top of the original data, thereby mitigating concerns regarding data licensing.

#### Biased and Offensive Language

Considering that our data is generated by large language models, it might contain offensive or biased language; we did not try to control for this, replying on the inetnal safety mechanisms of the LLMs we used.

#### Positive Impact of Fine-Grained Detection

LLM-DetectAIve expands the conventional binary classification in machine-generated text detection to more fine-grained levels, which is more aligned with real-life scenarios. We believe this approach could be applied in various scenarios, e.g., for students’ essays to ensure the originality of their work. Moreover, LLM usage detection may find applications in authorship detection as well as in digital forensics.

## Peer Reviews

confidence: 3
contribution: 3
initial_rating: 5
presentation: 2
questions: Q1. Please explain the notation used in the Definition and Theorems in Sec. 3.3
- What is Patch? Does it correspond to a set of tokens, or a subset of features (e.g. coordinates) in the text embeddings, or smth else?
- What is feature vector $v_k$? The index indicates the whole cluster, but it is used for the description of the individual data point 
- What does the notation $y\alpha v_k$ mean? Is it a vector multiplied by 2 scalars $y$ and $\alpha$ ?

In general, could you please provide the example of the considered setup in the Machine-Generated Text Detector domain, defining patches, clusters, data features, distraction features and the noise.

Q2. How many detectors were used for each dataset? Does this number correspond to the theoretical bound from Theorem 2?

Q3. How are ChatGPT-D and Roberta-MPU atapted to fine-grained MTD setup? Are they fine-tuned on the same dataset as MoD? 

Q4. Please describe the statistics of the train/test dataset used in Tables 1 - 3 (its fine-grained part). For Generator models, please specify which version of each model is used (i.e. number of parameters)
soundness: 2
strengths: - The paper proposes novel important tasks of fine-grained MTD. The presented analysis demonstrates that indeed Machine-Generated, Machine-Translated and Machine-Paraphrased texts have unique features and different usage scenarios, and it is important to distinguish them
- Novel architecture of MTD is proposed
- The evaluation is done of large amount of data domains and generator models, and includes OOD setup. The proposed method outperforms most of the considered baselines by a large margin.
summary: The paper presents novel task of fine-grained MTD detection, where the detector should be able to predict 4 labels: humn-generated, machine-generated, machine-translated and machine-paraphrased. Novel architecture for this task is roposed, refered as MoD (Mixture of Detectors), consisting several detectors for individual domains, and the trainable router. 

Theoretical results are presented demonstrating the theoretical benefits of the proposed architecture over the individual detector.

The evaluation is done on several popular MTD datasets for various generator models. Besides, the OOD evaluation is presented. The method outperformes  the most of the considered baselines by a large margin
weaknesses: - The theoretical framework in Sec 3.3 is not quite clear (see Questions)

- The benefit of the proposed method over standard Mixture-of-Expert is not clear
 
- In section 3.1, the statistics of the generated data is absent (see Q4)

- The most of the baseline models are Roberta-based classifiers. Comparing to the proposed method, they have k times less parameters, where k is the number of individual detectors. Only two baselines do not belong to this class: RoBERTa-MoS and Binoculars; comparing to them, the proposed method have marginal to no improvement

- No comparison to Roberta-MoS in OOD setup    

- In Sec. 4.3, the Qualitative result paragraph describes the properties of the dataset rather then the classifier results. There is no information about the typical errors, or any other qualitative description of the results of the proposed detector.

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 3
questions: LN053: what is the special of your method compared to Abassy et al., 2024 given that they both do fine-grained MGT detection?

LN091: have you considered GPT models for the detection? What is the underlying reason for choosing RoBERTa? 

LN320: how do you calculate the AUROC for the 4-class classification problem?

Table 1: the experiments are limited to generations from open-source models. Have you considered latest closed-source models like GPT4, Gemini, and Claude3?

Table 1: In my view, a RoBERTa-single model trained on the same data is a demanded baseline to demonstrate the advantage of the MoE design.

Table 1: Are models like ChatGPT-D, RoBERTa-MPU, and LLM-DetectAIve trained on the same data as the proposed method?

LN427: How are your zero-shot experiments designed? When we say zero-shot, we refer to a method without any task specific training. How could your MoD method do the zero-shot given that it requires a joint training of the router and the detectors?
soundness: 2
strengths: 1) It presents an in-depth study on fine-grained Machine-Generated Text (MGT) detection, a topic that has been overlooked in previous studies. By classifying text into four categories (human-written, machine-generated, machine-paraphrased, and machine-translated), the research contributes significantly to the field. 
2) The paper introduces the RoBERTa-based Mixture of Detectors (RoBERTa-MoD), a novel method that uses multiple domain-optimized detectors for more robust and generalized performance. This method addresses the performance drop on out-of-domain texts, a key challenge in MGT detection. 
3) The paper's quality is evident in the theoretical proof provided for the method's superiority over a single detector. Additionally, the research is significant as it achieved a 5-9% improvement in mean Average Precision (mAP) over prior work on six diverse datasets.
summary: The paper presents an in-depth study on fine-grained Machine-Generated Text (MGT) detection, which can classify text into four categories: human-written, machine-generated, machine-paraphrased, and machine-translated. The authors note that existing detectors struggle with out-of-domain text, particularly for translated or paraphrased text. To address this, they propose a RoBERTa-based Mixture of Detectors (RoBERTa-MoD) which uses multiple detectors optimized for different domains to improve performance. The authors provide a theoretical proof that their method outperforms a single detector and experiments show a 5-9% improvement in mean Average Precision (mAP) over previous work on six diverse datasets. The authors also introduce a data preparation process to generate articles across different fine-grained categories, enabling automatic creation of training and evaluation data for the task.
weaknesses: 1) The concept of employing a mixture of experts (MoE) for a task is not unusual, considering its extensive application in various tasks such as general LLM, summarization, and machine translation. While the application of MoE in text detection is relatively new, it is not a groundbreaking concept in terms of its fundamental idea. 
2) The study lacks an ablation analysis on MoD. Questions such as the influence of the number of detectors on the results, the effect of different corpus (domains) on detection, and the likelihood of the router specializing to specific detectors given an input article, remain unanswered. 
3) The paper doesn't delve deep into the confusion between classes, for instance, between translate and human/paraphrase. As the paper is centered on the fine-grained detection of machine-generated text, such analyses about the challenges of fine-grained detection are anticipated and would offer valuable insights to the readers. 
4) The experimental results may not have been compared in a fair manner. However, due to the lack of clear descriptions of the settings for the baselines, I will reserve my judgment until the rebuttal period, during which I expect a response.

---

confidence: 3
contribution: 2
initial_rating: 3
presentation: 3
questions: * Could you address the weakness mentioned above?
soundness: 2
strengths: * An interesting twist to an important problem

* An analysis that combines both theoretical insights and empirical ablations
summary: This paper addressing the detecting of machine generated text proposed to extend the traditional binary classification (machine or human generated) to a four-classes problem: human-authored, machine-generated, machine-translated or machine-paraphrased. The authors propose a multi-class classification model uses 4 Roberta-models and a gating mechanism. The whole network is trained in two stages
weaknesses: * Motivation: the idea behind a 4-class problem (instead of the more traditional 2-class) is motivated by the fact that machine-modified text (translated or corrected) might not be as harmful as text that is generated by a model from scratch. While this is a somehow appealing explanation, it would be interesting to see if that difference actually has an empirical impact. Sect 4.5 makes this attempts, by applying the proposed method on existing benchmarks, obtaining good scores (although not always outperforming the current state-of-the-art). More interesting to me would be an analysis on how `Binoculars` performs on the new datasets (considered as binary problem). Another interesting aspect would be to verify if the more fine-grained classification actually results in a better binary classification.

---

confidence: 3
contribution: 2
initial_rating: 5
presentation: 2
questions: — The translated and paraphrased texts, if created using LLMs, can also contain misinformation and factual errors. It depends on the LLM; the percentage of errors is much rarer than that of machine generation, but it still needs to be checked. 

— "As discussed in the Introduction," add the link to the Introduction section.

— Line 307: Set the spaces in cite like in:  `GoodNews(B`

— The paper would improve from the footnotes to direct links for the open datasets and a clear explanation of the steps with data processing.

— Table 1 would improve if the information about the model's size was added.

— It's a bit strange not to see the results section.

— The results would be interesting to check on the different lengths of the output. We could see the correlation with length.

— Factual error: LLM-DetectAIve distinguishes four categories: (i) human-written, (ii) machine-generated, (iii) machine-written, then machine-humanized, and (iv) human-written, then machine-polished. The idea is different from paraphrasing.
soundness: 3
strengths: — The idea of separating into three categories is intuitive and transferred from the actual use cases. However, it makes the task more complicated as the generation style is similar for paraphrasing/translating/generation using the same models.

— It is beneficial for nature to theoretically assert if some models and approaches are worse or better without extensive model training.

— Reproducibility statements, limitations, and ethical considerations are included. Clear contribution. The code and data will be publicly released upon acceptance.
summary: The paper investigates fine-grained MGT detection; the authors propose to categorize an input text into four classes: human-written, machine-generated, machine-paraphrased, and machine-translated. They introduce a data preparation process to generate articles across different fine-grained categories, enabling the automatic creation of training and evaluation data for the task. The paper introduces a RoBERTa-based Mixture of Detectors (RoBERTa-MoD) for fine-grained MGT detection, which leverages multiple domain-optimized detectors for more robust and generalized performance. The paper presents theoretical proof that the method outperforms a single detector, and experimental findings show an improvement in mAP over prior work on six various datasets: GoodNews, VisualNews, WikiText, Essay, WP, and Reuters.
weaknesses: Some questions for reproducibility of the research.

— Please include information about the strategy of round-trip translation and paraphrasing. The rationale behind round-trip translation is not discussed. Was the translation performed for each language using all the models? 
Are the authors certain that the prompt "Paraphrase/Translate the following article: x." was used consistently across all models? If so, how does LLAMA determine which language to translate? What are the specific languages used? 
How do authors gather paraphrases and translations? For each article, there should be a minimum of four different translations or paraphrases. Have the authors confirmed that removing the first sentence from the text does not alter the meaning of the translation? 

— For the training data, have the authors saved the model's proportion distribution? No statistics for the training corpora. How many examples were used for each class? 
Section 3.1 needs more details.

— Line 372  `All methods were fine-tuned on data from Llama-3 (Touvron et al., 2023) and Qwen-1.5 (Bai et al., 2023) and then evaluated on all LLMs.`
Which data? How much data in which format? Maybe the authors mean: "evaluated on the data from all the LLMs"? I guess the methodology was to check in out-of-domain evaluation so that the data formed with different models and not evaluated by the same models in an LM-as_judge manner. The formulations are confusing. It is not clear from the texts what data you trained detectors, fine-tuned which exact models, and with what models you evaluated what.

`Llama-3 and Qwen-1.5 are in-domain generators for training the detector, and StableLM-2, ChatGLM-3, and Qwen-2.5 are out-of-domain generators to evaluate the model’s generalization ability.`
Am I right that, based on the data from models StableLM-2, ChatGLM-3, and Qwen-2.5, were the detectors not trained?

`However, in the same dataset, human-written articles in the training and testing sets may follow similar data distributions`
There is no information on whether they may or may not.

— Line 325: `LLM-DetectAIve is directly trained on fine-grained MGT data, which can be considered as a fine-tuned RoBERTa.`
Why should it be considered? No explanations/justifications

— Figure 5. It seems like the Qualitative Results are based on one example. Paraphrasing can change factual information as well as translation. The paper needs some quantitative metrics to catch it.

Meta-review: This paper proposes a new multi-class task formulation for machine text detection by subcategorizing machine text into generated, translated and paraphrased text. A RoBERTA-based mixture of domains model with a router mechanism is used for the task. Experiments are presented across a variety of domains.

**Strengths:** The paper poses an interesting question about machine text, by claiming that not all machine text may be the same. They present results across different domains and compare with multiple baseline approaches. 

**Weaknesses:** The authors claim that their main contribution is the task - but I’m not convinced that model translated and model paraphrased text might be in principle different from machine generated text. The assumption about factuality seems somewhat strong: humans also make many false claims! Some empirical evidence of this would make for a more compelling case for this paper. Secondly, it is unclear how their RoBERTa-MoD classifier does on the OOD task when adapted to a binary setting (also pointed out by reviewer CWCv). 

**Reason for decision**: See above. The main contribution of the work: the four-way classification task seems somewhat arbitrarily defined; reviewers were not satisfied with the depth of analysis and experimentation.

Decision: Reject