CMI - AIGCX at G en AI Detection Task 2: Leveraging Multilingual Proxy LLM s for Machine-Generated Text Detection in Academic Essays (2025)

Abstract: This paper presents the approach we proposed for GenAI Detection Task 2, which aims to classify a given text as either machine-generated or human-written, with a particular emphasis on academic essays. We participated in subtasks A and B, which focus on detecting English and Arabic essays, respectively. We propose a simple and efficient method for detecting machine-generated essays, where we use the Llama-3.1-8B as a proxy to capture the essence of each token in the text. These essences are processed and classified using a refined feature classification network. Our approach does not require fine-tuning the LLM. Instead, we leverage its extensive multilingual knowledge acquired during pretraining to significantly enhance detection performance. The results validate the effectiveness of our approach and demonstrate that leveraging a proxy model with diverse multilingual knowledge can significantly enhance the detection of machine-generated text across multiple languages, regardless of model size. In Subtask A, we achieved an F1 score of 99.9%, ranking first out of 26 teams. In Subtask B, we achieved an F1 score of 96.5%, placing fourth out of 22 teams, with the same score as the third-place team.

## introduction the capabilities of large language models (llms) are advancing rapidly, with models like, chat-gpt (openai, 2022), @xcite , google gemini @xcite , and llama3.1 @xcite generating increasingly fluent and human-like text. students can easily leverage these models to produce coherent, logical texts for assignments or essays, which profoundly impacts traditional educational methods of learning and evaluation, leading to issues in academic integrity and a weakening of critical thinking skills. however, humans perform * corresponding author: zhendong mao. only slightly better than random chance in distinguishing between machine-generated and humanwritten text @xcite , underscoring the urgent need for an automated system to identify machine-generated content. to address this, @xcite organized the genai detection task 2, a challenge focused on detecting machine-generated academic essays in english and arabic to uphold academic authenticity and prevent the misuse of llms in educational contexts. most current methods for detecting machinegenerated text can be generally categorized into two approaches @xcite : zero-shot detection and supervised detection. the former is time-consuming and suffers from performance degradation when the generation model is unknown, while the latter like roberta-based detection @xcite requires fine-tuning large models, which is resource-intensive and often lacks multilingual capabilities. in contrast, we employed a multilingual model, such as llama-3.1-8b @xcite , as a proxy. by extracting high-dimensional token essences and classifying them with a convolutional neural network, our model achieves high accuracy even without knowledge of the generation model. furthermore, it does not require fine-tuning and effectively utilizes the multilingual knowledge embedded in the llm's pretraining, making it a simple, efficient solution for detecting machine-generated text in both english and arabic. in subtask a, our model achieved an f1 score of 0.999, ranking first among 26 teams. in subtask b, we obtained an f1 score of 0.965, securing fourth place among 22 teams. in short, our contributions are as follows: (1) utilizing the last-layer essences of proxy llms as features enhances detection performance. (2) the scale of the proxy llms does not significantly improve detection accuracy. (3) proxy llms with broader multilingual knowledge exhibit higher detection accuracy. 2 related work machine-generated text detection methods can generally be divided into two categories. the first category is zero-shot detection, where the simplest approach involves calculating the average loglikelihood of a text @xcite , establishing a strong baseline for many zero-shot detection methods. more advanced techniques, such as detectgpt @xcite and its improved version fast-detectgpt @xcite , have shown that machine-generated text tends to fall within regions of negative probability curvature, effectively enabling machine-generated text detection. however, these approaches are often time-intensive and experience a significant performance drop when the generation model is unknown. the second category involves supervised detection methods. for instance, @xcite employed a fine-tuned roberta-large @xcite as a detector, but found it challenging to generalize effectively across different generation models. the t5-sentinel @xcite addresses text detection by leveraging the next-token prediction capability of t5 @xcite . additionally, @xcite introduced an iterative training process involving both a paraphraser and a detector, aiming to enhance robustness against paraphrasing attacks. @xcite integrated the text to be detected into the prompt and directly asked chatgpt whether the text is machine-generated or human-written, which is similar to our method, as both approaches leverage llms. however, our method does not directly inquire whether a text is machine-generated using llm, nor does it require fine-tuning the llm. instead, it harnesses the high-dimensional multilingual representation capabilities of llama-3.1-8b and the text is simply input into llama-3.1-8b to extract token essences (refer to the last layer hidden states) as features, which are then fed into a classifier for final classification. ## system overview to obtain a meaningful representation for the input text, we feed it into a proxy llm, llama-3.1-8b @xcite , to extract essences from the last layer of the proxy llm and subsequently pass the average of the essences through the refined feature classification network (rfcn), the overall model structure is shown in figure 1 . the original text to be detected is first tokenized, with shorter sequences padded and longer ones truncated to a maximum length of 1024 tokens, resulting in the tokenized sequence @xmath0, x 2 , . . . , x n }, the procedure is as follows: token essences from the proxy llm the tokenized sequence x is input into the llama-3.1-8b model, which supports text across multiple languages. as x passes through the proxy llm, it generates hidden states for each token at each layer. we specifically focus on the last-layer token essences (hidden states) of the proxy llm, which serve as the high-level representations of each token. these token essences encapsulate both their individual meanings and the broader context within the text. here, the representation quality across different languages is consistent. supplementary details can be found in the appendix b. to derive a single representation h of the input text, we take the average of the essences across all n tokens. refined feature classification network the averaged representation h is then input into the rfcn for classification. in the first stage, the cnn extracts relevant features from the input through three convolutional and pooling layers, progressively capturing more complex patterns information. in the second stage, the refined features are passed through three fully connected layers, where each layer fine-tunes the representations by learning complex relationships and interactions between 5 and 6 in appendix e. evaluation metrics for both subtask a and subtask b, the primary evaluation metric is macro-f1, calculated as the harmonic mean of precision and recall. ## results in this section, we present the results of our final submission to demonstrate the effectiveness of our approach, comparing our system's performance with that of several top-performing teams, and highlight key insights from our analysis. ## conclusion this paper presents our approach and results for the genai detection task 2, where our system ranked first in the english track and tied for third in the arabic subtask. we adopted an efficient strategy, using proxy llm to generate fused token essences, which were then classified via a refined feature classification network. this method capitalizes on the multilingual representational capacity of llms without fine-tuning, enhancing performance in detecting machine-generated text. our findings further underscore that proxy models with extensive multilingual knowledge markedly improve detection in multilingual contexts. future work will explore the broader application of multilingual llms in language generation detection and investigate optimized strategies to leverage llm token essences.