Grape at G en AI Detection Task 1: Leveraging Compact Models and Linguistic Features for Robust Machine-Generated Text Detection (2025)

Abstract: In this project, we aim to address two subtasks of Task 1: Binary Multilingual Machine-Generated Text (MGT) Detection (Human vs. Machine) as part of the COLING 2025 Workshop on MGT Detection (Wang et al., 2025) using different approaches. The first method involves separately fine-tuning small language models tailored to the specific subtask. The second approach builds on this methodology by incorporating linguistic, syntactic, and semantic features, leveraging ensemble learning to integrate these features with model predictions for more robust classification. By evaluating and comparing these approaches, we aim to identify the most effective techniques for detecting machine-generated content across languages, providing insights into improving automated verification tools amidst the rapid growth of LLM-generated text in digital spaces.

## introduction the rapid development of large language models (llms) such as gpt-4o, claude3.5, and gemini1.5-pro has led to an explosion of machinegenerated text across various channels, including news, social media, and academic publications. @xcite , based on 24 studies of academic domains, points out that using artificial intelligence enhances the productivity of researchers. while this advancement is promising, it has raised significant concerns about misuse, including spreading misinformation and potential disruptions in educational contexts due to the unpredictable nuance of these language models. to address these issues, it is crucial to develop effective systems for distinguishing between humanwritten and machine-generated content. there are two subtasks in the task 1: the primary goal of this project is to develop an automatic detection system capable of distinguishing machine-generated text from human-written text using small-sized language models. by integrating models with fewer parameters-thus lower computational demands-we aim to demonstrate that effective detection does not require large, resourceintensive models. specifically, our objectives are to: ## related work recent research has focused on detecting machinegenerated text using various techniques. for the traditional methodologies, gltr uses statistical methods to detect generated text with an improvement in human detection of fake text from 54% to 72% @xcite . with the explosive growth of transformers and large language models(llms), @xcite shows that fair_wmt20 and gpt-3 excel at generating human-like text. recently, in the semeval-2024 task 8: multidomain, multimodel and multilingual machine-generated text detection @xcite , many researchers have tried to apply different approaches, such as statistical, language models, and llms to solve the subtask a: human vs. machine classification. @xcite study mixing llama-2 features, achieved top accuracy. their performance relies on multiple llms and features, focusing on the last tokens. other teams also attempt to use language models, such as roberta or xlm-roberta @xcite @xcite . regarding our hardware limitations, we want to try to evaluate newly released language models in small sizes. in addition, all previous works on the semeval-2024 task 8 mostly work with llms. based on the success of @xcite with llama-2, we consider using llama3 @xcite . while spiegel and macko (2024) proposed combined fine-tuned llms with zero-shot statistical methods, employing a two-step majority voting system for predictions, @xcite utilized a fine-tuned baseline -roberta augmented with diverse linguistic features. all these methods surpass the baseline and achieve good results, supporting our approach, which is a potential way to mix llms with traditional linguistic features. ## proposed approach recent released large language models, such as llama 3 @xcite or gemma 2 @xcite , are now available in smaller configurations. these smaller models still perform well on popular benchmarks while being more compatible with hardware constraints. therefore, we decided to fine-tune these models for our task, utilizing their smaller versions to match our hardware limitations. ## conclusion we successfully addressed both subtasks using majority voting and ensemble methods. our approach comprised fine-tuned small language models and linguistic features, contributing to robust task performance. specifically, fine-tuning small language models allowed us to capture critical nuances in the data while maintaining computational efficiency. meanwhile, incorporating linguistic features, such as syntactic complexity, readability metrics, and lexical diversity, added a complementary layer of information that enhanced the ensemble's overall effectiveness.