E ssay D etect at G en AI Detection Task 2: Guardians of Academic Integrity: Multilingual Detection of AI -Generated Essays (2025)

Abstract: Detecting AI-generated text in the field of academia is becoming very prominent. This paper presents a solution for Task 2: AI vs. Hu- man – Academic Essay Authenticity Challenge in the COLING 2025 DAIGenC Workshop 1. The rise of Large Language models (LLMs) like ChatGPT has posed significant challenges to academic integrity, particularly in detecting AI-generated essays. To address this, we pro- pose a fusion model that combines pre-trained language model embeddings with stylometric and linguistic features. Our approach, tested on both English and Arabic, utilizes adaptive training and attention mechanisms to enhance F1 scores, address class imbalance, and capture linguistic nuances across languages. This work advances multilingual solutions for detecting AI-generated text in academia.

## introduction the exponential growth of large language models (llms) has led to widespread applications, including language translation, question answering, text generation, and beyond. however, their unauthorized use by students to complete homework, write essays, and write content-specific questions compromises academic integrity, highlighting the need for ai-driven llm text detection. using aigenerated content in academic contexts also poses challenges related to plagiarism @xcite . the existing literature proposes various methods for ai-generated text detection, including featurebased models, supervised, zero-shot, and adversarial approaches. all of these models are designed to improve the result of detection in different languages and styles. despite achieving decent overall accuracy, these methods still suffer from high false positives, where human-generated text is misclassified as ai-generated. furthermore, class-wise accuracy remains a challenge, indicating room for improvement in distinguishing between humangenerated text and ai-generated text. to address these issues, the coling 2025 workshop on daigenc @xcite task 2, "ai vs. human -academic essay authenticity challenge" aims to identify machine-generated essays to safeguard academic integrity and prevent misuse of llms in education. the task, framed as-"given an essay, identify whether it is generated by a machine or authored by a human"-is a binary classification challenge divided into two sub-tasks: subtask a for english essays and subtask b for arabic. our final model is a fusion of feature-based models and plm embeddings. initially, the plm showed poor performance with a bias toward the majority class. by integrating linguistic and stylistic features, we improved the overall macro f1 score. our focus addressed three key challenges: capturing feature dependencies, handling class imbalance, and optimizing training to preserve linguistic representations in lower layers while enabling higher layers to capture task-specific (essay) stylistic differences. ## related work over the last few years, numerous approaches have been proposed to tackle the task of ai-generated text detection. detecting machine-generated text is formulated primarily as a binary classification task @xcite @xcite , naively distinguishing between human-written and machine-generated text. in general, there are three main approaches: the supervised methods @xcite @xcite @xcite , the unsupervised ones, such as zero-shot methods @xcite @xcite , 2023; hans et al.; shijaku and canhasi, 2023) and adversarial measures on detection accuracy (susnjak and mcintosh, 2024; liang et al., 2023), especially within the education domain. for example, (antoun et al., 2023) evaluates the robustness of the detectors against character-level perturbations or misspelled words, focusing on french as a case study. @xcite train a generative model (dipper) to paraphrase paragraphs to evade detection. although supervised approaches yield relatively better results, they are susceptible to overfitting @xcite . there are some techniques like feature-based, fusion, and ensemble methods, such as word count, vocabulary richness, and readability concatenated ml, neural based or finetuned @xcite @xcite @xcite . ## proposed model we use a fusion model that combines stylometric features with plm embeddings, fine-tuned together for binary classification of ai vs. human text. ## results and analysis ## conclusion the unethical misuse of llms in academic contexts poses challenges to integrity, highlighting the need for effective ai-generated text detection. our fusion model, combining stylometric features with plm embeddings, addresses 3 key challenges identifying highly discriminative ones using lime, focal loss for addressing class imbalance and apply layer-wise freezing during fine tuning to capture task-specific stylistic differences in essays. these strategies have significantly improved model performance. for subtask a (english), our deberta + features model achieved a macro f1 score of 0.978, while for subtask b (arabic), the arabert + features model scored 0.9429. future work may refine these techniques to further enhance model's classwise f1 and generalization.