CNLP - NITS - PP at G en AI Detection Task 2: Leveraging D istil BERT and XLM - R o BERT a for Multilingual AI -Generated Text Detection (2025)

Abstract: In today’s digital landscape, distinguishing between human-authored essays and content generated by advanced Large Language Models such as ChatGPT, GPT-4, Gemini, and LLaMa has become increasingly complex. This differentiation is essential across sectors like academia, cybersecurity, social media, and education, where the authenticity of written material is often crucial. Addressing this challenge, the COLING 2025 competition introduced Task 2, a binary classification task to separate AI-generated text from human-authored content. Using a benchmark dataset for English and Arabic, developing a methodology that fine-tuned various transformer-based neural networks, including CNN-LSTM, RNN, Bi-GRU, BERT, DistilBERT, GPT-2, and RoBERTa. Our Team CNLP-NITS-PP achieved competitive performance through meticulous hyperparameter optimization, reaching a Recall score of 0.825. Specifically, we ranked 18th in the English sub-task A with an accuracy of 0.77 and 20th in the Arabic sub-task B with an accuracy of 0.59. These results underscore the potential of transformer-based models in academic settings to detect AI-generated content effectively, laying a foundation for more advanced methods in essay authenticity verification.

## introduction large language models (llms), such as chat-gpt foot_0 have made remarkable advances in generating human-like text responses, leading to diverse and sophisticated outputs tailored to specific prompts. while these advancements open up extensive practical applications, they also present challenges, including concerns over academic integrity and questions surrounding actual authorship. considerable research has been devoted to distinguishing machine-generated texts (mgt) from human-written texts (hwt). this has primarily involved model-based techniques @xcite and statistical analysis methods to examine distinct text characteristics @xcite . several platforms like gptzero @xcite and sapling have effectively differentiated mgt from hwt. detection of mgt has commonly employed a binary classification approach to distinguish between mgt and hwt. however, advancements in llms have blurred these distinctions, challenging the efficacy of straightforward classification techniques. for instance, in statistical detection, the linguistic features of an mgt might closely resemble those typically found in hwt, leading to potential misclassification. similarly, model-based approaches often need help to generalize effectively; they are typically trained on specific datasets or models and may not perform as accurately as newer models emerge with distinct characteristics. additionally, many detection systems need more transparency. although some detection tools attempt to integrate explanatory elements, they often fail to deliver insightful interpretations, as observed in evaluations of models like gptzero @xcite . ## related work zero-shot detection methods leverage statistical attributes to differentiate mgt from hwt. research in this area has explored various language model (lm) driven features such as entropy @xcite , average log probability scores @xcite , and perplexity @xcite as indicators. as lms advance, generating increasingly sophisticated text, recent zero-shot detection methods @xcite have evolved to capture high-level characteristics in generated content. one notable zero-shot detection model, binoculars @xcite , uses lms to make nexttoken predictions across text positions. by analyzing the log perplexity ratio relative to baseline text, binoculars identifies nuanced discrepancies that help distinguish mgt from hwt effectively. this technique represents an essential advancement in zero-shot detection, adapting to the sophisticated language features characteristic of current llm outputs. ## conclusion this study demonstrates the effectiveness of transformer-based models, such as distilbert and xlm-roberta, distinguishing between humanwritten and ai-generated essays across english and arabic. our team cnlp-nits-pp achieved a high detection recall of 0.825 for english and an accuracy of 0.590 for arabic, indicating these models' adaptability to diverse linguistic contexts. ensemble methods further improved classification accuracy, underscoring the importance of robust detection systems as ai-generated content continues to proliferate. future research could investigate additional linguistic features and cross-domain applications to enhance detection performance and address the specific challenges observed with arabic models.