CNLP - NITS - PP at G en AI Detection Task 1: AI -Generated Text Using Transformer-Based Approaches (2025)

Abstract: In the current digital landscape, distinguishing between text generated by humans and that created by large language models has become increasingly complex. This challenge is exacerbated by advanced LLMs such as the Gemini, ChatGPT, GPT-4, and LLaMa, which can produce highly sophisticated, human-like text. This indistinguishability introduces a range of challenges across different sectors. Cybersecurity increases the risk of social engineering and misinformation, while social media aids the spread of biased or false content. The educational sector faces issues of academic integrity, and within large, multi-team environments, these models add complexity to managing interactions between human and AI agents. To address these challenges, we approached the problem as a binary classification task using an English-language benchmark COLING dataset. We employed transformer-based neural network models, including BERT, DistilBERT, and RoBERTa, fine-tuning each model with optimized hyperparameters to maximize classification accuracy. Our team CNLP-NITS-PP has achieved the 23rd rank in subtask 1 at COLING-2025 for machine-generated text detection in English with a Main Score F1 Macro of 0.6502 and micro-F1 score of 0.6876.

## introduction large language models (llms) represent a significant advancement in natural language processing (nlp), advancing development in applications such as machine translation, text analysis, text generation, and question answering @xcite . in academic, industrial, and everyday contexts, the increasing deployment of llm-powered applications, such as chatgpt 1 , highlights their transformative potential. however, this rapid integration also underscores the importance of understanding their capa-1 https://chatgpt.com/ bilities and limitations to manage expectations and address ethical, societal, and technical challenges effectively @xcite . detecting ai-generated text (agt) focuses on leveraging artificial intelligence (ai) to identify and distinguish content produced by ai from @xcite . this area has gained significant importance due to the rapid advancements in deep learning (dl), which have enabled widespread applications of agt in content creation, virtual assistants, and more. however, these developments also introduce challenges, including the propagation of misinformation, potential privacy violations, and ethical risks @xcite . consequently, agt detection has emerged as a critical domain in ai research, aimed at mitigating these challenges and ensuring accountability in using generative ai technologies @xcite . agt detection research has emerged as a critical area within nlp, driven by advancements in dl. the introduction of robust models, such as recurrent neural networks (rnn) @xcite , long short-term memory networks (lstm) @xcite , and transformers @xcite , has significantly enhanced ai capabilities in text generation. these models now produce high-quality content, including articles, dialogues, and news reports. however, their misuse poses substantial risks, such as disseminating misinformation, deception of readers, and propagation of harmful content. dl plays a pivotal role in generating agt. dl models can produce realistic and coherent text by analyzing linguistic patterns and structures through training on extensive datasets. pre-trained language models (plms) based on the transformer architecture (e.g., gpt-3, bert) have demonstrated exceptional performance across various nlp tasks, contributing significantly to agt development @xcite . however, the widespread adoption of dl tech-niques for agt generation has raised several challenges. these include the potential for spreading misinformation, such as ai-generated fake news and deceptive advertisements that could influence public opinion @xcite . additionally, personal data may be exploited to generate misleading or targeted fraudulent content @xcite . furthermore, dl-powered ai can be misused to create inappropriate material, including violent, pornographic, or hate speech content, which may be widely disseminated @xcite . numerous researchers are developing strategies to detect and identify problematic content to address the challenges associated with agt. these strategies include rule-based and statistical approaches, as well as ml techniques like support vector machines (svm) and random forests (rf), which are commonly used for building detection models @xcite . additionally, combining these techniques with dl models, such as those based on the transformer architecture, is being explored to improve detection accuracy. this paper introduces a tool designed to detect llmgenerated ai text using transformer-based models to improve detection accuracy and provide insights for future research. ## related work various commercial and open-source tools, such as gptzero, zerogpt foot_0 , ai content detector, and gpt-2 output detector @xcite , have emerged to detect ai-generated content effectively. additionally, active research focuses on curating specialized datasets and determining which features and classifiers can enhance classification performance. for example, @xcite compiled a dataset of human and ai-generated abstracts to assess commercial and non-commercial detection systems, though the dataset is currently limited to english. recent studies have experimented with different detection methodologies, such as using xg-boost classifiers @xcite , decision tree algorithms @xcite , and transformer-based models @xcite . notably, analyzed text from english customer reviews, developing a transformer-based classifier that achieved a classification accuracy of 79%. these efforts demonstrate a trend toward optimiz-3 proposed methodology ## conclusion our approach to the mgt detection task 1 effectively combined distilbert's semantic embeddings with rule-based linguistic features and hyperparameters. this hybrid strategy enhanced