CNLP - NITS - PP at G en AI Detection Task 3: Cross-Domain Machine-Generated Text Detection Using D istil BERT Techniques (2025)

Abstract: This paper presents a Cross-domain Machine-Generated Text Detection model developed for the COLING 2025 Workshop on Detecting AI-generated Content (DAIGenC). As large language models evolve, detecting machine-generated text becomes increasingly challenging, particularly in contexts like misinformation and academic integrity. While current detectors perform well on unseen data, they remain vulnerable to adversarial strategies, including paraphrasing, homoglyphs, misspellings, synonyms, whitespace manipulations, etc. We introduce a framework to address these adversarial tactics designed to bypass detection systems by adversarial training. Our team DistilBERT-NITS detector placed 7th in the Non-Adversarial Attacks category, and Adversarial-submission-3 achieved 17th in the Adversarial Attacks category.

## introduction large language models (llms) @xcite have quickly established themselves as transformative tools in natural language processing (nlp). these models gain substantial internal knowledge by undergoing extensive pre-training on massive datasets in a selfsupervised manner, enabling them to excel in various tasks, from answering factual queries and generating coherent text to handling intricate reasoning processes. this versatility has brought substantial advancements across various nlp application areas. despite these advancements, ethical concerns have surfaced regarding inherent risks @xcite @xcite , such as the potential for misinformation, hallucinations in generated outputs, and even biases against certain groups. growing awareness of these issues has spurred research into detecting ai-generated text. however, ai-text detectors may carry similar vulnerabilities as neural network models @xcite , inspiring related studies @xcite that explore paraphrasing attacks designed to deceive detector predictions. it is contended that examining potential adversarial attacks on text detectors is crucial, as weaknesses in ai detection systems can be identified before deployment in practical settings, such as academic plagiarism detection, thereby supporting the development of effective countermeasures. current detection methods are generally classified into three main categories: statistical approaches @xcite that use metrics like entropy, perplexity, and log-likelihood; neural classifiers @xcite trained on supervised datasets labeled as human or ai-generated; and watermarking techniques @xcite that embed subtle patterns into ai-generated text. however, research on adversarial perturbations specifically targeting ai-text detectors still needs to be completed. for example, @xcite investigated paraphrasing to alter machine-generated text (mgt) in adversarial attacks, while @xcite employed llms to create adversarial word candidates through a search-based approach. although these studies have shown that ai detectors can be vulnerable to adversarial modifications, the impact of such attacks on detector performance in complex, real-world conditions is still largely unexamined. ## related work most research on adversarial attacks has focused on image detection @xcite @xcite , as text data presents unique challenges due to its discrete structure, making it harder to create imperceptible modifications compared to image data, where subtle pixel changes can go largely unnoticed @xcite . some general text classification adversarial attacks, such as those by @xcite , have demonstrated this. recently, studies have turned toward adversarial attacks on neural text detectors: @xcite found that introducing minor spelling errors and homoglyph replacements can significantly lower detection rates for gpt-2-generated text. similarly, @xcite showed that character-level perturbations also affect roberta-based detectors @xcite further revealed that existing detectors are vulnerable to simple rephrasing and may even mistakenly label texts written by non-native speakers as aigenerated. due to the susceptibility of current methods to adversarial attacks, several researchers have proposed approaches to enhance their robustness, including work by @xcite . although watermarking techniques have also been explored for identifying ai-generated text, they are generally considered vulnerable to adversarial tactics, particularly those based on mutation and paraphrasing @xcite . ## conclusion in this study, we developed a robust framework using a fine-tuned distilbert-nits model to detect mgt across diverse domains, focusing on adversarial scenarios. our approach ranked 7 th in non-adversarial detection and 17 th in adversarial detection at the coling workshop, involves preprocessing text to mitigate detected adversarial manipulations, enhancing detection accuracy. these findings support the potential of lightweight models to handle adversarial and cross-domain mgt detection effectively. future work will be focused on refining this method to improve robustness and adaptability against evolving adversarial tactics.