L ux V eri at G en AI Detection Task 3: Cross-Domain Detection of AI -Generated Text Using Inverse Perplexity-Weighted Ensemble of Fine-Tuned Transformer Models (2025)

Abstract: This paper presents our approach for Task 3 of the GenAI content detection workshop at COLING-2025, focusing on Cross-Domain Machine-Generated Text (MGT) Detection. We propose an ensemble of fine-tuned transformer models, enhanced by inverse perplexity weighting, to improve classification accuracy across diverse text domains. For Subtask A (Non-Adversarial MGT Detection), we combined a fine-tuned RoBERTa-base model with an OpenAI detector-integrated RoBERTa-base model, achieving an aggregate TPR score of 0.826, ranking 10th out of 23 detectors. In Subtask B (Adversarial MGT Detection), our fine-tuned RoBERTa-base model achieved a TPR score of 0.801, securing 8th out of 22 detectors. Our results demonstrate the effectiveness of inverse perplexity-based weighting for enhancing generalization and performance in both non-adversarial and adversarial MGT detection, highlighting the potential for transformer models in cross-domain AI-generated content detection.

## introduction the proliferation of advanced language models such as @xcite and roberta @xcite , machine-generated content has become prevalent across social media, journalism, and academia, raising concerns about authenticity and misinformation. detecting ai-generated text is especially challenging across diverse domains, where variations in language and style can hinder detection efforts. in task 3 of the coling 2025 workshop on detecting ai-generated content @xcite , we tackle cross-domain machine-generated text (mgt) detection using an ensemble approach that combines fine-tuned roberta-base models @xcite and openai detection tools @xcite . our method leverages inverse perplexity weighting to enhance the contributions of high-confidence models, yielding a robust detection system. our approach achieved an aggregate score of 0.826 in non-adversarial cross-domain mgt detection (subtask a), ranking 10th, and 0.801 in adversarial cross-domain mgt detection (subtask b), ranking 8th. this paper outlines our ensemblebased methodology, dataset considerations, and insights for effective cross-domain ai-generated text detection. ## system overview we developed an ensemble approach for aigenerated text detection across multiple domains, using transformer models with inverse perplexitybased weighted voting for improved accuracy. the system overview is shown in figure 1 . ## results ## limitations our approach, while effective, has several limitations. focusing on roberta models for finetuning and ensemble weighting excluded alternatives like @xcite and xlm-roberta @xcite , which might better handle longer sequences, noisy data, and multi-label tasks. due to computational constraints, we trained on a subset of the raid dataset, limiting the model's ability to capture its full diversity. training on the full dataset could greatly improve detection performance, especially for underrepresented domains. performance variability across generator models (e.g., gpt-4 vs. mistral) and limited multilingual capabilities highlight the need for better cross-domain generalization and robust multilingual detection. while the ensemble approach enhanced generalization, it increased computational overhead, warranting exploration of more efficient strategies in future work. ## discussion and conclusion in this paper, we proposed an ensemble-based approach for cross-domain mgt detection, combining fine-tuned roberta base and roberta base openai detectors with inverse perplexity weighting. our method achieved competitive results, ranking 10th and 8th in non-adversarial and adversarial tasks, respectively, in task 3 of the genai content detection workshop at coling-2025. inverse perplexity weighting improved generalization by prioritizing more confident models across diverse domains. for non-adversarial tasks, we explored an inverse perplexity-based ensemble approach. however, the detectors in this ensemble underperformed compared to the fine-tuned roberta model, highlighting the value of fine-tuning on task-specific data and suggesting avenues for refining ensemble techniques. our results show that transformer-based models, particularly roberta, are effective for nonadversarial and adversarial mgt detection. for non-adversarial detection (subtask a), we achieved a score of 0.826, and for adversarial detection (subtask b), we scored 0.801. however, cross-domain detection remains challenging, especially with varied generator models and multilingual data. our system performed well with generators like chat-gpt and gpt-4 but struggled with others like cohere and mistral, indicating the difficulty of detecting diverse machine-generated content. due to limited computational resources, we trained on a subset of the available data. despite this, our models performed well, demonstrating the potential of our approach even with partial data. this work lays the foundation for further progress in mgt detection, especially in adversarial and cross-lingual settings. future research can focus on enhancing multilingual capabilities, incorporating more diverse language models, and exploring dynamic ensemble strategies to improve performance across domains and attack scenarios.