L ux V eri at G en AI Detection Task 1: Inverse Perplexity Weighted Ensemble for Robust Detection of AI -Generated Text across E nglish and Multilingual Contexts (2025)

Abstract: This paper presents a system developed for Task 1 of the COLING 2025 Workshop on Detecting AI-Generated Content, focusing on the binary classification of machine-generated versus human-written text. Our approach utilizes an ensemble of models, with weights assigned according to each model’s inverse perplexity, to enhance classification accuracy. For the English text detection task, we combined RoBERTa-base, RoBERTa-base with the OpenAI detector, and BERT-base-cased, achieving a Macro F1-score of 0.7458, which ranked us 12th out of 35 teams. We ensembled RemBERT, XLM-RoBERTa-base, and BERT-base-multilingual-case for the multilingual text detection task, employing the same inverse perplexity weighting technique. This resulted in a Macro F1-score of 0.7513, positioning us 4th out of 25 teams. Our results demonstrate the effectiveness of inverse perplexity weighting in improving the robustness of machine-generated text detection across both monolingual and multilingual settings, highlighting the potential of ensemble methods for this challenging task.

## introduction the rapid advancement of language models such as gpt @xcite and bert @xcite has increased machine-generated content, raising significant concerns about misinformation and academic integrity. identifying ai-generated text becomes more challenging in multilingual contexts, where linguistic diversity adds further complexity to model generalization. while existing approaches perform well in english, their effectiveness decreases when applied to languages with diverse syntactic and semantic structures. in task 1 of the coling 2025 workshop on detecting ai-generated content @xcite , we propose an ensemble-based solution to address these issues. for english detection, we combine roberta-base @xcite , openai's ai text detector @xcite , and bertbase-cased @xcite . for multilingual detection, we integrate @xcite , xlm-roberta-base @xcite , and bert-base-multilingual-cased @xcite . to further improve performance, we incorporate inverse perplexity weighting to give greater priority to models that produce lower perplexity scores. our ensemble approach achieved a macro f1-score of 0.7458 (micro f1: 0.7568) in english, placing us 12th out of 35 teams, and a macro f1-score of 0.7513 (micro f1: 0.7527) for the multilingual tasks, ranking 4th out of 25 teams. we encountered several challenges during this work. one major issue was data imbalance, as human-written content vastly outnumbered aigenerated samples. to address this, we employed data augmentation and optimized our sampling strategies. another challenge involved ensuring the models' generalization across different languages and writing styles, often with limited training data. this highlights the importance of additional finetuning and the need to explore alternative architectures that can better handle diverse linguistic inputs. this paper presents a robust ensemble approach for detecting ai-generated content, with strong performance across both english and multilingual tasks. however, significant opportunities remain for improving model generalization and addressing data imbalance, which will be crucial for future advancements in this field. the following sections will discuss the dataset, methodology, results, a detailed analysis of the findings, and conclusions drawn from this study. ## system overview we developed an ensemble approach for aigenerated text detection across english and multilingual contexts, using transformer models with a weighted voting strategy based on inverse perplexity for improved accuracy. the system overview is shown in figure 1 . ## results this section presents the performance of our ensemble approach for task 1 at the coling 2025 workshop on detecting ai-generated content, evaluated using the macro f1-score. detailed results are shown in the baseline scores provided by the task organizers @xcite used roberta-base @xcite for the english track and xlm-roberta-base @xcite for the multilingual track. these scores serve as benchmarks for our ensemble method. during testing, models in the ensemble were weighted based on perplexity, with lowerperplexity models given greater influence. this approach, along with sorting test data by text length, reduced inference time by 40% while generating predictions using softmax and weighted averaging. for the english-only task, our ensemble of roberta-base, roberta-base openai detector, and bert-base-cased achieved a macro f1-score of 0.7458, outperforming the baseline (0.7342) and ranking 12th out of 35 teams. similarly, for the multilingual task, our ensemble of rem-bert, xlm-roberta-base, and bert-basemultilingual-cased achieved a macro f1-score of 0.7513, surpassing the baseline (0.7416) and ranking 4th out of 25 teams. the combination of these models effectively enhanced both languagespecific and cross-lingual accuracy. our ensemble also outperformed individual models, such as roberta-base (0.7342) for the english track and the dual combination of rembert + xlm-r (0.7435) for the multilingual track. this demonstrates the effectiveness of combining diverse models to achieve better performance. ## discussion and conclusion in this work, we presented an ensemble approach to detect ai-generated content across english and multilingual datasets. by combining multiple pre-trained models, including roberta-base, openai detector, bert-base-cased for english, and rembert, xlm-roberta-base, bert-basemultilingual-cased for multilingual tasks, and applying inverse perplexity weighting, our ensemble demonstrated strong performance. it achieved a macro f1-score of 0.7458 (micro f1: 0.7568) for english, ranking 12th, and 0.7513 (micro f1: 0.7527) for multilingual tasks, ranking 4th. compared to individual models, our ensemble consistently outperformed or matched their performance. for example, in the english task, the ensemble scored 0.7568, surpassing roberta + ope-nai detector (0.7381) and bert (0.7275). similarly, the multilingual task achieved 0.7527, exceeding rembert + xlm-r (0.7473) and rem-bert (0.7507). notably, our ensemble also outperformed baseline models, with a macro f1-score of 0.7458 for english (baseline: 0.7342 achieved by roberta) and 0.7513 for multilingual (baseline: 0.7416 achieved by xlm-roberta). these results highlight the effectiveness of combining model strengths to improve detection accuracy. a key challenge faced during the multilingual task was the data imbalance between wellrepresented languages like english and chinese and underrepresented ones such as urdu, arabic, and russian. this disparity hindered the model's accuracy for underrepresented languages. to address this, we scale down samples from overrepresented languages to balance the dataset. this adjustment improved performance across languages, validating the effectiveness of our approach. despite these successes, challenges persist. detecting ai-generated content in multilingual contexts remains complex and demands further refinement in model architectures and data processing techniques. future work could explore advanced methods for mitigating data imbalance, such as data augmentation or active learning, to enhance the model's generalization ability across diverse languages. additionally, more sophisticated ensemble strategies or domain-specific models could improve detection accuracy. in conclusion, this study demonstrates the effectiveness of an ensemble approach for detecting aigenerated content across english and multilingual datasets. addressing data imbalance and using inverse perplexity weighting improved performance, though ongoing challenges highlight the need for continuous innovation in ai detection systems.