Advacheck at G en AI Detection Task 1: AI Detection Powered by Domain-Aware Multi-Tasking (2025)

Abstract: The paper describes a system designed by Advacheck team to recognise machine-generated and human-written texts in the monolingual subtask of GenAI Detection Task 1 competition. Our developed system is a multi-task architecture with shared Transformer Encoder between several classification heads. One head is responsible for binary classification between human-written and machine-generated texts, while the other heads are auxiliary multiclass classifiers for texts of different domains from particular datasets. As multiclass heads were trained to distinguish the domains presented in the data, they provide a better understanding of the samples. This approach led us to achieve the first place in the official ranking with 83.07% macro F1-score on the test set and bypass the baseline by 10%. We further study obtained system through ablation, error and representation analyses, finding that multi-task learning outperforms single-task mode and simultaneous tasks form a cluster structure in embeddings space.

## introduction with the continuous improvement of large language models (llms), the task of detection machine-generated texts demands more and more attention from the community. the potential cases of misuse include malicious usage by students @xcite and scientists @xcite . furthermore, this is often the cause of plagiarism @xcite and spam @xcite . the mentioned things are encouraging researchers to improve methods for detecting artificial text simultaneously with enhancing generation methods. the task of detecting machine-generated texts is usually formulated as a binary text classification task @xcite . the most common solutions are to fine-tune the transformerbased model @xcite or to use zero-shot approaches with intrinsic statistics of the text @xcite . while these methods perform well on in-domain tasks @xcite , they are not robust to change of the domain, generator model, or language of the texts @xcite @xcite . meanwhile, for the detection of ai-content in the wild such a change is, on the contrary, a more realistic setup @xcite . beyond this, the data presented for the detection task may be of poor quality, which also complicates the challenge of the detection task @xcite . therefore, the goal is to obtain a model that is robust to the presence of data of poor quality and with a lot of noise, and, in addition, to make this model capable to adapt to new domains. the high coherence and quality of writing achieved by modern llms makes it difficult to find a specific sample-counted qualitative feature by which it would be possible to create a hyperplane in the space of texts and separate generated from human-generated ones. one of the possible refinements of the representations from the singletask learning architecture is the multi-task learning (mtl) @xcite . it was also noted that systems with mtl architecture had achieved high results in the previous competitions @xcite , therefore we decided to utilize this approach in our work. in this paper we discuss our solution as the advacheck team at genai content detection task 1: english and multilingual machine-generated text detection: ai vs. human @xcite . our method shows that with additional internal data analysis and embedding alignment using mtl, it is still possible to achieve high performance in detecting fragments in cross-domain and cross-generator setups on texts from the advanced llms. as we forced model to focus on various domains, it allowed us to form a cluster domain-wise structure for the text representations in the vector space. in our research, we show that (1) multi-task learning outperforms the default single-task, (2) cluster structure is formed at the shared encoder (3) compare different configurations of the system and (4) analyse the errors of the system. ## task definition the monolingual subtask of task 1: binary multilingual machine-generated text detection focuses on identifying whether the english text was entirely authored by a human or generated by a language model. the competition is the continuation and improvement of the semeval shared task 8 (subtask a) @xcite and combines refreshed training and testing samples from different domains and novel llms. the statistics of the dataset are summarised in appendix c. the official evaluation metric for the monolingual subtask is macro f 1 -score and the additional metric is micro f 1 -score. ## system overview why multi-task learning? in the current task formulation we have more than 600k texts in the training set with dozens of domains and generation models. such an amount is very noisy for a binary classification task, because it can be challenging for a model to distinguish between relevant and irrelevant features. multi-task learning may help the model focus on those features that actually matter as other tasks will update representation of samples with inner information. our aim is to obtain fine-grained representations of the data that ideally ignores data-dependent noise and generalises well. since different tasks involve distinct noises, a model trained on multiple tasks simultaneously is able to learn a more general representation. furthermore, it reduces the risk of overfitting. model. we propose an mtl architecture with hard parameter sharing (hps), it is depicted in figure 1. in hps, a common transformer-based encoder is used for multiple tasks. after several variations of set of parallel heads, we focused on three custom classification heads (cch) for simultaneous training: ## experiments we focused on the monolingual subtask, carrying out comparisons among models and ablations of the best system. for these we employed the original training and development splits provided by the organizers. our objective here was to reveal the quality improvement in multi-task training compared to single-task training. ## results our solution surpassed the claimed baseline by 10%. ## analysis 6.1 embeddings after mtl we made a comparison of text embeddings after fine-tuning stages. samples from development part of data were forwarded to the transformer-encoder and [cls] vectors were extracted as outputs. we visualised these vectors using pca in figure 2 . we observe that the alignment of the representations, which was mentioned above, introduces a cluster domain-wise structure. although the clusters are not perfectly separable, a meaningful difference between the standard bert-like model and mtl task head development test hc3 92.27 82.70 m4gt 91.70 81.07 mtl (hc3 + m4gt) 87.96 83.07 hc3 + m4gt + mage 91.43 79.23 ## conclusion in this paper we described the system by the advacheck team in the monolingual subtask at genai detection task 1 competition. we proposed solution with multi-task learning architecture that consists of shared transformer encoder and composition of one binary and two multiclass custom classification heads. our system obtained the best results in the official ranking bypassing the baseline by 10%. adding tasks for training in parallel reveal the formation of a cluster structure in the space of embeddings, helping to achieve high results despite the presence of a large amount of noisy data. also, it has been demonstrated that training a similar model but in single-task mode loses to the proposed approach, and configurations with one or three multiclass heads also perform worse than our final system.