MOSAIC at GENAI Detection Task 3 : Zero-Shot Detection Using an Ensemble of Models (2025)

Abstract: MOSAIC introduces a new ensemble approach that combines several detector models to spot AI-generated texts. The method enhances the reliability of detection by integrating insights from multiple models, thus addressing the limitations of using a single detector model which often results in performance brittleness. This approach also involves using a theoretically grounded algorithm to minimize the worst-case expected encoding size across models, thereby optimizing the detection process. In this submission, we report evaluation results on the RAID benchmark, a comprehensive English-centric testbed for machine-generated texts. These results were obtained in the context of the “Cross-domain Machine-Generated Text Detection” shared task. We show that our model can be competitive for a variety of domains and generator models, but that it can be challenged by adversarial attacks and by changes in the text generation strategy.

## introduction large language models (llms) have greatly improved the fluency and diversity of machinegenerated texts. the release of chatgpt and gpt4 by openai has sparked global discussions regarding the effective use of ai-based writing assistants. this progress has also introduced major threats related to the generation of fake news @xcite , of toxic or dishonest content @xcite , or more generally regarding misuses of machine generation abilities. in response, the automatic detection of such machine generated texts (mgt) has attracted a lot of recent work. from a bird's eye view, mgt detection uses detector models to discriminate generator models' outputs from human writings. multiple instances of this basic text classification problem have been considered, varying e.g. the number of possible categories to distinguish, the amount of available supervision or the granularity of the task (e.g. at the text, sentence, or even token level). owing to its large user base and applications, most efforts to date have focused on specifically detecting chatgpt, for which training and test data is easily obtained. a more difficult problem, that we study here, is unsupervised generator-agnostic artificial text detection, where the models to detect are not known in advance, and for which we also assume no training data. as pointed out e.g., in @xcite @xcite , the performance of mgt detection systems varies depending on the choices of the detector(s) / generator(s) pair. the detector may serve to assess probabilities, as in @xcite , or to regenerate content, as e.g., in @xcite . in most cases, optimal detection performance will require a systematic exploration of the space of possible detectors. as the number and diversity of llms keep increasing, such exploration seems not only challenging but also unrealistic. furthermore, @xcite demonstrated that the current detection methods are brittle and easily fooled by changing the generator or altering the associated sampling method, a finding that we reproduce in this study. in an attempt to increase the robustness of existing detectors, we consider here ensemble methods, where a coalition of several models is exploited to build the detector. for this, we generalize perplexity-based approaches, which flag as "artificial" texts having a suspiciously small perplexity. as perplexity is also an encoding measure, our ensemble technique will seek to identify time-varying mixture models, in order to minimize the worstcase expected encoding size. the corresponding architecture is in figure 1 . further details, explanations, and proofs can be found in a companion paper @xcite . not only is this method fully unsupervised, it also dispenses with the need to search for the best detector(s). this method nonetheless helps to develop mgt detection systems that can robustly detect multiple generators. in this short contribution, we briefly present the detection task, then our detection algorithms, before discussing and analyzing the experimental results. ## our method : mosaic language models predict the probability of a token conditioned on the preceding ones, thus defining a probability distribution over the set of all possible sequences. the probability of generating a sequence @xmath0, y 1 , . . . , y t ⟩ is computed as the product of conditional probabilities for each token, given its preceding context. a central concept in our method is information, which measures the "surprise" of observing a particular token for a model parameterized by θ. this surprisal is quantified aslog p θ (y t |@xmath1, where lower values indicate higher predictability. this is akin to compression in information theory: the lower the surprisal, the better the corresponding token can be compressed by the model p θ (.|@xmath2. instead of relying on a detector single model, as in most unsupervised methods, our method leverages a diverse set of llms, denoted as p m (y). the key idea is to assign each token in a sequence to the model that best "explains" it, i.e., the model that can compress it most effectively. given a sequence @xmath3, we combine the models logits to obtain q ⋆ t , the distribution minimizing the excess codelength w.r.t any distribution p θ ∈ p m . a larger score indicates that the text is likely human-written because the observed codelength is significantly larger than what would be expected from ai-generated content. conversely, a lower score suggests that the text closely resembles outputs from the models in m. since this competition requires the machine outputs to get higher scores, we simply inverted use a negated version of (1) before submission. more information about mo-saic can be found in @xcite .