MOSAIC : Multiple Observers Spotting AI Content (2025)

Abstract: The dissemination of Large Language Models (LLMs), trained at scale, and endowed with powerful text-generating abilities, has made it easier for all to produce harmful, toxic, faked or forged content. In response, various proposals have been made to automatically discriminate artificially generated from human-written texts, typically framing the problem as a binary classification problem. Early approaches evaluate an input document with a well-chosen detector LLM, assuming that low-perplexity scores reliably signal machine-made content. More recent systems instead consider two LLMs and compare their probability distributions over the document to further discriminate when perplexity alone cannot. However, using a fixed pair of models can induce brittleness in performance. We extend these approaches to the ensembling of several LLMs and derive a new, theoretically grounded approach to combine their respective strengths. Our experiments, using a variety of generator LLMs, suggest that this approach effectively harnesses each model’s capabilities, leading to strong detection performance on a variety of domains.

## introduction large language models (llms) have greatly improved the fluency and diversity of machinegenerated texts. the release of chatgpt and gpt4 by openai has sparked global discussions regarding the new opportunities offered by ai-based writing assistants. these advances have also introduced considerable threats such as fake news generation @xcite , and the potential for harmful outputs such as toxic or dishonest content @xcite , among others. as it seems, the research on methods to detect the origin of a given text to mitigate the dissemination of forged content and to prevent technology-aided plagiarism still using llama models : using falcon models : lorem ipsum dolor sit amet, consectetur adipiscing elit. ut sed purus luctus, rutrum orci in, elementum erat. lags behind the rapid advancement of ai itself. 1 many studies have focused on tools that could spot such ai-generated outputs and mitigate these underlying risks. from a bird's eye view, this typically involves using detector models to discriminate generator models' outputs from legitimate human writings. multiple versions of this generic text classification task have been considered, varying, e.g. the number of possible categories to distinguish and the amount of supervision (see section 2). owing to its large user base and applications, the largest effort has focused on one specific generator, chatgpt, for which training and test data are easily obtained. yet, the corresponding supervised binary problem, with a unique known generator, is not the only way to frame this task. a more challenging problem, that we study here, is generator-agnostic artificial text detection, where the models to be detected are not predefined. our contributions. in this paper, using fundamental information-theoretic principles from universal compression, we derive a new ensemble method (depicted in figure 2 ) that combines the strength of multiple llms into a single score to detect forged texts. by using several models, this approach dispenses with the need to look for the optimal set of detector models, and thus does not require a validation dataset. our experiments use both standard benchmarks comprising multiple domains and languages, as well as a new corpus of machine-generated texts. they confirm that ensembling strong llms yields detectors that can robustly identify a multiplicity of generators and that compare favourably with several recent proposals using a predefined set of detector models. we also report a number a complementary analyses exploring the effect of incrementally growing the ensemble, of considering adapted versions of one single model, of using a naive ensembling method, and of processing noised versions of artificial texts. we publicly release the code and data produced for this study. supervised methods. supervised detection with a single generator often achieves detection accuracy rates in the high 90s @xcite @xcite , using classifiers based on roberta @xcite or t5 @xcite . however, these approaches are brittle and their success depends on particular generatordetector pairs @xcite , prompting e.g. @xcite to design automatic feature extractors from multiple detectors to improve the robustness of their system. zero-shot methods. unsupervised detection is more challenging. most approaches rest on the idea that human-written texts are more "surprising" than artificial texts foot_1 , leading to a difference in token-wise log-probability foot_2 . this idea is used in gptzero foot_3 and thresholding perplexity usually provides strong baselines (see, inter alia, @xcite @xcite ). such techniques heavily rely on the detector model(s) used to compute the log-probabilities of input texts, which must be robust to variations in domains, genres, styles, and languages @xcite ; and to variations in the generator itself @xcite . perturbation-based. @xcite and @xcite exploit a similar intuition, arguing that small random perturbations of an artificial text will on average make it less likely, unlike human-written texts. they develop a statistical criterion based on the curvature of the log-probability function and achieve near-perfect detection scores across three types of texts generated by five different models. the binoculars score of @xcite also relies on a function of the per-token log-perplexity, contrasted with the cross-entropy of an auxiliary model. these valuable works point to the over-reliance on one specific detector model as a major limitation of the state-of-the-art. our proposed mitigation relies on ensemble techniques, that are also considered in the supervised detection setting, e.g. in @xcite @xcite . other methods. abandoning generator-detector based techniques altogether, @xcite develop effective detection approaches based on regeneration, prompting the (known) generator to regenerate part of the input text. the intuition is that artificial inputs are likely to be regenerated exactly, unlike human texts. other strategies include text watermarking @xcite @xcite , though its efficiency and robustness are still subject to discussions, e.g., @xcite . ## introducing mosaic building on the principles used in previous systems, we now present mosaic, a scoring method designed to leverage multiple models simultaneously. the key difference compared to previous approaches is that instead of using a single fixed llm for q, mosaic defines it as a positiondependent mixture of all models in the ensemble. the weights of this mixture are formally defined in the next proposition and depicted in figure 2 . proposition 1 (optimal model). the optimal model, q ⋆ -which minimizes the encoding length for tokens-is the position-dependent mixture: where the distribution µ ⋆ (•|@xmath0: furthermore, the weights {µ ⋆ (m|@xmath1; they can be efficiently computed using the well-known blahut-arimoto algorithm @xcite . definition 1 (mosaic score). for an input sentence @xmath2, y 1 , . . .⟩, and models indexed by @xmath3, . . . , m } sharing a common tokenizer, the mosaic score is then defined as: a formal description of how this scoring system is implemented is provided in algorithm 1. raid contains about 15k natural texts in english from a variety of domains; the artificial part version contains approximately 500k, generated with a diverse set of recent models, also varying the sampling procedure. as the test set is not publicly released, we select a random subset of 1,000 human texts and their generated counterparts with 11 different models and ancestral sampling for our experiments. raid also includes an artificially noised subcorpus, we report the results for the different adversarial attacks in we augmented the raid extracts with texts generated using models that are in our ensembles, 7 using the same prompts as in the raid dataset. we refer to this augmented raid as raid+. these datasets, presented in ## conclusion through all of our experiments, we have shown that the mosaic method effectively combines all the models in the ensemble, achieving very strong results across all datasets and languages. this method has multiple advantages: it is fully unsupervised, dispenses with the search for the optimal detector(s) when several are available, while offering a scalable solution that can incorporate a growing number of models. even adversarial attacks have only a minimal effect on the performance, despite our scoring system not being optimized for perturbations. however, mosaic is currently computationally costly, as each model must run on the input text. while not in the main scope on this paper, a potential improvement of computational efficiency is proposed in section 6.4, and further solutions are discussed in appendix e. furthermore, more work needs to be done in order to fully understand how to evaluate the "distance" in-between models' outputs, in order to choose the best ensemble that would cover all potential generations. the optimal mixture in mosaic, defined by blahut-arimoto weights to minimize encoding length in worst-case scenarios, could be useful beyond detecting machine-generated texts. we leave this exploration for future work.