The Million Authors Corpus: A Cross-Lingual and Cross-Domain W ikipedia Dataset for Authorship Verification (2025)

Abstract: Authorship verification (AV) is a crucial task for applications like identity verification, plagiarism detection, and AI-generated text identification. However, datasets for training and evaluating AV models are primarily in English and primarily in a single domain. This precludes analysis of AV techniques for generalizability and can cause seemingly valid AV solutions to, in fact, rely on topic-based features rather than actual authorship features. To address this limitation, we introduce the Million Authors Corpus (), a novel dataset encompassing contributions from dozens of languages on Wikipedia. It includes only long and contiguous textual chunks taken from Wikipedia edits and links those texts to their authors. includes 60.08M textual chunks, contributed by 1.29M Wikipedia authors. It enables broad-scale cross-lingual and cross-domain AV evaluation to ensure accurate analysis of model capabilities that are not overly optimistic. We provide baseline evaluations using state-of-the-art AV models as well as information retrieval models that are not AV-specific in order to demonstrate ‘s unique cross-lingual and cross-domain ablation capabilities.

## introduction authorship verification (av) aims to determine whether two or more texts are written by the same author. av has attracted significant attention from researchers because of its practical and diverse applications, such as identity verification, accountlinking, historical linguistics, forensic analysis, and ai-generated text detection. despite various approaches having been explored for av, the stateof-the-art (sota) models mainly focus on a datadriven representation learning approach @xcite @xcite . although these representation models demonstrate promising * denotes equal contribution performance on large-scale av on various datasets @xcite @xcite @xcite , they are limited mainly to av on english texts in a single domain because most dataset studies are single-domain and english-only. these constraints on domain and language further hinder the study of cross-lingual and cross-domain av, which requires texts written by the same author but in different languages and/or domains. the objective of this study is to research and address these limitations. to do so, we introduce the million authors corpus (mac )-a large dataset of content from wikipedia that covers 60 languages. 1 the potential of wikipedia as a corpus for linking textual content to its contributing users in multiple languages and domains has remained underexplored. unlike existing datasets, mac contains texts in dozens of languages and four different domains, as well as authors writing in multiple languages and domains. mac enables a more comprehensive evaluation of av models, which is impossible with existing datasets. in addition, mac is suitable for training and evaluating other nlp tasks such as text-style transfer @xcite , and semantic-shift @xcite . we choose to base mac on wikipedia since: (i) wikipedia is one of the largest public textual repositories in the world @xcite @xcite the association between an editor and a page-edit is transparent, verified, and maintained by the wikimedia foundation @xcite ; and (iii) wikipedia allows its users multiple ways to contribute and communicate, such as editing article pages foot_1 or discussing various topics over talk pages. users' writing styles and topics widely differ between these page types. wikipedians are also welcome to contribute to multiple wikipedia projects (i.e., languages). we make use of this and collect data from the different wikipedia page types and languages to enable cross-domain and cross-language evaluation. therefore, mac consists of users that contribute to multiple wikipedia namespaces (which we refer to as domains) as well as multilingual users. we develop a novel pipeline to extract substantive contributions from wikipedia in any language. these contributions, which we refer to as text chunks, must be long and contiguous new textual edits. we associate each text chunk with its corresponding author (i.e., a wikipedian), we henceforth refer to as an author. due to wikipedia's diversity, these chunks represent multiple genres, including encyclopedic text, social discussions, and debates, all across multiple topics and languages. including only long enough contributions makes mac practical and useful for av models and makes the conclusions from av experiments meaningful. figure 1 illustrates the creation-evaluation flow of mac . overall, mac contains more than 60.08m text chunks from 60 wikipedia languages. 1.29m contributers made at least two contributions to be included in mac . it contains more than 560k authors who have substantial contributions to more than a single domain and more than 250k authors who have substantial contributions in more than a single language. these cross-domain and crosslingual characteristics of mac makes it valuable for various nlp tasks, primarily av. to demonstrate the value of mac , we conduct an exploratory study and show that mac could help answer five fundamental research questions (rqs) in av. rq1: does an av model trained on one language and domain perform well on unseen authors in the same language and domain? rq2 (respectively, rq3): does an av model trained on one language (respectively, domain) perform well on unseen authors on other languages (respectively, domains)? rq4 (respectively, rq5): given a text written by an author, could an av model find another text written by the same author in another language (respectively, domain)? the first three rqs are answerable using existing datasets, while the latter two are only answerable with mac . notably, our results on four av models show that their performance differences are generally consistent across the first two rqs but starkly different across the latter three rqs, including the two only assessable by mac , confirming the necessity of the cross-lingual and cross-domain evaluation powered by mac . to summarize, in this study, we make the following contributions: ## mac creation here, we detail how mac was built, illustrated by the three leftmost blocks in figure 1 . ## authorship verification experiments here, we introduce the av tasks and experiments. ## results in this section, we present the evaluation results for all av models on all five rqs. due to space limitations, we mainly discuss the results for av models trained on english data. please see appendix section e for results over the rest of the languages. the results for rqs 1 to 5 are shown in bar groups 1 to 5 in figure 3 , respectively. rq1: training on in-language in-domain data improves av model performance, as shown in the first bar group in figure 3 ; here, the two fine-tuned models (sbert av and sadiri) perform better than both ir baselines (bm25 and sbert), and between the two fine-tuned models, sadiri outperforms sbert av , which aligns with our expectation because we expect the hard negative batching of sadiri benefits the model when evaluated on the same distribution as training data. rq2: trained av models can generalize to unseen languages, as seen in the second bar group in figure 3 . the trend also closely resembles the patterns observed in rq1, but the performance gain over bm25 is slightly less than that in rq1, which indicates that while the fine-tuned models are generalizable to other languages, the improvement in out-of-language performance is relatively smaller. the fine-grained results on the top five languages in figure 4 also indicate that for each evaluation language, the models trained in the same language perform the best or just worse than the model trained in english data in most cases, which falls within our expectation because the english subset of mac contains the most data. ## discussion overall, as seen in figure 3 , mac enables the comparison of av models under conventional scenarios such as those on the left (rqs 1 to 3). the results indicate that fine-tuned av models perform better than ir baselines for av in the same language and domain as training. fine-tuned models also generalize well to av in the same domain but unseen languages, but do not generalize to unseen domains at all (rq3, 3 rd bar group in figure 3 ). over the other 2 rqs, which are only assessable using mac , the results are starkly different on rq4, whereas the pattern on rq5 closely resembles that in rq3. in general, the results indicate that av models' performances are not consistent across different evaluation conditions, so to have a more comprehensive understanding of the capability of av models, testing them under different conditions is necessary. existing datasets can cover some conditions, but mac enables the evaluation of other situations like cross-lingual av and cross-domain av in different languages. moreover, the poor performance of the finetuned models in rqs 3 to 5 reveals challenges in av involving multiple domains and multiple languages, which may serve as new directions for future research in this area, and mac , as it contains high volumes of data from multiple domains and languages, enables researchers to build models to tackle these specific aspects of av. while building and evaluating mac , we paid extra attention to excluding translation cases. furthermore, a practical way to avoid the overrepresentation of accounts that are shared by multiple people is to limit the number of samples we use per author (while running experiments). 12 we use this approach in our research and extract only one pair of texts for each author in each language. ## related work we first summarize av-related research and then discuss existing wikipedia corpora used in nlp. av models with the development of nlp in the past few decades, the techniques in av have evolved through three main stages. in the preneural era, av works mainly focused on featurebased approaches @xcite @xcite . later, with the widespread use of neural networks (nn), especially (pre-trained) transformers, nn-based representation models have become mainstream and demonstrated superior performance on large-scale av @xcite @xcite . recently, leveraging the strong zero-and few-shot capability of llms, some studies showed that llms could achieve a decent performance on av without further downstream fine-tuning, but the findings are limited to small-scale av @xcite . in this work, we concentrate on large-scale av due to its higher practicality and, therefore, primarily adopt nn-based representation models in our experiments. av datasets datasets also play a crucial role in av research for both model training and evaluation. av requires corpora with authorship labels which could be collected from various sources such as email @xcite , newswire agency @xcite , newspaper @xcite . data from these sources often contain a relatively small number of authors. with the increasing number of internet users, later works started to collect data from public websites such as amazon @xcite and reddit @xcite . datasets from these sources are substantially larger in scale and serve as the foundation of the sota av models @xcite . these datasets contain up to one million authors @xcite and texts in six languages @xcite , while mac is 10 times larger in terms of both the numbers of authors and languages. moreover, mac also contains texts from four domains and authors writing in multiple languages and domains, enabling a more comprehensive evaluation of the av models presented in section 3.2.3. wikipedia corpora wikipedia serves the research community as a textual source in various domains and nlp tasks @xcite @xcite @xcite . a series of studies have used information about wikipedia editors to create data sets that associate text with authors @xcite @xcite and others have leveraged edit histories for a variety of purposes such as simplification @xcite or grammatical error correction @xcite . these and other wikipedia-related corpora have been released under the 'cc by-sa' wikipedia license terms, which we also rely on while releasing mac to the research community. most wikipedia-related studies focus on a single language, usually the most popular ones, such as english or german. while some others @xcite ) leverage wikipedia's multilingual capability, none tackle the broad combination of languages and domains as we do in mac by extracting long textual content from edits done by wikipedians. ## summary our novel million authors corpus (mac ) spans 60 languages and four domains, linking long textual contributions to their respective authors while excluding minor edits. mac consists of more than 60m cleaned text passages from contributions by wikipedian authors. here, we study av models using the 1.29m authors who made 2+ contributions in mac across the 60 languages; more than 560k authors contributed in different domains, such as an article page and a talk page. in addition, mac includes multilingual authors who contributed to at least two languages. mac enables new studies of cross-lingual and cross-domain av evaluations; i.e." given a text written by an author, finding another text by the same author written in a different language or domain. our study demonstrates that in conventional av scenarios, a selection of fine-tuned models surpass ir baselines on rq1 and rq2, but not on rq3. however, we show that in new novel and challenging settings of cross-lingual and cross-domain av (rq4 and rq5), we observe starkly different patterns and models, as a whole, perform substantially lower. our results highlighting the value of the comprehensive av model evaluation powered by mac and point to the future research directions of av in cross-domain and cross-lingual modeling.