The Million Authors Corpus: A Cross-Lingual and Cross-Domain Wikipedia Dataset for Authorship Verification
Authors: Abraham Israeli, Shuai Liu, Jonathan May, David Jurgens
Accepted at: acl25

The Million Authors Corpus: A Cross-Lingual and Cross-Domain Wikipedia Dataset for Authorship Verification

Abraham Israeli*1, Shuai Liu*2, Jonathan May2 and David Jurgens1
1University of Michigan
2Information Sciences Institute, University of Southern California
{isabrah, jurgens}@umich.edu, {liushuai, jonmay}@isi.edu

Abstract
Authorship verification (AV) is a crucial task for applications like identity verification, plagiarism detection, and AI-generated text identification. However, datasets for training and evaluating AV models are primarily in English and primarily in a single domain. This precludes analysis of AV techniques for generalizability and can cause seemingly valid AV solutions to, in fact, rely on topic-based features rather than actual authorship features. To address this limitation, we introduce the Million Authors Corpus (MAC), a novel dataset encompassing contributions from dozens of languages on Wikipedia. It includes only long and contiguous textual chunks taken from Wikipedia edits and links those texts to their authors. MAC includes 60.08M textual chunks, contributed by 1.29M Wikipedia authors. It enables broad-scale cross-lingual and cross-domain AV evaluation to ensure accurate analysis of model capabilities that are not overly optimistic. We provide baseline evaluations using state-of-the-art AV models as well as information retrieval models that are not AV-specific, in order to demonstrate MAC’s unique cross-lingual and cross-domain ablation capabilities.

1 Introduction
Authorship verification (AV) aims to determine whether two or more texts are written by the same author. AV has attracted significant attention from researchers because of its practical and diverse applications, such as identity verification, account-linking, historical linguistics, forensic analysis, and AI-generated text detection. Despite various approaches having been explored for AV, the state-of-the-art (SOTA) models mainly focus on a data-driven representation learning approach (Rivera-Soto et al., 2021; Zhu and Jurgens, 2021; Wegmann et al., 2022; Fincke and Boschee, 2024). Although these representation models demonstrate promising performance on large-scale AV on various datasets (Klimt and Yang, 2004; Schler et al., 2006; Seroussi et al., 2014; Ni et al., 2019; Bevendorff et al., 2020; Khan et al., 2021), they are limited mainly to AV on English texts in a single domain because most dataset studies are single-domain and English-only. These constraints on domain and language further hinder the study of cross-lingual and cross-domain AV, which requires texts written by the same author but in different languages and/or domains.

The objective of this study is to research and address these limitations. To do so, we introduce the Million Authors Corpus (MAC)—a large dataset of content from Wikipedia that covers 60 languages.1 The potential of Wikipedia as a corpus for linking textual content to its contributing users in multiple languages and domains has remained underexplored. Unlike existing datasets, MAC contains texts in dozens of languages and four different domains, as well as authors writing in multiple languages and domains. MAC enables a more comprehensive evaluation of AV models, which is impossible with existing datasets. In addition, MAC is suitable for training and evaluating other NLP tasks such as text-style transfer (Hallinan et al., 2023; Liu and May, 2024), and semantic-shift (Hamilton et al., 2016; Kutuzov et al., 2018).

We choose to base MAC on Wikipedia since: (i) Wikipedia is one of the largest public textual repositories in the world (Medelyan et al., 2009; Mesgari et al., 2015); (ii) The association between an editor and a page-edit is transparent, verified, and maintained by the Wikimedia Foundation (Ayers et al., 2008); and (iii) Wikipedia allows its users multiple ways to contribute and communicate, such as editing article pages2 or discussing various topics over

* denotes equal contribution

1MAC includes 60 languages, two of which are English. One of these is “simple English,” a simplified variant of standard English used as a separate Wikipedia project.
2Article pages are the content pages people normally think of as Wikipedia.
talk pages. Users’ writing styles and topics widely differ between these page types. Wikipedians are also welcome to contribute to multiple Wikipedia projects (i.e., languages). We make use of this and collect data from the different Wikipedia page types and languages to enable cross-domain and cross-language evaluation. Therefore, MAC consists of users that contribute to multiple Wikipedia namespaces (which we refer to as domains) as well as multilingual users.

We develop a novel pipeline to extract substantive contributions from Wikipedia in any language. These contributions, which we refer to as text chunks, must be long and contiguous new textual edits. We associate each text chunk with its corresponding author (i.e., a Wikipedian), we henceforth refer to as an author.

Due to Wikipedia’s diversity, these chunks represent multiple genres, including encyclopedic text, social discussions, and debates, all across multiple topics and languages. Including only long enough contributions makes MAC practical and useful for AV models and makes the conclusions from AV experiments meaningful. Figure 1 illustrates the creation-evaluation flow of MAC.

Overall, MAC contains more than 60.08M text chunks from 60 Wikipedia languages. 1.29M contributors made at least two contributions to be included in MAC. It contains more than 560K authors who have substantial contributions to more than a single domain and more than 250K authors who have substantial contributions in more than a single language. These cross-domain and cross-lingual characteristics of MAC makes it valuable for various NLP tasks, primarily AV.

To demonstrate the value of MAC, we conduct an exploratory study and show that MAC could help answer five fundamental research questions (RQs) in AV. RQ1: Does an AV model trained on one language and domain perform well on unseen authors in the same language and domain? RQ2 (respectively, RQ3): Does an AV model trained on one language (respectively, domain) perform well on unseen authors on other languages (respectively, domains)? RQ4 (respectively, RQ5): Given a text written by an author, could an AV model find another text written by the same author in another language (respectively, domain)? The first three RQs are answerable using existing datasets, while the latter two are only answerable with MAC. Notably, our results on four AV models show that their performance differences are generally consistent across the first two RQs but starkly different across the latter three RQs, including the two only assessable by MAC, confirming the necessity of the cross-lingual and cross-domain evaluation powered by MAC.

To summarize, in this study, we make the following contributions:

1. We introduce MAC—a novel corpus that relies on Wikipedia data. Its uniqueness is the extraction of long text chunks from Wikipedia and its linkage to the contributing author. MAC is cross-lingual and covers 60 languages. It is also cross-domain as we collect data from different Wikipedia page types (e.g., article and talk pages). We make MAC available for the research community. The full dataset is available in the Zeondo data repository.3 The concise data we use to train the presented models in the manuscript are also available in the Hugging-Face repository.4

2. Based on MAC, we conduct an exploratory experiment using four AV models to examine five fundamental RQs, two of which are unanswerable with existing datasets regarding cross-lingual and cross-domain AV.

3. The experimental results show that the performance of the models exhibits significantly different patterns on the two RQs only answerable by MAC compared to both the first two RQs and the third RQ. By this, we underlie the value of the cross-lingual and cross-domain characteristics of MAC.

2 MAC Creation

Here, we detail how MAC was built, illustrated by the three leftmost blocks in Figure 1.

3 https://zenodo.org/records/15538126
4 https://huggingface.co/datasets/Blablablab/MAC

Figure 1: MAC creation and usage. We process 60 languages from Wikipedia to create MAC. It contains 60.08M text chunks contributed by 1.29M users. Using MAC, we train authorship verification models and compare their performance against existing baselines in novel cross-lingual and cross-domain settings.
<table>
  <tr>
    <th>NS</th>
    <th>Description</th>
    <th># T.Chunks</th>
    <th>%</th>
  </tr>
  <tr>
    <td>0</td>
    <td>Article page</td>
    <td>30.75M</td>
    <td>51.18%</td>
  </tr>
  <tr>
    <td>1</td>
    <td>Article talk page</td>
    <td>5.60M</td>
    <td>9.33%</td>
  </tr>
  <tr>
    <td>2</td>
    <td>User page</td>
    <td>1.08M</td>
    <td>1.81%</td>
  </tr>
  <tr>
    <td>3</td>
    <td>User talk page</td>
    <td>22.65M</td>
    <td>37.69%</td>
  </tr>
</table>

Table 1: Wikipedia namespaces (NSs) that we include in MAC, which we refer to as ‘domains’ in this study. The two rightmost columns are the number and percentage of text chunks included in MAC.

2.1 Wikipedia Raw Data

Wikipedia maintains its content in multiple languages, each functioning as an independent project. In this study, we process the 60 languages with most of the Wikipedia content to get sufficient coverage. Appendix Table 3 contains a full list of the languages we process. The English Wikipedia is the largest and most active project, with over 6M articles and 48M registered editors. However, many other languages (e.g., French and German) have shown significant growth in recent years in both content and registered editors.5

Content in Wikipedia comes in multiple forms. Apart from collaboratively written article pages, each user has their own publicly readable ‘user page’. Further, for each article and user page, there is a ‘talk page’—a virtual space for the Wikipedia community to discuss, comment, and ask questions about relevant topics of the main article or user. Each of these four types of pages contains different content and writing styles. Wikimedia maintains different namespaces (NSs) for each, which we use throughout our study and call them domains. We use the Wikipedia numbering notation, which ranges from 0 to 3. While building MAC, we process all four domains, summarized in Table 1.

We collect the data of all Wikipedia pages using the Wikipedia dump files, which are released monthly by the Wikimedia Foundation.6 The Wikipedia dump files allow us to process all existing information on each page. This way, we can track and collect information about each revision of both the content pages and the talk pages. The dump files contain detailed information per revision, including time, editor name, textual content, etc. We use the April 2024 release of Wikipedia dump files for each of the 60 languages we process.

2.2 Raw Data Processing

A unique attribute of MAC is that it contains only substantial, long and contiguous contributions (i.e., text chunks). By doing so, we avoid the long tail of minor contributions, which are very common both in Wikipedia and in existing AV datasets and which, when included, call into question the validity of AV. In this section, we describe in detail how we extract these text chunks.

We process each Wikipedia project (i.e., language) independently. Our language-agnostic methodology makes it feasible to process Wikipedia pages in any language. To do that, we avoid using language-specific techniques that are not necessarily available across all languages (e.g., Arabic POS tagging). To process the textual content of each revision, we use existing Python packages built for Wikipedia (see Appendix Table 9).

In the remainder of the paper, we use the following notation. \( P \) is the set of Wikipedia pages that we process. For each page \( p \in P \), we mark its \( N \) sequence of sorted revisions as \( R_p^1, ..., R_p^i, ...R_p^N \). We mark \( R_p^* \) as the latest relevant revision of article \( p \), which we compare each new revision against (i.e., to find added content).

Per page \( p \), its sequence of \( N \) sorted revisions is taken as input one after the other while building the corpus. The Wikipedia dump files contain the page textual content for each \( R_p^i, \forall i \in N \). They do not contain the textual added content of \( R_p^i \) compared to \( R_p^{i-1} \), which forces us to compare revisions to identify new long textual contributions.

Before processing revision \( R_p^i \), we check whether it is relevant. There are two scenarios where we skip \( R_p^i \), such as a vandalism edit that deletes the page content. In Appendix Section A, we further explain those two scenarios. In cases when \( R_p^i \) is relevant, we follow two steps to extract its information and add to MAC:

Step 1: Revisions Comparison In this step, we extract the newly added textual content in \( R_p^i \). We do so by comparing \( R_p^i \) with \( R_p^* \).7 By splitting the text into paragraphs and sentences, we can detect new or edited text at the sentence level. Sequences of new or edited text are concatenated into the same text chunk. Note that multiple text chunks can be added or edited within a single revision. Hence, \( R_p^i \) is associated with a list of new text chunks.

7As previously explained, some revisions are skipped, so the comparison is not always to \( R_p^{i-1} \).
Step 2: Length Filtering Each text in MAC needs to be long enough to contain stylistic information for AV. However, many Wikipedia changes are minimal (e.g., typo correction), and therefore, we impose a length restriction where any edit must introduce at least \( \alpha \) contiguous words. We set \( \alpha = 100 \) in English, and it is adjusted for each language due to morphological and typological differences (e.g., a language with shorter sentences than English but more morphologically rich words); Details of this are in Appendix Section B.

This length presents a challenging setting on par with current datasets focused on social media and online reviews (e.g., Tyo et al., 2022).

We also impose a maximum length restriction. We exclude contributions that are longer than 50 words (e.g., 500 in English) to avoid edits, which are a recreation of existing articles, including translations and archiving of discussions in talk pages.

Wikipedia is a rich collaborative platform that allows users to contribute in different ways. Not all such contributions are relevant to our needs. Hence, we apply three data-cleaning processes to ensure we do not include undesired contributions. Those are: removal of tables, bots detection, and mixed languages content exclusion.8 We further explain these cleaning processes in Appendix Section C.

For each text chunk that we save, we also record a list of essential attributes such as the author details (name and ID), page details (title, ID, and namespace), and edit time. Row-level examples of MAC are presented in Appendix Table 4.

2.3 MAC Analysis

In Table 2, we present basic statistics for the ten most dominant languages in MAC. The complete statistics list with each language in the corpus is found in Appendix Table 3. Here, MAC is used to train and evaluate AV models. Therefore, we monitor and report the number of authors with at least two contributions, as those authors are helpful for training AV models. The two rightmost columns in Table 2 highlight two uniquenesses of MAC—cross-domain and cross-language contributions. Since each text chunk is associated with a specific Wikipedian (i.e., author), we are able to track authors even when they contribute in different domains and languages. This tracking ability is thanks to Wikipedia’s unified login mechanism.9

<table>
  <tr>
    <th>Language</th>
    <th># T.Chunks</th>
    <th>&ge;2 Contrib.</th>
    <th>&ge;2 Domains</th>
    <th>&ge;2 Langs.</th>
  </tr>
  <tr>
    <td>English (en)</td>
    <td>22.95M</td>
    <td>618.76K</td>
    <td>294.83K</td>
    <td>93.47K</td>
  </tr>
  <tr>
    <td>German (de)</td>
    <td>5.52M</td>
    <td>109.89K</td>
    <td>59.55K</td>
    <td>23.77K</td>
  </tr>
  <tr>
    <td>French (fr)</td>
    <td>6.77M</td>
    <td>85.88K</td>
    <td>39.28K</td>
    <td>20.82K</td>
  </tr>
  <tr>
    <td>Spanish (es)</td>
    <td>3.16M</td>
    <td>73.64K</td>
    <td>28.49K</td>
    <td>17.61K</td>
  </tr>
  <tr>
    <td>Russian (ru)</td>
    <td>2.48M</td>
    <td>49.32K</td>
    <td>17.32K</td>
    <td>12.90K</td>
  </tr>
  <tr>
    <td>Italian (it)</td>
    <td>2.21M</td>
    <td>46.73K</td>
    <td>19.38K</td>
    <td>9.62K</td>
  </tr>
  <tr>
    <td>Portuguese (pt)</td>
    <td>1.86M</td>
    <td>35.09K</td>
    <td>11.22K</td>
    <td>7.62K</td>
  </tr>
  <tr>
    <td>Polish (pl)</td>
    <td>1.66M</td>
    <td>29.81K</td>
    <td>16.03K</td>
    <td>4.98K</td>
  </tr>
  <tr>
    <td>Dutch (nl)</td>
    <td>1.15M</td>
    <td>20.48K</td>
    <td>8.76K</td>
    <td>6.26K</td>
  </tr>
  <tr>
    <td>Ukrainian (uk)</td>
    <td>883.37K</td>
    <td>15.05K</td>
    <td>4.41K</td>
    <td>3.80K</td>
  </tr>
  <tr>
    <td>All (60 Langs.)</td>
    <td>60.08M</td>
    <td>1.29M</td>
    <td>568.30K</td>
    <td>253.37K</td>
  </tr>
</table>

Table 2: MAC statistics. We present the top ten dominant languages in MAC and the total (i.e., sum) numbers over all languages in the corpus. 'T.Chunks' and 'Contrib.' refer to Text Chunks and Contributions, respectively.

This way, users keep their username while editing different page types (i.e., domains) and languages.

English (en) is the most dominant language in the corpus, aligned with the pattern in other Wikipedia corpora (Guo et al., 2020; Perez-Beltrachini and Lapata, 2021). 48% of the authors in MAC are English-speaking.

MAC analysis MAC comprises both authors (i.e., Wikipedians) and their textual contributions. Accordingly, we examine the distribution of authors across languages, as well as the length of each textual instance in MAC. Figure2 presents these two distributions: a log-log plot of author contributions and a histogram of text lengths. As expected, the distribution of author contributions closely follows a power-law pattern, while the distribution of text lengths is left-skewed, reflecting the dataset’s minimum length requirement of approximately 100 words per text.

Authors Overlap A practical way to analyze MAC is to calculate the overlap of authors between language pairs. We expect ‘close’ languages, such as Ukrainian and Belarusian, to have a high authors’ overlap as both are East Slavic languages, with a high lexical similarity, commonly spoken by many in the post-Soviet states. By applying this approach across all language pairs, we construct a network in which nodes represent languages, and weighted edges denote connections based on the extent of author overlap.

Appendix Figure 5 illustrates this network. As expected, the most edited Wikipedia languages (e.g., English, German) dominate the network. Closely related languages such as Ukrainian-Russian-Belarusian and Arabic-Egyptian-Farsi
Figure 2: MAC characteristics. Subfigure (a) demonstrates the power-law distribution of authors’ contributions. Subfigure (b) displays the left-skewed distribution of text lengths, with French being a notable exception. For clarity, we present only the five most prevalent languages in MAC.

maintain tight connections in the network.

3 Authorship Verification Experiments

Here, we introduce the AV tasks and experiments.

3.1 Task Definition

AV aims to identify whether two texts are written by the same author. Following recent works (Rivera-Soto et al., 2021; Zhu and Jurgens, 2021; Wegmann et al., 2022; Fincke and Boschee, 2024), we adopt a similarity-based verification approach in which a model is trained to compute similarity scores between a pair of texts and assign a high score to texts written by the same author. Formally, given a set of candidate texts \( \mathcal{T} = \{t_1, \cdots, t_n\} \), and an authorship function \( A(t) \), where \( A(t_i) = A(t_j) \) iff \( t_i \) and \( t_j \) are written by the same author, the goal of AV is to train a model M such that \( M(t_i, t_j) > M(t_i, t_k) \ \forall \ i, j, k \in [n] \) where \( A(t_i) = A(t_j) \neq A(t_k) \). In practice, since \( \mathcal{T} \) may be too large to exhaustively evaluate M for all pairs, and an operator of an AV model may wish to obtain authorship matches for a small set of candidate texts (e.g. for plagiarism detection), AV is formulated as an information retrieval (IR) task, where a small query set of texts is used to probe a much larger candidate set. Success of M is demonstrated (via standard IR metrics; see Section 3.2.4) by its ability to correctly assign scores such that a candidate text matching a query’s author is found.

3.2 Experimental Setup

3.2.1 Data Post-processing

To evaluate AV models in an IR manner, we restructure MAC as an IR task and thus denote query-candidate pairs for training, validation, and test. Specifically, for each author in each language, we extract one pair of texts. To maximally reduce the effect of content in AV (Wegmann et al., 2022), we select the hard positive pair with the lowest SBERT (Reimers and Gurevych, 2019) cosine similarity if we have more than two texts to choose from. Then, per language, we split all extracted text pairs into training, validation, and test sets in a 7:1:2 ratio.

We only keep text pairs from domain 0 in the training and validation sets to hold out the other domains for evaluating out-of-domain generalizability. We also filter out texts of more than 300 words to avoid the risk of including translated texts. The text pair selection (and length filtering) substantially reduce the amount of data used in the AV experiments, thereby leading to the different number of authors in Table 2 (\( \geq 2 \) Contrib.) and Table 8. For convenience, we denote the set of query and candidate texts as \( Q = \{q_0, \cdots, q_n\} \) and \( C = \{c_0, \cdots, c_n\} \), respectively.

3.2.2 AV Models

We consider two categories of models: off-the-shelf IR models and fine-tuned authorship representation models. The IR models serve as baselines for assessing the performance gain from fine-tuning.

IR Model  We select two representative IR models: BM25 (Robertson et al., 1994) and SBERT (Reimers and Gurevych, 2019). BM25 is one of the most widely used information retrieval (IR) models. Given a pair of texts, a relevance score is calculated based on token matching. We use this relevance score as the AV similarity score \( sim_{ij} \) for each pair
of texts \( q_i \) and \( c_j \). Unlike BM25, SBERT is a neural representation model that transforms texts into embedding vectors, tuned explicitly for semantic similarity. For two texts \( q_i \) and \( c_j \), we use the cosine similarity of their embedding vectors, \( v_{q_i} \) and \( v_{c_j} \) as their AV similarity score \( \mathrm{sim}_{ij} \), which can be formally defined as \( \mathrm{sim}_{ij} = \frac{v_{q_i} \cdot v_{c_j}}{\|v_{q_i}\|\|v_{c_j}\|} \).

Authorship Representation Model  We evaluate two models directly fine-tuned on MAC. One is trained with the Sentence Transformer codebase (Reimers and Gurevych, 2019) using the multiple negatives ranking loss. For convenience, we refer to this model as **SBERT\(_{AV}\)** to distinguish it from the semantic SBERT in the previous category. The other is the state-of-the-art AV system **SADIRI** (Fincke and Boschee, 2024) that uses hard positives and in-batch hard negatives, which are negative examples whose authorship is difficult to distinguish from the anchor text, to improve the model performance in more challenging situations. Note that, in this work, we extract the hard positive pairs during data post-processing, so the main difference between SBERT\(_{AV}\) and SADIRI is the hard negative batching technique.

We use the same equation for SBERT to calculate the AV similarity score for both SBERT\(_{AV}\) and SADIRI. In this study, we only focus on AV models trained in a single language. Hence, we train an SBERT\(_{AV}\) and a SADIRI model for each language in the training set so that we can hold out other languages and domains to test the generalizability of the models in RQs 2 and 3.

3.2.3 Evaluation Setup

We first train an SBERT\(_{AV}\) and a SADIRI model on domain 0 data for each of the top 10 languages in MAC according to the size of the domain 0 data. We then evaluate these trained models, as well as the BM25 and SBERT baselines, on held-out test sets from MAC to examine the five RQs introduced in Section 1. Please see Appendix Section D.3 for training and test set statistics.

For RQs 1 to 3, we create a single-language single-domain test set for each language \( L \) and domain \( D \) by extracting a text pair with two texts in \( D \) from the test split of \( L \) for each author. Then, in RQ1, we examine the in-language in-domain performance of models using in-language domain 0 test sets; in **RQ2**, we examine the **out-of-language generalizability** of models using out-of-language in-domain\(^{10}\) test sets. In particular, we evaluate the models on domain 0 single-language test sets in all unseen languages; in **RQ3**, we examine the **out-of-domain generalizability** using in-language\(^{10}\) out-of-domain test sets. In particular, we evaluate the models on in-language single-domain test sets for all unseen domains (domains 1 to 3) and take the average score across all evaluation domains for each AV approach.

In **RQ4**, we examine the **cross-lingual verification** performance using a cross-lingual in-domain test set. This is created by gathering all authors in the test split for all languages, and for each of those who have contributions in domain 0 in at least two languages, constructing a single text pair using two texts in domain 0 randomly chosen from two different languages. Also, we always choose two texts from *different articles* to ensure they are not translations of each other. The same test set is used for all models; in **RQ5**, we examine the **cross-domain verification** performance using in-language cross-domain test sets. In particular, for each language, we construct a single text pair by randomly matching texts in different domains for each author who has contributions in at least two domains in the test split.

Note that, RQ2 (respectively, RQ3) is similar to RQ4 (respectively, RQ5) because they both involve multiple languages (respectively, domains). However, they differ fundamentally in how the document pairs are constructed. In RQ2 (respectively, RQ3), the two documents for each author are in the same language (respectively, domain), which is different from the language (respectively, domain) of the training data, whereas in RQ4 (respectively, RQ5), the two documents for each author are in different languages (respectively, domains).

3.2.4 Evaluation Metrics

We evaluate the AV models using Success@k, which is one of the most commonly used metrics in the AV literature (Khan et al., 2021; Rivera-Soto et al., 2021; Fincke and Boschee, 2024).\(^{11}\) For-

\(^{10}\)We use “in-domain” (respectively, “in-language”) to indicate that the domain (respectively, language) for evaluation is the same as that used for training the fine-tuned models. For the two off-the-shelf IR baselines, we use the same model for all domains and languages.

\(^{11}\)Some of them use Recall@k which is equivalent to Success@k in this work because we have exactly one candidate for each query in the evaluation sets.
mally, given two sets of texts \( \mathcal{C} \) and \( \mathcal{Q} \),

\[
\text{Success}@k = \sum_{i=1}^{|\mathcal{Q}|} \prod_{r=1}^k \left( \mathbb{I}(A(c_{r,i}) = A(q_i)) \right) / |\mathcal{Q}|
\]

where \( c_{r,i} \) refers to the text in \( \mathcal{C} \) that has the \( r \)th highest AV similarity to \( q_i \) w.r.t. an AV model. For each experiment, we form \( \mathcal{Q} \) (respectively, \( \mathcal{C} \)) from the query (respectively, candidate) side of each pair of a validation or test subset, as described in Section 3.2.1. Validation subsets are used to choose the final model in each experiment, and we report results on the test subsets only. We use \( k = 1 \) as the main metric in this work for a strict evaluation. However, we also present results while using \( k = 8 \) in Appendix Section E.

3.2.5 Implementation Details
We use OkapiBM25Model from the Gensim Python library (Rehůřek and Sojka, 2010) for BM25 and the Sentence Transformer Python library (Reimers and Gurevych, 2019) for SBERT models and training. We use an implementation of the SADIRI system based on its description in the original paper (Fincke and Boschee, 2024). We use only multilingual language models in our experiments. We use paraphrase-multilingual-mpnet-base-v2 as the base model for semantic SBERT and the xlm-roberta-base (Conneau et al., 2020) as the base model for all fine-tuned authorship representation models. Please see Appendix Section D.1 for all hyperparameters.

4 Results
In this section, we present the evaluation results for all AV models on all five RQs. Due to space limitations, we mainly discuss the results for AV models trained on English data. Please see Appendix Section E for results over the rest of the languages. The results for RQs 1 to 5 are shown in bar groups 1 to 5 in Figure 3, respectively.
RQ1: Training on in-language in-domain data improves AV model performance, as shown in the first bar group in Figure 3; here, the two fine-tuned models (SBERT\(_{AV}\) and SADIRI) perform better than both IR baselines (BM25 and SBERT), and between the two fine-tuned models, SADIRI outperforms SBERT\(_{AV}\), which aligns with our expectation because we expect the hard negative batching of SADIRI benefits the model when evaluated on the same distribution as training data.
RQ2: Trained AV models can generalize to unseen languages, as seen in the second bar group in Figure 3. The trend also closely resembles the patterns observed in RQ1, but the performance gain over BM25 is slightly less than that in RQ1, which indicates that while the fine-tuned models are generalizable to other languages, the improvement in out-of-language performance is relatively smaller. The fine-grained results on the top five languages in Figure 4 also indicate that for each evaluation language, the models trained in the same language perform the best or just worse than the model trained in English data in most cases, which falls within our expectation because the English subset of MAC contains the most data.
RQ3: In contrast to RQ2, fine-tuned models do not generalize to new domains, as seen in the third bar group in Figure 3. Neither fine-tuned model is better than the strongest baseline, BM25, and SBERT\(_{AV}\) even exhibits a worse performance than BM25, which suggests that out-of-domain generalization is harder than out-of-language generalization for AV, and training in single domain data may even harm the performance in unseen domains. Note that, although the task in RQ3 is harder than RQ1, the scores in RQ3 are higher than RQ1. The main reason is that the size of the test set for RQ3 is much smaller than RQ1, which makes the retrieval in RQ3 easier.
RQ4: Unlike the observations in RQs 2 and 3, fine-tuned AV models may or may not generalize to identifying authors across language, depending on the training algorithm used, as shown in the 4th bar group in Figure 3. This result exhibits a markedly different pattern where SBERT also outperforms BM25, and SBERT\(_{AV}\) outperforms both BM25 and SADIRI. BM25 performs the worst mainly because it relies on exact token matching, which does not work across languages with different vocabularies, while a potential reason for the under-performance of SADIRI is that its training strategy overemphasizes in-language in-domain hard negative optimization which harms cross-lingual performance.
RQ5: Fine-tuned models do not generalize to identifying authors across domains, as shown in the 5th bar group in Figure 3. Both trained models are worse than the strong baseline BM25 and marginally better than the weak baseline SBERT. The worse performance of SADIRI differs from the observation in Fincke and Boschee (2024) that training on hard positives and in-batch hard negatives dramatically improves the performance on cross-genre AV. One possible reason for the in-
Figure 3: Success@1 scores for the two baselines (SBERT and BM25) and the two fine-tuned models (SBERT_{AV} and SADIRI) trained on English data. The scores for RQ2 are the averages across only unseen evaluation languages, and the scores for RQ3 are the averages across only unseen domains (domains 1 to 3). In general, the performance of the four models exhibits similar patterns on RQs 1 and 2, whereas the patterns on the other RQs, especially RQs 4 and 5, are markedly different. All differences in this figure are statistically significant based on paired t-tests (p < 0.05), except for the difference between BM25 and IR-AA in RQ3 and the difference between IR-AA and SBERT in RQ4.

Figure 4: The Success@1 scores for out-of-language generalization for the top five languages. The scores highlighted in the orange boxes are the in-language scores for which the models are trained and evaluated in the same language. Boxes in this figure are auto-shaded such that darkness correlates with Success scores across the entire table.

consistency is that we only trained the models on single-domain data, which significantly degraded the quality of the hard positives, thereby constraining the models’ cross-domain generalizability. Comparing with the training data in Fincke and Boschee (2024), it reveals that training on multi-domain data or data from various sources may be the key to stronger cross-lingual generalizability.

5 Discussion

Overall, as seen in Figure 3, MAC enables the comparison of AV models under conventional scenarios such as those on the left (RQs 1 to 3). The results indicate that fine-tuned AV models perform better than IR baselines for AV in the same language and domain as training. Fine-tuned models also generalize well to AV in the same domain but unseen languages, but do not generalize to unseen domains at all (RQ3, 3rd bar group in Figure 3). Over the other 2 RQs, which are only assessable using MAC, the results are starkly different on RQ4, whereas the pattern on RQ5 closely resembles that in RQ3. In general, the results indicate that AV models’ performances are not consistent across different evaluation conditions, so to have a more comprehensive understanding of the capability of AV models, testing them under different conditions is necessary. Existing datasets can cover some conditions, but MAC enables the evaluation of other situations like cross-lingual AV and cross-domain AV in different languages.

Moreover, the poor performance of the fine-tuned models in RQs 3 to 5 reveals challenges in AV involving multiple domains and multiple languages, which may serve as new directions for future research in this area, and MAC, as it contains high volumes of data from multiple domains and languages, enables researchers to build models to tackle these specific aspects of AV.

While building and evaluating MAC, we paid extra attention to excluding translation cases. Furthermore, a practical way to avoid the over-representation of accounts that are shared by multiple people is to limit the number of samples we use per author (while running experiments).12 We use this approach in our research and extract only one pair of texts for each author in each language.

12This assumes that most accounts are not shared by multiple people.
6 Related Work

We first summarize AV-related research and then discuss existing Wikipedia corpora used in NLP.

AV Models  With the development of NLP in the past few decades, the techniques in AV have evolved through three main stages. In the pre-neural era, AV works mainly focused on feature-based approaches (Koppel and Schler, 2004; Stamatatos, 2009; Stolerman et al., 2014). Later, with the widespread use of neural networks (NN), especially (pre-trained) transformers, NN-based representation models have become mainstream and demonstrated superior performance on large-scale AV (Rivera-Soto et al., 2021; Zhu and Jurgens, 2021; Wegmann et al., 2022; Fincke and Boschee, 2024). Recently, leveraging the strong zero- and few-shot capability of LLMs, some studies showed that LLMs could achieve a decent performance on AV without further downstream fine-tuning, but the findings are limited to small-scale AV (Hung et al., 2023; Huang et al., 2024). In this work, we concentrate on large-scale AV due to its higher practicality and, therefore, primarily adopt NN-based representation models in our experiments.

AV Datasets  Datasets also play a crucial role in AV research for both model training and evaluation. AV requires corpora with authorship labels which could be collected from various sources such as email (Klimt and Yang, 2004), newswire agency (Lewis et al., 2004), newspaper (Stamatatos, 2013). Data from these sources often contain a relatively small number of authors. With the increasing number of internet users, later works started to collect data from public websites such as Amazon (Ni et al., 2019; Keung et al., 2020) and Reddit (Rivera-Soto et al., 2021). Datasets from these sources are substantially larger in scale and serve as the foundation of the SOTA AV models (Rivera-Soto et al., 2021; Fincke and Boschee, 2024). These datasets contain up to one million authors (Rivera-Soto et al., 2021) and texts in six languages (Keung et al., 2020), while MAC is 10 times larger in terms of both the numbers of authors and languages. Moreover, MAC also contains texts from four domains and authors writing in multiple languages and domains, enabling a more comprehensive evaluation of the AV models presented in Section 3.2.3.

Wikipedia Corpora  Wikipedia serves the research community as a textual source in various domains and NLP tasks (Pan et al., 2017; Kaffee et al., 2018; Zhu et al., 2019; Sathe et al., 2020; Guo et al., 2020; Ta et al., 2023). A series of studies have used information about Wikipedia editors to create data sets that associate text with authors (Yang et al., 2017; Maki et al., 2017; Jaidka et al., 2021) and others have leveraged edit histories for a variety of purposes such as simplification (Yatskar et al., 2010) or grammatical error correction (Faruqui et al., 2018). These and other Wikipedia-related corpora have been released under the ‘CC BY-SA’ Wikipedia license terms, which we also rely on while releasing MAC to the research community.

Most Wikipedia-related studies focus on a single language, usually the most popular ones, such as English or German. While some others (Pan et al., 2017) leverage Wikipedia’s multilingual capability, none tackle the broad combination of languages and domains as we do in MAC by extracting long textual content from edits done by Wikipedians.

7 Summary

Our novel Million Authors Corpus (MAC) spans 60 languages and four domains, linking long textual contributions to their respective authors while excluding minor edits. MAC consists of more than 60M cleaned text passages from contributions by Wikipedian authors. Here, we study AV models using the 1.29M authors who made 2+ contributions in MAC across the 60 languages; more than 560K authors contributed in different domains, such as an article page and a talk page. In addition, MAC includes multilingual authors who contributed to at least two languages.

MAC enables new studies of cross-lingual and cross-domain AV evaluations; i.e., given a text written by an author, finding another text by the same author written in a different language or domain. Our study demonstrates that in conventional AV scenarios, a selection of fine-tuned models surpass IR baselines on RQ1 and RQ2, but not on RQ3. However, we show that in new novel and challenging settings of cross-lingual and cross-domain AV (RQ4 and RQ5), we observe starkly different patterns and models, as a whole, perform substantially lower. Our results highlighting the value of the comprehensive AV model evaluation powered by MAC and point to the future research directions of AV in cross-domain and cross-lingual modeling.
8 Limitations

We process 60 languages out of more than 300 languages that exist in Wikipedia. Unfortunately, many languages have a relatively small user base and content, which is irrelevant to our study. We select the 60 major Wikipedia languages in terms of the number of pages and user base. Notably, we do not include the Cebuano Wikipedia13 in MAC, although it is the second largest Wikipedia project, articles-wise. The reason is the extremely low number of contributors to this project—153 active users. We also limit our study to the four main page types in Wikipedia (i.e., namespaces 0 to 3) out of the 26 page types Wikipedia maintains.14

Another limitation of the new corpus is the association between text and users. Page edits on Wikipedia are automatically recorded by those who edit the page. However, we do not control the way people generate their textual content and whether those are ‘copied-paste’ or translated from other sources. This limitation is mostly relevant to article pages (namespace 0) rather than user or talk pages.

However, we handle translations to some extent using two methods: (i) We filter out from MAC too long textual contributions (see Step 2 in Section 2.2), which are likely to be an article translation; and (ii) While evaluating the models for RQ4, we do not include the same article for the same author in two different languages (see Section 3.2.3). Most of the Wikipedia content, and so of MAC has been created before the GenAI era, where both humans and LLMs create textual content.

In our study, we make sure to cover different domains within Wikipedia by including data from different namespaces. However, future research is needed to establish the degree to which authorship verification in the Wikipedia domains is portable to other domains and vice versa. For example, literary authorship style may not be captured here, nor may authorship identity over many years of an author’s lifetime.

9 Ethical Considerations

This research relies exclusively on publicly available data from Wikipedia, adhering strictly to ethical standards for data collection and analysis. All data used in this study are accessible to the public through Wikipedia’s platform, ensuring transparency and compliance with open data principles.

We adhere to the terms and conditions specified by Wikipedia’s licensing framework, including the Creative Commons Attribution-Share license (CC BY-SA). All derivative works, analyses, and datasets generated from Wikipedia data comply with these licensing requirements, ensuring proper attribution and alignment with open knowledge practices.

Finally, we use the data solely for research and modeling purposes, advancing the research of AV models in the NLP domain while respecting the rights and intentions of the Wikipedia community. By following these principles, we aim to balance the pursuit of scholarly insights with the ethical responsibility of protecting individuals and maintaining data integrity.

Authorship verification is a dual-use technology with societal benefits and risks. While AV—and forensic linguistics in general—has been used to help law enforcement identify individuals involved in crimes such as human trafficking (e.g., Olsson, 2009; Saxena et al., 2023) and has uses in historical and cultural applications (e.g., Gurney and Gurney, 1998; Juola et al., 2008), such technology could also be used maliciously, such as to de-anonymize individuals posting under pseudonyms. We publish MAC to facilitate the research in AV models that can be useful in various areas. Such malicious usages have the potential to threaten internet users’ privacy and other unnecessary effects. Therefore, we will release MAC with guidance to forbid such malicious use cases and intend it solely for research purposes.

Acknowledgments

This research is supported in part by the Office of the Director of National Intelligence (ODNI), Intelligence Advanced Research Projects Activity (IARPA), via the HIATUS Program contract #2022-22072200006. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied, of ODNI, IARPA, or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for governmental purposes notwithstanding any copyright annotation therein.

13Cebuano Wikipedia: https://tinyurl.com/mr3d6z8h
14Wikipedia namespaces: https://tinyurl.com/46jw4u7u
References

Phoebe Ayers, Charles Matthews, and Ben Yates. 2008. How Wikipedia works: And how you can be a part of it. No Starch Press.

Janek Bevendorff, Bilal Ghanem, Anastasia Giachanou, Mike Kestemont, Enrique Manjavacas, Ilia Markov, Maximilian Mayerl, Martin Potthast, Francisco Rangel, Paolo Rosso, Günther Specht, Efstatios Statmatatos, Benno Stein, Matti Wiegmann, and Eva Zangerle. 2020. Overview of pan 2020: Authorship verification, celebrity profiling, profiling fake news spreaders on twitter, and style change detection. In Experimental IR Meets Multilinguality, Multimodality, and Interaction, pages 372–383, Cham. Springer International Publishing.

Bernard Comrie. 1989. Language universals and linguistic typology: Syntax and morphology. University of Chicago press.

Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8440–8451, Online. Association for Computational Linguistics.

Manaal Faruqui, Ellie Pavlick, Ian Tenney, and Dipanjan Das. 2018. WikiAtomicEdits: A multilingual corpus of Wikipedia edits for modeling language and discourse. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 305–315, Brussels, Belgium. Association for Computational Linguistics.

Steven Fincke and Elizabeth Boschee. 2024. Separating style from substance: Enhancing cross-genre authorship attribution through data selection and presentation. arXiv preprint arXiv:2408.05192.

Mandy Guo, Zihang Dai, Denny Vrandečić, and Rami Al-Rfou. 2020. Wiki-40B: Multilingual language model dataset. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 2440–2452, Marseille, France. European Language Resources Association.

Penelope J Gurney and Lyman W Gurney. 1998. Authorship attribution of the scriptores historiae augustae. Literary and Linguistic Computing, 13(3):119–131.

Skyler Hallinan, Faeze Brahman, Ximing Lu, Jaehun Jung, Sean Welleck, and Yejin Choi. 2023. STEER: Unified style transfer with expert reinforcement. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 7546–7562, Singapore. Association for Computational Linguistics.

William L. Hamilton, Jure Leskovec, and Dan Jurafsky. 2016. Diachronic word embeddings reveal statistical laws of semantic change. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1489–1501, Berlin, Germany. Association for Computational Linguistics.

Martin Haspelmath. 2005. The world atlas of language structures. Oxford university press.

Baixiang Huang, Canyu Chen, and Kai Shu. 2024. Can large language models identify authorship? In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 445–460, Miami, Florida, USA. Association for Computational Linguistics.

Chia-Yu Hung, Zhiqiang Hu, Yujia Hu, and Roy Lee. 2023. Who wrote it and why? prompting large-language models for authorship verification. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 14078–14084, Singapore. Association for Computational Linguistics.

Kokil Jaidka, Andrea Ceolin, Iknoor Singh, Niyati Chhaya, and Lyle Ungar. 2021. WikiTalkEdit: A dataset for modeling editors’ behaviors on Wikipedia. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2191–2200, Online. Association for Computational Linguistics.

Patrick Juola et al. 2008. Authorship attribution. Foundations and Trends® in Information Retrieval, 1(3):233–334.

Lucie-Aimée Kaffee, Hady Elsahar, Pavlos Vougiouklis, Christophe Gravier, Frédérique Laforest, Jonathon Hare, and Elena Simperl. 2018. Learning to generate Wikipedia summaries for underserved languages from Wikidata. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 640–645, New Orleans, Louisiana. Association for Computational Linguistics.

Phillip Keung, Yichao Lu, György Szarvas, and Noah A. Smith. 2020. The multilingual Amazon reviews corpus. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4563–4568, Online. Association for Computational Linguistics.

Aleem Khan, Elizabeth Fleming, Noah Schofield, Marcus Bishop, and Nicholas Andrews. 2021. A deep metric learning approach to account linking. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5275–5287, Online. Association for Computational Linguistics.

Bryan Klimt and Yiming Yang. 2004. The enron corpus: A new dataset for email classification research. In European Conference on Machine Learning.
Moshe Koppel and Jonathan Schler. 2004. Authorship verification as a one-class classification problem. In Machine Learning, Proceedings of the Twenty-first International Conference (ICML 2004), Banff, Alberta, Canada, July 4-8, 2004, volume 69 of ACM International Conference Proceeding Series. ACM.

Andrey Kutuzov, Lilja Øvrelid, Terrence Szymanski, and Erik Velldal. 2018. Diachronic word embeddings and semantic shifts: a survey. In Proceedings of the 27th International Conference on Computational Linguistics, pages 1384–1397, Santa Fe, New Mexico, USA. Association for Computational Linguistics.

David D. Lewis, Yiming Yang, Tony G. Rose, and Fan Li. 2004. Rcv1: A new benchmark collection for text categorization research. J. Mach. Learn. Res., 5:361–397.

Shuai Liu and Jonathan May. 2024. Style transfer with multi-iteration preference optimization. ArXiv preprint, abs/2406.11581.

Keith Maki, Michael Yoder, Yohan Jo, and Carolyn Rosé. 2017. Roles and success in Wikipedia talk pages: Identifying latent patterns of behavior. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1026–1035, Taipei, Taiwan. Asian Federation of Natural Language Processing.

Olena Medelyan, David Milne, Catherine Legg, and Ian H Witten. 2009. Mining meaning from wikipedia. International Journal of Human-Computer Studies, 67(9):716–754.

Mostafa Mesgari, Chitu Okoli, Mohamad Mehdi, Finn Årup Nielsen, and Arto Lanamäki. 2015. “the sum of all human knowledge”: A systematic review of scholarly research on the content of w ikipedia. Journal of the Association for Information Science and Technology, 66(2):219–245.

Tempestt Neal, Kalaiyani Sundararajan, Aneez Fatima, Yiming Yan, Yingfei Xiang, and Damon Woodard. 2017. Surveying stylometry techniques and applications. ACM Computing Surveys (CSUR), 50(6):1–36.

Jianmo Ni, Jiacheng Li, and Julian McAuley. 2019. Justifying recommendations using distantly-labeled reviews and fine-grained aspects. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 188–197, Hong Kong, China. Association for Computational Linguistics.

John Olsson. 2009. Wordcrime: Solving crime through forensic linguistics. A&C Black.

Xiaoman Pan, Boliang Zhang, Jonathan May, Joel Nothman, Kevin Knight, and Heng Ji. 2017. Cross-lingual name tagging and linking for 282 languages. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1946–1958, Vancouver, Canada. Association for Computational Linguistics.

Laura Perez-Beltrachini and Mirella Lapata. 2021. Models and datasets for cross-lingual summarisation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 9408–9423, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.

Radim Řehůřek and Petr Sojka. 2010. Software Framework for Topic Modelling with Large Corpora. In Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks, pages 45–50, Valletta, Malta. ELRA. http://is.muni.cz/publication/884893/en.

Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992, Hong Kong, China. Association for Computational Linguistics.

Rafael A. Rivera-Soto, Olivia Elizabeth Miano, Juanita Ordonez, Barry Y. Chen, Aleem Khan, Marcus Bishop, and Nicholas Andrews. 2021. Learning universal authorship representations. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 913–919, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.

Stephen E. Robertson, Steve Walker, Susan Jones, Micheline Hancock-Beaulieu, and Mike Gatford. 1994. Okapi at trec-3. In Text Retrieval Conference.

Aalok Sathe, Salar Ather, Tuan Manh Le, Nathan Perry, and Joonsuk Park. 2020. Automated fact-checking of claims from Wikipedia. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 6874–6882, Marseille, France. European Language Resources Association.

Vageesh Saxena, Benjamin Ashpole, Gijs van Dijck, and Gerasimos Spanakis. 2023. IDTraffickers: An authorship attribution dataset to link and connect potential human-trafficking operations on text escort advertisements. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 8444–8464, Singapore. Association for Computational Linguistics.

Jonathan Schler, Moshe Koppel, Shlomo Engelson Argamon, and James W. Pennebaker. 2006. Effects of age and gender on blogging. In AAAI Spring Symposium: Computational Approaches to Analyzing Weblogs.

Yanir Seroussi, Ingrid Zukerman, and Fabian Bohnert. 2014. Authorship attribution with topic models. Computational Linguistics, 40(2):269–310.
Efstathios Stamatatos. 2009. A survey of modern authorship attribution methods. J. Assoc. Inf. Sci. Technol., 60:538–556.

Efstathios Stamatatos. 2013. On the robustness of authorship attribution based on character n -gram features. Journal of law and policy, 21:7.

Ariel Stoleruman, Rebekah Overdorf, Sadia Afroz, and Rachel Greenstadt. 2014. Breaking the closed-world assumption in stylometric authorship attribution. In Advances in Digital Forensics X, pages 185–205, Berlin, Heidelberg. Springer Berlin Heidelberg.

Hoang Thang Ta, Abu Bakar Siddiqur Rahman, Navonil Majumder, Amir Hussain, Lotfollah Najjar, Newton Howard, Soujanya Poria, and Alexander Gelbukh. 2023. Wikides: A wikipedia-based dataset for generating short descriptions from paragraphs. Information Fusion, 90:265–282.

Jacob Tyo, Bhuwan Dhingra, and Zachary C Lipton. 2022. On the state of the art in authorship attribution and authorship verification. ArXiv preprint, abs/2209.06869.

Anna Wegmann, Marijn Schraagen, and Dong Nguyen. 2022. Same author or just same topic? towards content-independent style representations. In Proceedings of the 7th Workshop on Representation Learning for NLP, pages 249–268, Dublin, Ireland. Association for Computational Linguistics.

Diyi Yang, Aaron Halfaker, Robert Kraut, and Eduard Hovy. 2017. Identifying semantic edit intentions from revisions in Wikipedia. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2000–2010, Copenhagen, Denmark. Association for Computational Linguistics.

Mark Yatskar, Bo Pang, Cristian Danescu-Niculescu-Mizil, and Lillian Lee. 2010. For the sake of simplicity: Unsupervised extraction of lexical simplifications from Wikipedia. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 365–368, Los Angeles, California. Association for Computational Linguistics.

Lei Zheng, Christopher M Albano, Neev M Vora, Feng Mai, and Jeffrey V Nickerson. 2019. The roles bots play in wikipedia. Proceedings of the ACM on Human-Computer Interaction, 3(CSCW):1–20.

Jian Zhu and David Jurgens. 2021. Idiosyncratic but not arbitrary: Learning idiolects in online registers reveals distinctive yet consistent individual styles. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 279–297, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.

Qi Zhu, Xiang Ren, Jingbo Shang, Yu Zhang, Ahmed El-Kishky, and Jiawei Han. 2019. Integrating local context and global cohesiveness for open information extraction. In Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining, WSDM 2019, Melbourne, VIC, Australia, February 11-15, 2019, pages 42–50. ACM.
A  Revisions Filtering

As we highlight in Section 2, we filter some of the revisions (i.e., edits). In this Section, we further explain the two criteria we apply to filter some of the revisions.

Not all edits of a Wikipedia page should be considered while building MAC. We consider two scenarios where we omit \( R_p^i \). The first is when \( R_p^i \) is later deleted by a Wikipedia admin. In such a case, the textual content of the deleted revision appears as empty text in the dump file. It makes no sense for this to be considered a valid edit. The second case is when a user makes a sequence of consecutive edits to a page. If so, we ‘aggregate’ their contributions and consider only the last revisions out of the sequence. These two special cases can explain why \( R_p^* \) is not always the latest revision we processed but rather the last relevant revisions which \( R_p^i \) should be compared against.

B  Edit Filtering

Each edit in MAC is intended to be a sufficiently long piece of text to encoder some aspect of an author’s style. While stylometric work has generally used used longer text on the order of several hundred word (Neal et al., 2017), shorter text still contain rich style information and, given their higher frequency in data, are valuable for learning to recognize style. Here, we adopt a threshold-based approach for selecting edits, where any contiguous edit of at least \( \alpha \) words is counted. We adopt \( \alpha = 100 \) for English for three reasons: (1) this length is in the same range as more recent and challenging AV datasets (e.g., Tyo et al., 2022), (2) the shorter length allows models trained on MAC to potentially generalize to other common short-document domains such as reviews and social media, which are more common in practice, and (3) recent shared research programs such as IARPA HIATUS have adopted a 100 word document length in their own metrics.\footnote{https://www.iarpa.gov/images/PropsersDayPDFs/HIATUS/IARPA-BAA-22-01_HIATUS_Amend001_C.pdf}

Setting \( \alpha = 100 \) for all languages will not work well, however. Languages vary in their information density per word (Comrie, 1989), with some languages having much shorter or longer sentences than English due to morphology (Haspelmath, 2005). There is no standard approach to determine how to rescale such a threshold for all languages. Therefore, we adopt a data-driven approach to set \( \alpha \) for each language based on its relative morphological density. Here, for each language \( \ell \), we first calculate the mean number of characters per whitespace-separated token, \( \overline{w}_\ell \). Note that we intentionally avoid using an LLMs tokenizer to decide on intra-token units (i.e., from subwords), as these tokenizers’ outputs vary significantly in our data based on whether the tokenizer was originally fit to language in that data so subword units for different languages would not be equivalently scoped.

We then rescale \( \alpha \) for each language as \( \frac{100 \overline{w}_{\text{English}}}{\overline{w}_\ell} \), rounding to the nearest whole character. For example, Russian had 5.66 characters per tokens, compared with 4.80 for English, roughly correlating with the more complex morphology of Russian; as a result \( \alpha = 85 \) for Russian where shorter texts are allowed to account for richer intra-token stylistic variation. By rescaling \( \alpha \) using in-domain data, we aim to directly match authors’ writing behaviors in the data.

C  Data Cleaning – Further Information

We execute the cleaning process of MAC while maintaining a “precision-biased” approach – prioritizing methods that ensure the cleanliness of the corpus at the expense of potentially excluding some relevant textual content. We apply the following three filters:

1. Tables. Wikipedia contains information not only in the format of textual sentences/paragraphs but also in tables. To avoid minor contributions from users who add/edit big tables (which do not contain much text), we remove all tables from all page revisions.

2. Bots. Bots operate in Wikipedia (Zheng et al., 2019). Systematically, most of their ‘contributions’ are not included in MAC as we only capture long textual contributions, while bots are less likely to do so. However, we apply a simple rule in the English Wikipedia to detect bots. We filter out contributions of usernames that start/end with the ‘bot’ regex.\footnote{We apply this regex rule as case insensitive.}

3. Mixed languages. Naturally, the dominant language per project is the project’s language (e.g., Italian in the Italian Wikipedia). However,
Wikipedians are not limited to writing in a specific language. It is most common for users to use English on Talk pages in non-English Wikipedia projects. To avoid having a lot of English content in non-English projects, we apply a language detection method\footnote{Pycld2 package: https://tinyurl.com/4ewazbkp} and remove cases of English text used in non-English Wikipedia projects.

D More Experimental Details

D.1 Hyperparameters
For a more controlled comparison between SBERT\(_{AV}\) and SADIRI, we train them using the same set of hyperparameters listed in Table 5. Because the sizes of the subsets vary substantially by language, to ensure a similar number of gradient updates in one epoch for each language, we use different batch sizes for different languages. For BM25, we use the default hyperparameters for OkapiBM25Model in Gensim, which are listed in Table 6.

D.2 Hardware & Runtime
The MAC creation process over 60 languages took 16.5 days while using 100 Intel Xeon Gold 6138 CPU cores. All AV models in this work are trained using a single NVIDIA A40-48GB GPU. The AV models training times for SBERT\(_{AV}\) and SADIRI are listed in subsection D.2.

D.3 Training & Test Sets Statistics
We list the number of query-candidate pairs for each subset in each language in Table 8. The numbers also represent author counts because we keep exactly one pair of query and candidate texts for each author in all training, validation, and test subsets. For RQ4, we only have a single cross-lingual test set with 1,635 text pairs (authors).

E More Experimental Results
We present the out-of-language generalization scores for all top ten languages in Figure 6 and the scores for RQs 1 to 5 for the two baselines and the two fine-tuned models trained on each of the top ten languages in Figure 7 and Figure 8.

In general, the two fine-tuned models trained in the top five languages consistently outperform the two IR baselines for RQ1, but for RQ2, only the models trained in the top three languages are consistently better than the IR baselines; for other training languages, the results are mixed for RQs 1 and 2. This observation suggests that training an AV model with decent in-language in-domain performance requires a certain amount of data, and this requirement on data size is larger for in-domain out-of-language generalization.

In contrast, the observations for all top ten languages on the other three RQs are generally consistent with the results for English in Figure 3. Specifically, for RQs 3 and 5, both trained models are worse than the strongest baseline BM25 in most cases; otherwise, their performances are the same; for RQ 4, both trained models consistently outperform the two IR baselines with only two exceptions (SADIRI model trained on German and Ukrainian).

We use Success@1 in the main result analysis because some of our evaluation sets a small and Success@1 can provide stricter results on small dataset. However, we also present Success@8 results in Figure 6, Figure 7, and Figure 8 for reference. The patterns of Success@8 results are generally consistent with those observed in Success@1 results. The only main difference is that SBERT\(_{AV}\) outperforms SADIRI in most cases.

F Artifacts

F.1 Created Artifacts
In this study, the main artifact we make is *MAC*—a dataset of Wikipedia textual content associated with its author. The new corpus will be released for research use under the general Wikipedia licensing framework, including the Creative Commons Attribution-Share license (CC BY-SA). We further detail this in Section 9.

F.2 Use of Existing Artifacts
Table 9 outlines the models and code libraries employed in this work. Our use of these artifacts is consistent with their intended use.

G Use of AI Assistants
We use Grammarly and ChatGPT to check grammar and polish our manuscript, but only for grammar corrections and minor edits. We take full responsibility for all the content in our manuscript.
<table>
  <tr>
    <th>Language</th>
    <th># T.Chunks</th>
    <th># Chars</th>
    <th># Words</th>
    <th>≥2 Contrib.</th>
    <th>≥2 Domains</th>
    <th>≥2 Langs.</th>
  </tr>
  <tr><td>Afrikaans (af)</td><td>41942</td><td>54591376</td><td>7848343</td><td>446</td><td>190</td><td>476</td></tr>
  <tr><td>Amharic (am)</td><td>2929</td><td>4066547</td><td>713146</td><td>81</td><td>24</td><td>79</td></tr>
  <tr><td>Arabic (ar)</td><td>2611219</td><td>2281733767</td><td>357439206</td><td>20988</td><td>7447</td><td>4472</td></tr>
  <tr><td>Egyptian Arabic (arz)</td><td>307873</td><td>242898738</td><td>35042567</td><td>341</td><td>111</td><td>534</td></tr>
  <tr><td>Belarusian (be)</td><td>118570</td><td>146621963</td><td>17866988</td><td>939</td><td>251</td><td>915</td></tr>
  <tr><td>Bulgarian (bg)</td><td>265511</td><td>349182870</td><td>48966887</td><td>4823</td><td>1507</td><td>1405</td></tr>
  <tr><td>Bengali (bn)</td><td>27561</td><td>50031224</td><td>5839917</td><td>178</td><td>43</td><td>132</td></tr>
  <tr><td>Chechen (ce)</td><td>439</td><td>562391</td><td>68339</td><td>17</td><td>10</td><td>38</td></tr>
  <tr><td>Cherokee (chr)</td><td>59</td><td>78955</td><td>11556</td><td>6</td><td>1</td><td>6</td></tr>
  <tr><td>Czech (cs)</td><td>646080</td><td>84918442</td><td>114027574</td><td>12370</td><td>4310</td><td>2383</td></tr>
  <tr><td>German (de)</td><td>5525478</td><td>7257565213</td><td>934707414</td><td>109890</td><td>59549</td><td>23767</td></tr>
  <tr><td>Greek (el)</td><td>295937</td><td>396833046</td><td>53634195</td><td>5869</td><td>2199</td><td>1959</td></tr>
  <tr><td>English (en)</td><td>22953734</td><td>2831015431</td><td>4357302494</td><td>618758</td><td>294826</td><td>93465</td></tr>
  <tr><td>Spanish (es)</td><td>3164341</td><td>413059248</td><td>62955918</td><td>73644</td><td>28496</td><td>17605</td></tr>
  <tr><td>Persian (fa)</td><td>1146456</td><td>1207151742</td><td>204651153</td><td>20381</td><td>7182</td><td>2944</td></tr>
  <tr><td>Fula (ff)</td><td>96</td><td>128576</td><td>17060</td><td>6</td><td>0</td><td>14</td></tr>
  <tr><td>Finnish (fi)</td><td>239296</td><td>428914597</td><td>44023582</td><td>2794</td><td>1323</td><td>1936</td></tr>
  <tr><td>French (fr)</td><td>6766096</td><td>8576232169</td><td>1272364513</td><td>85884</td><td>39282</td><td>20821</td></tr>
  <tr><td>Gujarati (gu)</td><td>8988</td><td>10578958</td><td>1547331</td><td>311</td><td>118</td><td>240</td></tr>
  <tr><td>Hausa (ha)</td><td>23437</td><td>33542354</td><td>5385245</td><td>259</td><td>40</td><td>118</td></tr>
  <tr><td>Hebrew (he)</td><td>476210</td><td>617562793</td><td>97213800</td><td>13278</td><td>6599</td><td>2147</td></tr>
  <tr><td>Hindi (hi)</td><td>90692</td><td>120210890</td><td>21155221</td><td>4046</td><td>1449</td><td>2434</td></tr>
  <tr><td>Hungarian (hu)</td><td>568839</td><td>709210304</td><td>85380427</td><td>10610</td><td>4450</td><td>1973</td></tr>
  <tr><td>Armenian (hy)</td><td>198688</td><td>268542332</td><td>32330669</td><td>4269</td><td>2183</td><td>749</td></tr>
  <tr><td>Indonesian (id)</td><td>404223</td><td>503307831</td><td>64705169</td><td>11180</td><td>2725</td><td>2763</td></tr>
  <tr><td>Icelandic (is)</td><td>33321</td><td>37953918</td><td>5241063</td><td>828</td><td>294</td><td>384</td></tr>
  <tr><td>Italian (it)</td><td>2211421</td><td>2842813803</td><td>413914078</td><td>46727</td><td>19382</td><td>9623</td></tr>
  <tr><td>Japanese (ja)</td><td>599453</td><td>580791400</td><td>4205266</td><td>21560</td><td>5107</td><td>2497</td></tr>
  <tr><td>Javanese (jv)</td><td>20529</td><td>24534947</td><td>3135389</td><td>434</td><td>119</td><td>328</td></tr>
  <tr><td>Georgian (ka)</td><td>110869</td><td>128682863</td><td>13900251</td><td>1539</td><td>455</td><td>442</td></tr>
  <tr><td>Kazakh (kk)</td><td>82239</td><td>97437119</td><td>10856557</td><td>1767</td><td>328</td><td>463</td></tr>
  <tr><td>Malagasy (mg)</td><td>17806</td><td>22237969</td><td>2777122</td><td>48</td><td>12</td><td>79</td></tr>
  <tr><td>Macedonian (mk)</td><td>83544</td><td>109737982</td><td>15301346</td><td>1447</td><td>284</td><td>528</td></tr>
  <tr><td>Malayalam (ml)</td><td>254186</td><td>407844848</td><td>34966538</td><td>1635</td><td>578</td><td>798</td></tr>
  <tr><td>Marathi (mr)</td><td>34176</td><td>38964783</td><td>5104805</td><td>1072</td><td>333</td><td>528</td></tr>
  <tr><td>Burmese (my)</td><td>25050</td><td>32277716</td><td>2346444</td><td>466</td><td>130</td><td>192</td></tr>
  <tr><td>Mazanderani (mzn)</td><td>1867</td><td>2184441</td><td>358170</td><td>39</td><td>18</td><td>72</td></tr>
  <tr><td>Dutch (nl)</td><td>1149502</td><td>1455471618</td><td>205590939</td><td>20481</td><td>8757</td><td>6257</td></tr>
  <tr><td>Punjabi (pa)</td><td>26763</td><td>38065954</td><td>6661363</td><td>509</td><td>90</td><td>228</td></tr>
  <tr><td>Polish (pl)</td><td>1658171</td><td>2269438656</td><td>291628512</td><td>29805</td><td>16031</td><td>4987</td></tr>
  <tr><td>Portuguese (pt)</td><td>1861950</td><td>2200190604</td><td>329245240</td><td>35091</td><td>11217</td><td>7615</td></tr>
  <tr><td>Russian (ru)</td><td>2475746</td><td>3243403532</td><td>410405284</td><td>49317</td><td>17319</td><td>12900</td></tr>
  <tr><td>Simple English (simple)</td><td>1042</td><td>1648009</td><td>207574</td><td>79</td><td>19</td><td>166</td></tr>
  <tr><td>Serbian (sr)</td><td>311606</td><td>379683724</td><td>51996543</td><td>5357</td><td>2121</td><td>1535</td></tr>
  <tr><td>Sundanese (su)</td><td>10853</td><td>12784831</td><td>1631605</td><td>223</td><td>51</td><td>180</td></tr>
  <tr><td>Swedish (sv)</td><td>654611</td><td>798513490</td><td>108859092</td><td>13227</td><td>5407</td><td>4015</td></tr>
  <tr><td>Swahili (sw)</td><td>19860</td><td>24114993</td><td>3289445</td><td>601</td><td>131</td><td>281</td></tr>
  <tr><td>Telugu (te)</td><td>106079</td><td>155946105</td><td>17814054</td><td>800</td><td>295</td><td>397</td></tr>
  <tr><td>Thai (th)</td><td>228603</td><td>209536522</td><td>13635803</td><td>6014</td><td>1799</td><td>947</td></tr>
  <tr><td>Tagalog (tl)</td><td>33426</td><td>42459061</td><td>6052162</td><td>590</td><td>175</td><td>525</td></tr>
  <tr><td>Turkish (tr)</td><td>658897</td><td>950765875</td><td>109644757</td><td>12602</td><td>3709</td><td>2812</td></tr>
  <tr><td>Tatar (tt)</td><td>23609</td><td>35501616</td><td>3961305</td><td>281</td><td>95</td><td>197</td></tr>
  <tr><td>Ukrainian (uk)</td><td>883369</td><td>1149425114</td><td>143511866</td><td>15053</td><td>4414</td><td>5800</td></tr>
  <tr><td>Urdu (ur)</td><td>53131</td><td>72646222</td><td>14204222</td><td>990</td><td>360</td><td>562</td></tr>
  <tr><td>Uzbek (uz)</td><td>67241</td><td>101189529</td><td>11410171</td><td>2103</td><td>536</td><td>390</td></tr>
  <tr><td>Vietnamese (vi)</td><td>381356</td><td>576413011</td><td>113053125</td><td>7706</td><td>2372</td><td>1333</td></tr>
  <tr><td>Wu (wu)</td><td>325</td><td>441370</td><td>29505</td><td>32</td><td>6</td><td>29</td></tr>
  <tr><td>Yoruba (yo)</td><td>3157</td><td>3879056</td><td>666500</td><td>142</td><td>11</td><td>89</td></tr>
  <tr><td>Chinese (zh)</td><td>113722</td><td>131874811</td><td>9352539</td><td>7070</td><td>2018</td><td>2762</td></tr>
  <tr><td>Zulu (zu)</td><td>937</td><td>1334536</td><td>138028</td><td>51</td><td>8</td><td>57</td></tr>
  <tr><th>All (60 languages)</th><th>60083121</th><th>74727560205</th><th>10794832477</th><th>1287054</th><th>568296</th><th>253373</th></tr>
</table>

Table 3: Statistics for all 60 languages included in MAC. ‘T.Chunks’ and ‘Contrib.’ refer to Text Chunks and Contributions, respectively. In this table, we include data that comes from Wikipedia users with at least two contributions.
<table>
  <tr>
    <th>Wiki</th>
    <th>Page</th>
    <th>Revision</th>
    <th>NS</th>
    <th>Time</th>
    <th>User</th>
    <th>Text</th>
  </tr>
  <tr>
    <td>1</td>
    <td>en</td>
    <td>Don Henrie</td>
    <td>254495622</td>
    <td>0</td>
    <td>2008-11-27</td>
    <td>Ursula darling</td>
    <td>"His first feature film appearance was in the Aleister Crowley biopic Abbey ... performed a trio of tasks that gave no scientific proof to validate his condition and made extraordinary claims about himself."</td>
  </tr>
  <tr>
    <td>2</td>
    <td>en</td>
    <td>Talk: Aua, American Samoa</td>
    <td>332113525</td>
    <td>1</td>
    <td>2009-12-16</td>
    <td>Tama. falealili</td>
    <td>"Who ever wrote this article is so irresponsible ... Tuisamoa was also the founder of the Manua islands and was known as the Tu-iManua, Tufiti and Tuitoga."<br>Hi, my name is Travis ... My professional motto: to create intuitive design and motion while remaining innovative and fresh.</td>
  </tr>
  <tr>
    <td>3</td>
    <td>en</td>
    <td>Titledlive</td>
    <td>152989039</td>
    <td>2</td>
    <td>2007-08-22</td>
    <td>Titledlive</td>
    <td>"Hi, exactly what edit of mine are you having any problem with? ... There is no reason for anyone to change that as it makes no change to the visual part of page, only that it is faster to build."</td>
  </tr>
  <tr>
    <td>4</td>
    <td>en</td>
    <td>Talk: Gamer9678</td>
    <td>622240710</td>
    <td>3</td>
    <td>2014-08-21</td>
    <td>Qed237</td>
    <td></td>
  </tr>
</table>

Table 4: MAC examples. The examples are presented across the four distinct namespaces (NS). For clarity and accessibility, we include only examples from the English Wikipedia to ensure that all readers can comprehend the textual content.

<table>
  <tr>
    <th>Parameter</th>
    <th>Value</th>
  </tr>
  <tr>
    <td>Batch Size</td>
    <td>64 (en - ru) / 32 (it - uk)</td>
  </tr>
  <tr>
    <td># Epochs</td>
    <td>6</td>
  </tr>
  <tr>
    <td>Learning Rate</td>
    <td>5e-5</td>
  </tr>
</table>

Table 5: Training hyperparameters for both SBERT\(_{AV}\) and SADIRI. We use different batch sizes for different languages due to large differences in data size among them. Please see Table 8 for the order of the languages.

<table>
  <tr>
    <th>Parameter</th>
    <th>Value</th>
  </tr>
  <tr>
    <td>\( k_1 \)</td>
    <td>1.5</td>
  </tr>
  <tr>
    <td>\( b \)</td>
    <td>0.75</td>
  </tr>
  <tr>
    <td>\( \epsilon \)</td>
    <td>0.25</td>
  </tr>
</table>

Table 6: BM25 hyperparameters

<table>
  <tr>
    <th>Model</th>
    <th>Total Training Time (hr)</th>
  </tr>
  <tr>
    <td>SBERT\(_{AV}\)</td>
    <td>5.48</td>
  </tr>
  <tr>
    <td>SADIRI</td>
    <td>12.82</td>
  </tr>
</table>

Table 7: Training times for SBERT\(_{AV}\) and SADIRI on a single NVIDIA A40-48GB. The numbers are sums across the 10 languages used in the experiments. Please see Table 8 for the full list.
Figure 5: Authors overlap network in MAC. Each node is a Wikipedia project (i.e., language). Weighted edges are assigned according to the authors’ overlap between node pairs. The most popular languages in Wikipedia are the most central nodes in the network. Closely related languages are likely to have a tight connection in the network. Node colors are based on the language family, which is detailed in the figure legend. The network was rendered using the Fruchterman-Reingold layout.

<table>
  <tr>
    <th rowspan="2">Language</th>
    <th colspan="3">Training</th>
    <th colspan="4">Test</th>
  </tr>
  <tr>
    <th>Training</th>
    <th>Validation</th>
    <th>Domain 0</th>
    <th>Domain 1</th>
    <th>Domain 2</th>
    <th>Domain 3</th>
    <th>Cross-domain</th>
  </tr>
  <tr>
    <td>English (en)</td>
    <td>152,569</td>
    <td>21,680</td>
    <td>43,671</td>
    <td>6,708</td>
    <td>2,241</td>
    <td>3,761</td>
    <td>32,916</td>
  </tr>
  <tr>
    <td>German (de)</td>
    <td>31,326</td>
    <td>4,431</td>
    <td>9,029</td>
    <td>1,659</td>
    <td>219</td>
    <td>579</td>
    <td>6,915</td>
  </tr>
  <tr>
    <td>French (fr)</td>
    <td>24,512</td>
    <td>3,385</td>
    <td>6,936</td>
    <td>482</td>
    <td>299</td>
    <td>407</td>
    <td>3,743</td>
  </tr>
  <tr>
    <td>Spanish (es)</td>
    <td>21,490</td>
    <td>3,021</td>
    <td>6,074</td>
    <td>293</td>
    <td>92</td>
    <td>404</td>
    <td>2,602</td>
  </tr>
  <tr>
    <td>Russian (ru)</td>
    <td>20,234</td>
    <td>2,916</td>
    <td>5,738</td>
    <td>284</td>
    <td>42</td>
    <td>218</td>
    <td>1,731</td>
  </tr>
  <tr>
    <td>Italian (it)</td>
    <td>14,970</td>
    <td>2,164</td>
    <td>4,192</td>
    <td>207</td>
    <td>91</td>
    <td>391</td>
    <td>1,907</td>
  </tr>
  <tr>
    <td>Portuguese (pt)</td>
    <td>11,874</td>
    <td>1,728</td>
    <td>3,425</td>
    <td>77</td>
    <td>64</td>
    <td>181</td>
    <td>994</td>
  </tr>
  <tr>
    <td>Polish (pl)</td>
    <td>9,255</td>
    <td>1,342</td>
    <td>2,580</td>
    <td>69</td>
    <td>302</td>
    <td>137</td>
    <td>1,822</td>
  </tr>
  <tr>
    <td>Dutch (nl)</td>
    <td>7,118</td>
    <td>1,029</td>
    <td>2,027</td>
    <td>79</td>
    <td>18</td>
    <td>102</td>
    <td>825</td>
  </tr>
  <tr>
    <td>Ukrainian (uk)</td>
    <td>6,446</td>
    <td>940</td>
    <td>1,884</td>
    <td>27</td>
    <td>23</td>
    <td>37</td>
    <td>377</td>
  </tr>
</table>

Table 8: The number of text pairs (authors) for training, validation, and single-language test sets.
Figure 6: The Success@1 scores for out-of-language generalization for the top ten languages. The scores highlighted in the orange boxes are the in-language scores for which the models are trained and evaluated in the same language. Boxes in this figure are auto-shaded such that darkness correlates with Success scores across the entire table.

Figure 7: Success@1 scores for the two baseline models. The scores for RQ2 are the averages across only unseen evaluation languages, and the scores for RQ3 are the averages across only unseen domains (domains 1 to 3). Neither baseline model is trained on MAC. The training language indicates the evaluation set in each column is the same as the evaluation set for the fine-tuned models trained in that language.

<table>
  <tr>
    <th>Type</th>
    <th>Name</th>
    <th>License</th>
  </tr>
  <tr>
    <td rowspan="2">Model</td>
    <td>paraphrase-multilingual-mpnet-base-v2 (278M)</td>
    <td>Apache-2.0</td>
  </tr>
  <tr>
    <td>xlm-roberta-base (278M)</td>
    <td>MIT</td>
  </tr>
  <tr>
    <td rowspan="5">Library</td>
    <td>Gensim</td>
    <td>LGPL-2.1</td>
  </tr>
  <tr>
    <td>Transformers</td>
    <td>Apache-2.0</td>
  </tr>
  <tr>
    <td>Sentence Transformers</td>
    <td>Apache-2.0</td>
  </tr>
  <tr>
    <td>SADIRI</td>
    <td>N/A</td>
  </tr>
  <tr>
    <td>mwparserfromhell</td>
    <td>MIT</td>
  </tr>
  <tr>
    <td>wikitextparser</td>
    <td>GPL-3.0</td>
  </tr>
  <tr>
    <td>pycld2</td>
    <td>Apache-2.0</td>
  </tr>
</table>

Table 9: Existing artifacts used in this work as well as their corresponding licenses. The number of parameters for each model is provided in parentheses following the model name. The link to the documentation or the official website of each artifact is provided through the hyperlink embedded in the model name. For more details on SADIRI, please refer to the original work (Fincke and Boschee, 2024).
<table>
  <tr>
    <th rowspan="6">Research Question</th>
    <th colspan="10">SBERT<sub>AV</sub></th>
    <th colspan="10">SADIRI</th>
  </tr>
  <tr>
    <th>en</th><th>de</th><th>fr</th><th>es</th><th>ru</th><th>it</th><th>pt</th><th>pl</th><th>nl</th><th>uk</th>
    <th>en</th><th>de</th><th>fr</th><th>es</th><th>ru</th><th>it</th><th>pt</th><th>pl</th><th>nl</th><th>uk</th>
  </tr>
  <tr>
    <td>RQ1</td><td>0.34</td><td>0.28</td><td>0.27</td><td>0.28</td><td>0.22</td><td>0.22</td><td>0.27</td><td>0.21</td><td>0.26</td><td>0.23</td>
    <td>0.36</td><td>0.28</td><td>0.28</td><td>0.29</td><td>0.24</td><td>0.25</td><td>0.29</td><td>0.22</td><td>0.26</td><td>0.25</td>
  </tr>
  <tr>
    <td>RQ2</td><td>0.27</td><td>0.26</td><td>0.26</td><td>0.25</td><td>0.25</td><td>0.24</td><td>0.22</td><td>0.22</td><td>0.21</td><td>0.20</td>
    <td>0.29</td><td>0.26</td><td>0.27</td><td>0.26</td><td>0.26</td><td>0.26</td><td>0.25</td><td>0.24</td><td>0.23</td><td>0.24</td>
  </tr>
  <tr>
    <td>RQ3</td><td>0.36</td><td>0.43</td><td>0.57</td><td>0.47</td><td>0.52</td><td>0.46</td><td>0.49</td><td>0.44</td><td>0.53</td><td>0.54</td>
    <td>0.38</td><td>0.43</td><td>0.59</td><td>0.51</td><td>0.53</td><td>0.47</td><td>0.53</td><td>0.47</td><td>0.57</td><td>0.61</td>
  </tr>
  <tr>
    <td>RQ4</td><td>0.09</td><td>0.05</td><td>0.08</td><td>0.09</td><td>0.09</td><td>0.08</td><td>0.08</td><td>0.07</td><td>0.08</td><td>0.07</td>
    <td>0.06</td><td>0.05</td><td>0.07</td><td>0.06</td><td>0.06</td><td>0.07</td><td>0.07</td><td>0.07</td><td>0.07</td><td>0.04</td>
  </tr>
  <tr>
    <td>RQ5</td><td>0.13</td><td>0.10</td><td>0.20</td><td>0.17</td><td>0.13</td><td>0.13</td><td>0.19</td><td>0.21</td><td>0.17</td><td>0.27</td>
    <td>0.14</td><td>0.10</td><td>0.20</td><td>0.18</td><td>0.12</td><td>0.13</td><td>0.20</td><td>0.21</td><td>0.17</td><td>0.27</td>
  </tr>
</table>

Figure 8: Success@1 scores for the two fine-tuned models. The scores for RQ2 are the averages across only unseen evaluation languages, and the scores for RQ3 are the averages across only unseen domains (domains 1 to 3).

![Success@1 scores for the two fine-tuned models](page_186_186_1077_312.png)

<table>
  <tr>
    <th rowspan="11">Evaluation Language</th>
    <th colspan="10">SBERT<sub>AV</sub></th>
    <th colspan="10">SADIRI</th>
  </tr>
  <tr>
    <th>en</th><th>de</th><th>fr</th><th>es</th><th>ru</th><th>it</th><th>pt</th><th>pl</th><th>nl</th><th>uk</th>
    <th>en</th><th>de</th><th>fr</th><th>es</th><th>ru</th><th>it</th><th>pt</th><th>pl</th><th>nl</th><th>uk</th>
  </tr>
  <tr>
    <td>en</td><td>0.28</td><td>0.39</td><td>0.46</td><td>0.39</td><td>0.40</td><td>0.39</td><td>0.37</td><td>0.35</td><td>0.35</td><td>0.33</td><td>0.31</td>
    <td>0.45</td><td>0.40</td><td>0.40</td><td>0.39</td><td>0.39</td><td>0.39</td><td>0.38</td><td>0.36</td><td>0.35</td><td>0.36</td>
  </tr>
  <tr>
    <td>de</td><td>0.20</td><td>0.33</td><td>0.38</td><td>0.36</td><td>0.35</td><td>0.35</td><td>0.34</td><td>0.32</td><td>0.32</td><td>0.30</td><td>0.30</td>
    <td>0.38</td><td>0.37</td><td>0.36</td><td>0.35</td><td>0.35</td><td>0.35</td><td>0.34</td><td>0.33</td><td>0.33</td><td>0.32</td>
  </tr>
  <tr>
    <td>fr</td><td>0.22</td><td>0.34</td><td>0.37</td><td>0.35</td><td>0.38</td><td>0.35</td><td>0.34</td><td>0.32</td><td>0.31</td><td>0.31</td><td>0.30</td>
    <td>0.39</td><td>0.36</td><td>0.37</td><td>0.35</td><td>0.35</td><td>0.35</td><td>0.34</td><td>0.33</td><td>0.32</td><td>0.32</td>
  </tr>
  <tr>
    <td>es</td><td>0.24</td><td>0.36</td><td>0.40</td><td>0.38</td><td>0.39</td><td>0.41</td><td>0.37</td><td>0.37</td><td>0.36</td><td>0.35</td><td>0.34</td><td>0.33</td>
    <td>0.41</td><td>0.38</td><td>0.39</td><td>0.40</td><td>0.38</td><td>0.38</td><td>0.38</td><td>0.35</td><td>0.35</td><td>0.35</td>
  </tr>
  <tr>
    <td>ru</td><td>0.18</td><td>0.30</td><td>0.35</td><td>0.34</td><td>0.34</td><td>0.33</td><td>0.32</td><td>0.31</td><td>0.30</td><td>0.29</td><td>0.30</td>
    <td>0.36</td><td>0.33</td><td>0.33</td><td>0.33</td><td>0.33</td><td>0.33</td><td>0.33</td><td>0.31</td><td>0.30</td><td>0.30</td>
  </tr>
  <tr>
    <td>it</td><td>0.23</td><td>0.32</td><td>0.37</td><td>0.34</td><td>0.37</td><td>0.35</td><td>0.34</td><td>0.36</td><td>0.32</td><td>0.31</td><td>0.29</td>
    <td>0.38</td><td>0.34</td><td>0.34</td><td>0.34</td><td>0.34</td><td>0.35</td><td>0.33</td><td>0.33</td><td>0.31</td><td>0.31</td>
  </tr>
  <tr>
    <td>pt</td><td>0.28</td><td>0.39</td><td>0.45</td><td>0.41</td><td>0.43</td><td>0.43</td><td>0.41</td><td>0.40</td><td>0.41</td><td>0.39</td><td>0.38</td><td>0.36</td>
    <td>0.45</td><td>0.41</td><td>0.41</td><td>0.41</td><td>0.41</td><td>0.40</td><td>0.41</td><td>0.39</td><td>0.38</td><td>0.37</td>
  </tr>
  <tr>
    <td>pl</td><td>0.22</td><td>0.31</td><td>0.37</td><td>0.36</td><td>0.35</td><td>0.34</td><td>0.34</td><td>0.33</td><td>0.33</td><td>0.31</td><td>0.31</td>
    <td>0.36</td><td>0.35</td><td>0.34</td><td>0.33</td><td>0.33</td><td>0.33</td><td>0.32</td><td>0.31</td><td>0.31</td><td>0.31</td>
  </tr>
  <tr>
    <td>nl</td><td>0.28</td><td>0.36</td><td>0.41</td><td>0.39</td><td>0.40</td><td>0.39</td><td>0.38</td><td>0.37</td><td>0.36</td><td>0.38</td><td>0.34</td>
    <td>0.40</td><td>0.38</td><td>0.37</td><td>0.37</td><td>0.37</td><td>0.37</td><td>0.37</td><td>0.36</td><td>0.36</td><td>0.35</td>
  </tr>
  <tr>
    <td>uk</td><td>0.20</td><td>0.32</td><td>0.37</td><td>0.39</td><td>0.38</td><td>0.37</td><td>0.39</td><td>0.36</td><td>0.34</td><td>0.36</td><td>0.35</td>
    <td>0.39</td><td>0.37</td><td>0.37</td><td>0.37</td><td>0.37</td><td>0.37</td><td>0.36</td><td>0.35</td><td>0.34</td><td>0.32</td>
  </tr>
</table>

Figure 9: The Success@8 scores for out-of-language generalization for the top ten languages. The scores highlighted in the orange boxes are the in-language scores for which the models are trained and evaluated in the same language. Boxes in this figure are auto-shaded such that darkness correlates with Success scores across the entire table.

![Success@8 scores for out-of-language generalization](page_186_661_1077_312.png)
<table>
  <tr>
    <th rowspan="2">Research Question</th>
    <th colspan="10">SBERT</th>
    <th colspan="10">AV Model {Training Language}</th>
  </tr>
  <tr>
    <th>en</th><th>de</th><th>fr</th><th>es</th><th>ru</th><th>it</th><th>pt</th><th>pl</th><th>nl</th><th>uk</th>
    <th>en</th><th>de</th><th>fr</th><th>es</th><th>ru</th><th>it</th><th>pt</th><th>pl</th><th>nl</th><th>uk</th>
  </tr>
  <tr>
    <td>RQ1</td><td>0.28</td><td>0.20</td><td>0.22</td><td>0.24</td><td>0.18</td><td>0.23</td><td>0.28</td><td>0.22</td><td>0.28</td><td>0.20</td>
    <td>0.39</td><td>0.33</td><td>0.34</td><td>0.36</td><td>0.30</td><td>0.32</td><td>0.39</td><td>0.31</td><td>0.36</td><td>0.32</td>
  </tr>
  <tr>
    <td>RQ2</td><td>0.23</td><td>0.24</td><td>0.23</td><td>0.23</td><td>0.24</td><td>0.23</td><td>0.23</td><td>0.23</td><td>0.23</td><td>0.24</td>
    <td>0.34</td><td>0.34</td><td>0.34</td><td>0.34</td><td>0.35</td><td>0.35</td><td>0.34</td><td>0.35</td><td>0.34</td><td>0.34</td>
  </tr>
  <tr>
    <td>RQ3</td><td>0.36</td><td>0.45</td><td>0.57</td><td>0.54</td><td>0.53</td><td>0.52</td><td>0.63</td><td>0.54</td><td>0.73</td><td>0.69</td>
    <td>0.45</td><td>0.55</td><td>0.70</td><td>0.65</td><td>0.65</td><td>0.62</td><td>0.71</td><td>0.67</td><td>0.74</td><td>0.85</td>
  </tr>
  <tr>
    <td>RQ4</td><td>0.13</td><td>0.13</td><td>0.13</td><td>0.13</td><td>0.13</td><td>0.13</td><td>0.13</td><td>0.13</td><td>0.13</td><td>0.13</td>
    <td>0.03</td><td>0.03</td><td>0.03</td><td>0.03</td><td>0.03</td><td>0.03</td><td>0.03</td><td>0.03</td><td>0.03</td><td>0.03</td>
  </tr>
  <tr>
    <td>RQ5</td><td>0.14</td><td>0.09</td><td>0.18</td><td>0.17</td><td>0.12</td><td>0.14</td><td>0.20</td><td>0.22</td><td>0.22</td><td>0.26</td>
    <td>0.17</td><td>0.13</td><td>0.24</td><td>0.23</td><td>0.16</td><td>0.19</td><td>0.25</td><td>0.26</td><td>0.25</td><td>0.34</td>
  </tr>
</table>

Figure 10: Success@8 scores for the two baseline models. The scores for RQ2 are the averages across only unseen evaluation languages, and the scores for RQ3 are the averages across only unseen domains (domains 1 to 3). Neither baseline model is trained on MAC. The training language indicates the evaluation set in each column is the same as the evaluation set for the fine-tuned models trained in that language.

<table>
  <tr>
    <th rowspan="2">Research Question</th>
    <th colspan="10">SBERT<sup>AV</sup></th>
    <th colspan="10">AV Model {Training Language}</th>
  </tr>
  <tr>
    <th>en</th><th>de</th><th>fr</th><th>es</th><th>ru</th><th>it</th><th>pt</th><th>pl</th><th>nl</th><th>uk</th>
    <th>en</th><th>de</th><th>fr</th><th>es</th><th>ru</th><th>it</th><th>pt</th><th>pl</th><th>nl</th><th>uk</th>
  </tr>
  <tr>
    <td>RQ1</td><td>0.46</td><td>0.38</td><td>0.38</td><td>0.41</td><td>0.34</td><td>0.36</td><td>0.41</td><td>0.33</td><td>0.38</td><td>0.36</td>
    <td>0.45</td><td>0.37</td><td>0.37</td><td>0.40</td><td>0.33</td><td>0.35</td><td>0.41</td><td>0.32</td><td>0.36</td><td>0.35</td>
  </tr>
  <tr>
    <td>RQ2</td><td>0.39</td><td>0.37</td><td>0.38</td><td>0.37</td><td>0.37</td><td>0.36</td><td>0.34</td><td>0.34</td><td>0.32</td><td>0.31</td>
    <td>0.39</td><td>0.37</td><td>0.37</td><td>0.36</td><td>0.37</td><td>0.36</td><td>0.35</td><td>0.35</td><td>0.33</td><td>0.33</td>
  </tr>
  <tr>
    <td>RQ3</td><td>0.46</td><td>0.57</td><td>0.68</td><td>0.64</td><td>0.68</td><td>0.61</td><td>0.73</td><td>0.63</td><td>0.72</td><td>0.82</td>
    <td>0.46</td><td>0.55</td><td>0.69</td><td>0.63</td><td>0.68</td><td>0.63</td><td>0.73</td><td>0.62</td><td>0.70</td><td>0.83</td>
  </tr>
  <tr>
    <td>RQ4</td><td>0.23</td><td>0.14</td><td>0.20</td><td>0.20</td><td>0.20</td><td>0.20</td><td>0.19</td><td>0.17</td><td>0.17</td><td>0.18</td>
    <td>0.14</td><td>0.14</td><td>0.16</td><td>0.16</td><td>0.16</td><td>0.16</td><td>0.16</td><td>0.16</td><td>0.16</td><td>0.13</td>
  </tr>
  <tr>
    <td>RQ5</td><td>0.17</td><td>0.15</td><td>0.25</td><td>0.22</td><td>0.18</td><td>0.19</td><td>0.26</td><td>0.26</td><td>0.23</td><td>0.36</td>
    <td>0.17</td><td>0.14</td><td>0.24</td><td>0.23</td><td>0.17</td><td>0.18</td><td>0.26</td><td>0.25</td><td>0.23</td><td>0.33</td>
  </tr>
</table>

Figure 11: Success@8 scores for the two fine-tuned models. The scores for RQ2 are the averages across only unseen evaluation languages, and the scores for RQ3 are the averages across only unseen domains (domains 1 to 3).

Reviewer paper summaries: This paper introduces the Million Authors Corpus (MAC), a large-scale, cross-lingual, and cross-domain dataset derived from Wikipedia for authorship verification (AV). MAC consists of over 69 million textual chunks contributed by approximately 11.8 million authors in 60 languages

The paper introduces a cross-lingual and cross-domain Wikipedia dataset for authorship verification. The paper describes the construction of this dataset and the formulation of the task as an IR problem. The dataset provides long text chunks that can be used to verify authorship across domains and languages. The paper covers 5 research questions that can be answered by this dataset. The uniqueness of this dataset seems to be its size and the cross-language and cross-domain aspects. Four models of text similarity are used in experiments to answer the research questions.
Summary of strengths: - Unlike previous corpora, MAC enables evaluation in cross-lingual and cross-domain settings, filling a notable gap in AV datasets.
- The dataset has potential applications in identity verification, AI-generated text detection

1. The dataset provides a comprehensive extraction of text chunks and authorship attribution, and describes this process in detail.
2. The 4 models of text similarity is a good coverage of the major techniques.
3. 60 languages! Lots of text chunks with authorship info, potentially saving a lot of pre-processing time.
Summary of weaknesses: - Wikipedia’s collaborative nature can blur individual authorship.

1. The results and discussion sections are very short (~1 page). This section should be longer to discuss the research questions in more detail, perhaps providing some instance examples to ground the discussion.
2. The first 5 pages could be a lot more concise. The 5 research questions are really 3 questions with 2 dimensions of language and domain.
3. The paper is trying to cover too large of breadth of topics and has too many unnecessary filler paragraphs. This paper looks like it was copied from a thesis without much effort to rewrite as a paper.
4. The data itself and what it looks like is not clear. Some instance examples would help. How is this dataset accurate or reliable? All it has done is provide the labels of authorship for a chunk of text.
5. The motivation for the problem and the dataset isn't clear. Wikipedia has ways of tracking authors across domains and languages, and prominent authors tend to register their username so what interesting use cases does this dataset cover?
6. More analysis of the dataset is needed. Table 2 looks impressive in numbers, but what is meaningful about this breadth of languages?
7. Formulating the authorship verification task as an IR problem just looks prone to inaccuracies as the reliability of the data is not addressed. Success@8 is low and 8 seems generous.
8. Figure 3, the scores are almost the same for the across each matrix, are the models really generalizing across languages?
Comments, suggestions and typos: Include experiments with multi-lingual AV models (e.g., mBERT, XLM-R)

1. Section 3.2.4, is k=8 a common setting? What about other numbers for k?
2. The introduction mentioned that 3 research questions were answerable by the related work, but two were not. This should be highlighted better in the results, were the models better for those that were comparable? Do the current datasets not cover cross-lingual or cross-domain, what are the other work? The related work should follow the intro, given the breadth of topics that the paper is trying to cover.
3. There are a few typos in the paper.
Overall assessment scores: 4

1
Metareview score: 3
Metareview: The paper covers the topic of texts authorship identification. More specifically, the works presents a novel dataset of 69.85M from 11.84M authors covering all together all 60 languages of Wikipedia articles. The dataset is indeed massive. The work as well covers several languages of authorship identification across various domains, languages, and models.
Summary of reasons to publish: 1. Massive multilingual dataset on authorship identification.
2. The dataset (as collected from Wikipedia) will be published with very useful for the further research license.