M a C o C u: Massive collection and curation of monolingual and bilingual data: focus on under-resourced languages (2023)

Abstract: We present the most relevant results of the project MaCoCu: Massive collection and curation of monolingual and bilingual data: focus on under-resourced languages in its second year. To date, parallel and monolingual corpora have been produced for seven low-resourced European languages by crawling large amounts of textual data from selected top-level domains of the Internet; both human and automatic evaluation show its usefulness. In addition, several large language models pretrained on MaCoCu data have been published, as well as the code used to collect and curate the data.

## introduction this paper describes the main outcomes of the project macocu: massive collection and curation of monolingual and bilingual data: focus on underresourced languages @xcite , spanning from june 2021 to july 2023. macocu is aimed at building large and high-quality monolingual and parallel (with english) corpora for ten lowresourced european languages (see other existing initiatives, such as paracrawl 1 or oscar 2 exploit existing resources such as common crawl 3 or the internet archive. 4 our strategy con-sists in automatically crawling top-level domains (tld), potentially containing substantial amounts of text in the targeted languages, foot_0 and then applying a monolingual and a parallel curation pipelines. the evaluation of the first data release @xcite confirms the usefulness of these data for different natural-language processing tasks. ## collected corpora monolingual and parallel corpora are built from crawled data by applying a thorough cleaning process, including noise fixing/filtering and removal of near-duplicate/boilerplate text. corpora are then automatically annotated with: (a) document and paragraph ids; (b) language variety (e.g. british/american english); (c) documentlevel affinity to dsis identified through domain modelling @xcite ; (d) personal information; and (e) identification of translated text: either human or machine translations (only for parallel corpora). ## free/open-source pipeline the curation pipelines used to produce macocu corpora, monotextor foot_3 and bitextor, foot_4 have been re-leased under free/open-source licences. crawling and corpora-enrichment software have been also released under the macocu 10 github organisation.