O c W iki D isc: a Corpus of W ikipedia Talk Pages in O ccitan (2022)

Abstract: This paper presents OcWikiDisc, a new freely available corpus in Occitan, as well as language identification experiments on Occitan done as part of the corpus building process. Occitan is a regional language spoken mainly in the south of France and in parts of Spain and Italy. It exhibits rich diatopic variation, it is not standardized, and it is still low-resourced, especially when it comes to large downloadable corpora. We introduce OcWikiDisc, a corpus extracted from the talk pages associated with the Occitan Wikipedia. The version of the corpus with the most restrictive language filtering contains 8K user messages for a total of 618K tokens. The language filtering is performed based on language identification experiments with five off-the-shelf tools, including the new fasttext’s language identification model from Meta AI’s No Language Left Behind initiative, released in July 2022.

## introduction this paper provides two main contributions: we present ocwikidisc, a new, freely available corpus in occitan, and report results of language identification experiments executed as part of the corpusbuilding process. occitan is a romance language, mainly spoken in the south of france and in parts of spain and italy. it is considered a regional language in france but doesn't have the status of an official language. despite recent efforts to endow it with various nlp tools, it still remains low-resourced, especially when it comes to large, freely available corpora. our ocwikidisc corpus aims to remedy this lack by relying on user-generated content available on the web: we extract the corpus from the talk pages associated with the occitan wikipedia. thus, ocwikdisc contains messages posted by users, typically in direct user-to-user interactions. as such, it offers interesting possibilities for research not only in nlp, but also in corpus-based dialectology and wider linguistic studies. to the best of our knowledge, it is the first such corpus for occitan. it can be downloaded through zenodo foot_0 . since the extracted content contains a significant proportion of messages written in languages other than occitan, we perform language identification (lid) experiments. we test four off-the-shelf tools: langid @xcite and its python 3 implementation, py3langid foot_1 , developed by a. barbaresi; heli @xcite ; and the fasttext language identification models, both the original @xcite and the most recent @xcite , published as part of meta ai's no language left behind initiative. 3 we identify optimal lid strategies based on the desired outcome (optimizing for precision vs recall) and use them to filter the extracted corpus. these results also offer useful pointers for lid of occitan in general. the remainder of the paper is organized as follows. in section 2, we give a brief description of the main linguistic properties of occitan. section 3 offers more details on available nlp tools and resources for occitan. in section 4, we describe our corpus extraction process and present the initial corpus. section 5 is dedicated to language identification experiments, leading to several filtered versions of the corpus, which are presented in section 6. finally, in section 7, we give our conclusions and directions for future work. ## occitan: linguistic properties and dialectological situation occitan is a romance language spoken in the south of france, in parts of piedmont in italy and in val d'aran in spain. it does not have the status of an (1) t' aviái laissat un messatge totara you.dat have.1sg.impf leave.pst.ptcp a.sg.m message just 'i had just left you a message' ## occitan and nlp until recently, occitan belonged to the group of under-resourced languages. this situation was due to a combination of factors. first, the linguistic situation described above, compounding strong diatopic variation, absence of standardization, and use of multiple spelling norms, contributed to data sparsity. this was coupled with insufficient recognition on the institutional level, leading to a lack of human and financial resources available for nlp of occitan. this situation is currently evolving for the better: in france, regional languages have been recognized as part of the country's cultural heritage by the constitutional amendment article 75-1 published in 2008. since then, they have benefited from national and european initiatives to revitalize regional languages and help them enter the digital era. this has led to the creation of initial resources and tools for occitan. an electronic lexicon in lengadocian @xcite ) (850k entries), an online corpus of 3,4m words called batelòc @xcite and a pos-tagged corpus of 12k tokens @xcite were created as part of the restaure project 4 (2016-2018). during the linguatec project, foot_4 a 20k-token treebank following universal dependencies annotation guidelines was created @xcite . the existence of annotated training corpora led to initial experiments in pos-tagging and parsing occitan @xcite . a first neural text-to-speech model has also been created @xcite . all of these resources represent important steps forward for occitan. nonetheless, the language still remains low-resourced, especially when it comes to large, freely available corpora. the annotated corpora cited above are downloadable for research purposes, but they are fairly small, whereas batelòc, the largest currently available corpus in occitan, is not downloadable due to copyright limitations. a popular solution in this type of situation is to turn to the linguistic content available on the internet. this typically consists in crawling the top-level domain of the given language and transforming it into a corpus (see, e.g. @xcite . however, as pointed out by @xcite , such an approach can be ill-suited for lowresourced languages and linguistic varieties. many of them (including occitan) do not have a dedicated top-level domain, which makes the identification of url targets for crawling more challenging, and reliable lid systems more crucial in the process. moreover, low-resourced languages can also have a limited presence on the internet compared to more widely used languages. to illustrate, the latest version of the oscar corpus (ortiz suárez et al., 2019) foot_5 , based on the commoncrawl from november/december 2021, only contains 31k tokens in occitan, compared to, e.g. 41g tokens in french. we therefore turn to a more targeted solution: extracting content from wikipedia. ## extracting a corpus from wikipedia talk pages wikipedia content in occitan has been extracted and used as a corpus in previous research. for example, it is mentioned as part of the training material for the transformer-based multilingual language model mbert @xcite , but also for the lid tools fasttext @xcite , langid @xcite and heli @xcite . to the best of our knowledge, these training corpora have not been distributed. wikipedia content in occitan is also present in parallel corpora extracted from wikipedia such as wikimatrix @xcite . it would be possible to derive a monolingual occitan corpus from it, but the resulting corpus would contain individual sentences that would appear without their linguistic context, and would be accompanied by limited metadata. our goal, as stated in section 1, is to create a corpus suitable for a wider range of research: nlp applications, computational and corpus-based dialectology, as well as corpus-based linguistics. we therefore aim to preserve the linguistic integrity of the content in the corpus and to provide as much metadata as possible. also, we choose to focus on the talk pages rather than the encyclopedia pages themselves. wikipedia talk pages are dedicated to discussions between users, typically about article content and editing policies. these are direct user-to-user interactions on a variety of topics, but in general they share the same goal: improving the quality of the wikipedia content. they often combine elements of dialogue with elements of argumentative writing @xcite . given the nature of their content, the talk pages are a novel source of linguistic material for occitan. ## language identification experiments language identification is an nlp task which consists in automatically identifying the language of a given text @xcite . in order to perform this task on the extracted corpus, we first evaluate four off-the-shelf tools that integrate models for occitan. each of them is briefly presented below. ## filtered corpus in this section, we present the complete lidannotated corpus and its three filtered versions. the basic information about them is available in as expected, the version of the corpus favouring precision (ocwikidisc_precision) is the most restricted, with 8k messages and 618k tokens. this represents roughly half of the unfiltered corpus (in tokens). the difference between the corpus favouring recall (ocwikidisc_recall) and the one favouring f1-score (ocwikidisc_balanced) is relatively small for all reported measures. it remains to be seen if there is a qualitative difference in their content. it is important to note that the distribution of content across users is heavily skewed in all four versions of the corpus, both in terms of the number of messages and in terms of the number of tokens. the full distribution of messages across users is shown in figure 2 . as illustrated in the information on detected languages in ## conclusions and future work in this paper, we presented ocwikidisc, a new corpus in occitan extracted from wikipedia talk pages. the version of the corpus with the most restrictive language-based filtering contains 618k tokens. along with its extracted content, it also contains metadata about users, time of posting and discussion subjects, as well as language annotation produced using lid tools. to the best of our knowledge, it is the largest downloadable corpus for occitan. it can be downloaded from zenodo. we also presented lid experiments aimed at identifying occitan content in the initial extracted corpus, which is multilingual. we tested four offthe-shelf lid tools. in an initial experiment on an all-occitan sample, the best results were achieved by the new lid model from the fasttext tool and by heli. on a test sample extracted from ocwikidisc, fasttext's new model had the highest recall score, whereas heli achieved the most balanced precision and recall. combining the two tools optimized the precision. in the future, we will investigate making the lid on the corpus more fine-grained. currently, we perform lid at message level. given the amount of multilingual messages observed in our data, it could be beneficial to do it rather at sentence level, or even at word level. we will also examine the annotation of the romance languages found in the corpus, since a certain amount of confusion arising from the closely related languages in the corpus can be expected.