Dealing with dialectal variation in the construction of the B asque historical corpus (2020)

Abstract: This paper analyses the challenge of working with dialectal variation when semi-automatically normalising and analysing historical Basque texts. This work is part of a more general ongoing project for the construction of a morphosyntactically annotated historical corpus of Basque called Basque in the Making (BIM): A Historical Look at a European Language Isolate, whose main objective is the systematic and diachronic study of a number of grammatical features. This will be not only the first tagged corpus of historical Basque, but also a means to improve language processing tools by analysing historical Basque varieties more or less distant from present-day standard Basque.

## introduction in many languages other than basque, different historical corpora exist which are annotated morphologically and syntactically, and allow lexical, morphological or syntactic searches in historical texts, e.g. the penn parsed corpora of historical english @xcite @xcite , the tycho brahe corpus [historical corpus of portuguese] @xcite , the icelandic parsed historical corpus @xcite or the parsed old and middle irish corpus @xcite . however, no appropriate instruments of this type have ever been developed for basque. in this paper we present the work we are carrying out in the construction of a historical corpus of basque considering the dialectal variation of historical texts. this work is part of a more general ongoing project called basque in the making (bim): a historical look at a european language isolate 1 , which has two main objectives: first, an exhaustive diachronic study of different grammatical features of the basque language; second, the creation of a morphosyntactically annotated historical corpus that will enable comprehensive diachronic analysis. the final output of our project will be a search interface to browse the corpus. this interface must be useful for analysing diachronic syntax. therefore, it must be able to perform complex searches, both in terms of metadata @xcite and morphosyntactic characteristics (original form, lemma, part-of-speech, case ending, auxiliary verb root, time, aspect, mode, etc.), as well as any combination of them. taking all this into account, bim is an interdisciplinary project, where experts on linguistics (iker centre) 2 and natural language processing (hitz centre) 3 work together. this being the general scenario of the project, in this article we will focus on how we are dealing with dialectal variation in text normalisation. together with that, we will also mention the first steps taken towards the adaptation of the morphosyntactic analyser of standard basque developed by the ixa group to be able to correctly analyse historical texts (by definition dialectal ones). after having presented the overview of the project, in section 2 we will make an exposition of the related work. in section 3 we will present the corpus we are working with. the first steps taken in the adaptation of the morphosyntactic analyser will be presented in section 4. section 5 will be dedicated to the text normalisation process. in section 6 we will outline and discuss the main experimental results and the normalisation strategy adopted in the light of the results. finally, we will review the main conclusions and future work in section 7. ## related work historical and dialectal texts present problems from an nlp point of view, since nlp tools developed for contemporary standard language often fail in handling the linguistic varieties encountered in such texts. a majority of nlp tools are designed to process newspaper texts written in contemporary language, but the characteristics of the standardised modern texts are not shared by historical texts: standard variants adhere to orthographic and grammatical norms which may be comparatively recent in the written corpus of the language. therefore, the creation of a morphosyntactically annotated historical corpus using standard nlp tools needs a previous step such as normalisation of the texts. text normalisation has attracted a lot of interest over the past years, particularly normalisation of historical and dialectal texts, but also of informal texts such as those collected on twitter @xcite . several techniques have been used for this task, but it can be said that nowadays machine-learning based techniques are the most popular ones, that is, systems that learn from examples of standard-variant pairs. first methods used for automatic normalisation of historical texts were rule-based methods. for instance, in the construction of the tycho brahe corpus of historical portuguese, re-write rules were used to normalise historical texts @xcite . these methods do not need hand-annotating training data, but they do need linguist experts to detect which rules to apply in the normalisation process, which is costly and not always successful. character level statistical machine translation techniques (csmt) have been applied by @xcite by treating normalisation task as a translation problem. then, in @xcite they evaluate and compare their approach with a memory-based filtering and a levenshtein-based approach considering five languages. the smt-based approach generally works best. using the same csmt approach scherrer and erjavec (2016) develop a language-independent word normalisation method and test it on a task of modernising historical slovene words. they perform two sets of experiments: supervised and unsupervised. in the first one, they use the lexicon of word pairs as training data to build a csmt system. in the second one, they simulate a scenario in which word pairs are not available. they show that both methods produce significantly better results than the baselines. in our previous works @xcite a different approach is presented. the method learns to map phonological changes using a noisy channel model that combines weighted finite-state transducers (wfst) and language models. in @xcite our approach is compared with the csmt methods explained in @xcite and in @xcite using same historical corpora. the results show that the wfst approach produces similar or better scores for the six languages tested. lately, neural network architectures have become popular for a variety of nlp tasks, and they have been also applied to text normalisation @xcite @xcite . the results obtained by deep learning are good. however, these methods usually train on a lot of manually labelled data. in @xcite eight different nmt models are applied to the spelling normalisation task in several languages (english, german, hungarian, icelandic, and swedish). the authors carry out particular experimentation on swedish by increasing the size of the training set and they conclude that the performance of nmt models is highly related to the size. ## the corpus the basque historical corpus covers the most representative written production between the 15th and the 18th century. it is a time span within which all historical dialects of basque are represented and it is also the temporal span that divides archaic and old basque from early modern basque 4 . texts have been selected on the basis of: 1) their representativeness; 2) the existence of reliable editions; and 3) their social context. as far as corpus size is concerned, our goal is to get as much text as possible, because the wider the corpus, the more relevant the results of the research become. at the time being, we are creating a reference corpus of around one million words. given the problems and limitations associated to the written past of languages, specially in a language like basque, this range is considered acceptable for a historical corpus @xcite . ## methodology for text normalisation as we said before text normalisation is a necessary step in this project. once ancient and dialectal texts are normalised, the nlp tools developed for standard basque could be applied for the linguistic analysis of corpora. the normalisation method used is the wfst-based method presented in @xcite and @xcite and used also in @xcite . this approach uses phonetisaurus, a wfst-driven phonology tool @xcite , based on openfst @xcite , which learns mapping of phonological changes, using a noisy channel model. it is the strongest method found in previous work with dialect normalisation in basque @xcite and when it has been tested with other languages in order to be compared with the csmt methods proposed in @xcite and @xcite it has obtained similar results @xcite . for the moment we have ruled out neural methods due to the features mentioned of our corpus: several historical periods, several dialects and short texts. further research is necessary for a robust solution based on neural methods. the process of text normalisation will be carried out in two phases. first, part of the text will be manually annotated, and then, the rest of the text will be automatically normalised. the purpose of this article is not to explain the normalisation process that has already been explained in detail in @xcite and @xcite ; therefore here we will only give a brief summary of each of the normalisation phases. ## results and discussion in our previous work @xcite , the phonological induction inspired method we have chosen has been evaluated on three works of the 16th century selected from the sub-corpus: refranes y sentencias, leizarraga's translation of the new testament and etxepare's linguae vasconum primitiae. the method achieved an accuracy value between 80% and 95% in the experiments and the results suggested, on the one hand, that automatic normalisation should be done taking into account for the learning process texts from the same dialect or nearby dialects and, on the other hand, that the more text we have for the learning process the better the results we get. the main objective of the set of experiments presented below is to validate these hypotheses in order to define the normalisation strategy to be followed in the texts that will not be manually tagged. ## conclusions and future work in this paper we have presented the results obtained in the normalisation of basque historical texts using a weighted finite state transducer (wfst) driven phonology tool and taking into account the dialectal variation when designing the experiments. our initial hypotheses were two: i) by using texts from linguistically close dialects in the learning process, the results would improve, and ii) the results would improve as more text is fed. the results presented confirm the first hypothesis, since, in general, they are substantially improved by including in the learning process texts from the same dialectal group. however, the second hypothesis is not validated by the results, since in most cases by including all the texts available in the learning process, the results are worse than those obtained with texts from nearby dialects. this is with the exception of etxepare and lazarraga, but in both there are reasons that would explain these results and which have been discussed in section 6.2. these results have led us to conclude that the best normalisation strategy for texts that will be normalised in a fully automatic way is to create a normaliser for each dialectal group. in the same way, the results obtained with the work written in souletin have confirmed that given the marginal character of this dialect, the normalisation must be done separately from the rest of the eastern dialects. we would like to underline that the results obtained reinforce the theories of historical dialectology in a way that has not been explored so far, that is, using computational methods. we would like to continue exploring this line of research in future work and in collaboration with dialectologists. on the other hand, we have presented the preliminary work carried out to adapt the morphosyntactic analyser for standard basque in order to correctly recognise and analyse the historical and dialectal variation of auxiliary verbs. thanks to this work we have included five new paradigms in the lexicon used by the analyser. we have to emphasise that from the philological and linguistic point of view this has been a significant work, since three of these five dialectal verbal paradigms have been standardised for the first time. the next steps in this project are the following: i) the semi-automatic normalisation of the 18th century works included in the reference sub-corpus; ii) the automatic normalisation of the rest of the corpus, and finally, iii) the morphosyntactic analysis of the corpus. for the normalisation we have already defined the strategy to follow, and for the automatic morphosyntactic analysis of the corpus we will use the eustagger tool. a sample of this automatic analysis will be revised manually to detect errors and to proceed to the annotation of interesting morphosyntactic phenomena from the point of view of diachronic syntax. this syntactically annotated corpus will facilitate the systematic study of a number of grammatical features of the basque in a diachronic way by means of a search interface, on which we are already working, and it will be the only tool of these characteristics existing for the basque language. the annotated corpus and the search interface will be public and freely available to the research community. finally, we think it might be interesting to test the performance of different methods of normalisation based on neural networks. we are aware of the limitations of our corpus, in terms of its extension, in order to achieve adequate results using methods based on neural networks. for this reason, we have decided to work on this line of research in the elaboration of the corpus that will include the next stages in the history of the basque language, early and late modern basque. we are already working on the choice of the works that will make up this new corpus. with this new project, we plan to extend the historical corpus to 12 million words, which would allow us to work with more guarantees on methods based on neural networks. in any case, there are authors @xcite who have tried different strategies to deal with these types of limitations and we believe that it would be an interesting line of investigation in the case of the basque language.