Developing a F aroese P o S -tagging solution using I celandic methods (2020)

Abstract: We describe the development of a dedicated, high-accuracy part-of-speech (PoS) tagging solution for Faroese, a North Germanic language with about 50,000 speakers. To achieve this, a state-of-the-art neural PoS tagger for Icelandic, ABLTagger, was trained on a 100,000 word PoS-tagged corpus for Faroese, standardised with methods previously applied to Icelandic corpora. This tagger was supplemented with a novel Experimental Database of Faroese Inflection (EDFM), which contains morphological information on 67,488 Faroese words with about one million inflectional forms. This approach produced a PoS-tagging model for Faroese which achieves a 91.40% overall accuracy when evaluated with 10-fold cross validation, which is currently the highest reported accuracy for a dedicated Faroese PoS-tagger. The tagging model, morphological database, proposed revised PoS tagset for Faroese as well as a revised and standardised PoS tagged corpus are all presented as products of this project and are made available for use in further research in Faroese language technology

## introduction this paper describes the developement of a highaccuracy part-of-speech (pos) tagging solution for faroese, a north germanic spoken by about 50,000 people in the faroe islands, an autonomous territory in the kingdom of denmark. limited research has been performed on such an implementation for the language and as pos taggers are fundamental in various further implementations in natural language processing (nlp) and linguistic research, the need for new research is apparent. in the current project, as a basis for a faroese pos tagger, a stateof-the-art bi-lstm neural pos-tagging system for icelandic, abltagger @xcite , is used as a foundation to build on, in addition to various methods used in icelandic nlp research. the reason why these icelandic resources may also be applied to faroese is the extensive grammatical similarities between the two languages. these similarities are especially apparent in morphology, as both languages retain grammatical categories and nuances not apparent in related languages. in contrast with faroese, the last two decades have seen broad gains for icelandic in the field of language technology (lt), producing tools and databases which have enabled both various new technical implementations for the language and new opportunities in linguistic research. with the grammatical similarities of the two languages in mind, similar gains should be possible for faroese. using icelandic nlp tools and methods, a faroese pos-tagging model was produced which achieves an overall tagging accuracy of 91.40%. this is considerably higher than the previous dedicated pos tagger for faroese which achieved 87.00% using a similar tagset and trained on the same corpus, the faroese sosialurin corpus, which contains news articles, totalling about 100,000 words @xcite . furthermore, the current project produced various data set innovations which could prove useful in further nlp research for faroese. these include a proposed revised tagging scheme for faroese which is optimised for high-accuracy pos-tagging and a standardised version of hand-corrected pos tagged corpus for use in nlp projects. in addition to this, the project produced a novel inflection database for faroese, the experimental database of faroese morphology (edfm), which contains detailed morphological information on 67,488 faroese words, modelled on the dictionary of icelandic morphology @xcite . these data sets have been made available online 1 for fur-ther development and research in faroese language technology. section 2 outlines the abltagger pos-tagging system and previous work on faroese lt relevant to the current project, along with discussing the applicability and incentives to use icelandic materials and methods to develop lt solutions for faroese. section 3 describes the collection and preparation of the materials used to implement abltagger for faroese. section 4 describes the training and evaluation of the faroese abltagger model and section 5 discusses the current applicability of the faroese pos tagging model and the next steps in improving it as a whole. section 6 concludes. ## data collection and preparation a cornerstone of implementing the abltagger system for faroese is collecting and preparing the resources needed for the task. in the current project, this consisted of standardising the training corpus, revising an already existing tagging scheme for faroese and compiling an experimental morphological database. this is described in the following section. ## evaluation of tagger the abltagger system as described by @xcite uses dynet foot_7 @xcite . the architecture and model hyperparameters used in the evaluation were all unchanged from the original abltagger experiment. 9 this included stochastic gradient descent training with initial learning rate of 0.13, which decays 5% per epoch, running for 30 epochs for the full model. the hidden layer of the network has 32 layers. the input text, pos tags and morphological data are vectorized before use in the system and the resulting embeddings for words, characters and the morphological component have 128, 20 and 61 dimensions respectively. in accordance with previous experiments (see, e.g., @xcite @xcite , training and evaluation was done via 10-fold cross validation. in this approach, the whole data set at hand is used for both training and testing. this is especially useful when the data set is not large enough to effectively split into dedicated training and testing sets. ## application and further work it remains to be discussed how effectively the tagger produced in the current project can be applied in pos-tagging faroese text in general. in as is apparent in despite the high reported accuracy of the faroese model, two issues must be kept in mind. firstly, the overall accuracy, although high, does not approach the accuracy of the full abltagger model for icelandic, which, as discussed in section 2.3, has a similar tagset and overall morphology to faroese. in theory, a substantially higher tagging accuracy should be obtainable for faroese with the abltagger system, but it is limited by the size and contents of the training data used. secondly, as the sosialurin corpus only consists of news articles, its contents are likely not representative of faroeselanguage texts in general. the faroese abltagger model would thus perform well on news-like texts in its current form, but likely return sub-optimal results when tagging large, unseen texts in different genres, which is the main goal when developing a pos tagger. ## conclusion in this paper we have described the developement of a dedicated, high-accuracy pos-tagging solution for faroese. this was achieved by training abltagger, a state-of-the-art pos-tagging system for icelandic, on faroese language data which had been revised and formatted with methods and tools based on icelandic nlp research. this produced a faroese pos-tagging model which achieves a 91.40% overall tagging accuracy, when trained on the 100,000 word sosialurin corpus and evaluated using 10-fold cross validation. the last similar pos-tagging implementation for faroese achieved a 87.0% overall accuracy. thus, in the absence of recent comparable implementations, the faroese abltagger model may tentatively be considered the state-of-the-art for pos-tagging faroese. in addition to developing the pos tagger, this project produced various resources which could prove useful in further research in faroese language technology. these include a proposed revised postagging scheme for faroese, mainly based on the icelandic mim-gold tagging scheme, as well as the standardised and revised version of the corresponding sosialurin pos-tagged corpus. the same goes for the edfm, the experimental morphological database compiled for use with the tagger. although its format is based on its icelandic counterpart, dim, and is mostly compiled from already existing faroese dictionary data, with 67,488 word forms and about 1,000,000 inflectional forms, it is the first of its kind for faroese as a single, accessible data set designed for use in language technology implementations. as faroese digital language resources are, at the moment, few and far between, faroese language technology has ground to cover before it can be considered fully equipped to tackle recent innovations in the field. as the data sets and pos-tagging model produced in this project have been made available online, 12 they may well serve as a basis for further developments for fareoese, both to implement new nlp applications and provide further opportunities for linguistic research.