Towards cross-lingual application of language-specific P o S tagging schemes (2021)

Abstract: We describe the process of conversion between the PoS tagging schemes of two languages, the Icelandic MIM-GOLD tagging scheme and the Faroese Sosialurin tagging scheme. These tagging schemes are functionally similar but use separate ways to encode fine-grained morphological information on tokenised text. As Faroese and Icelandic are lexically and grammatically similar, having a systematic method to convert between these two tagging schemes would be beneficial in the field of language technology, specifically in research on transfer learning between the two languages. As a product of our work, we present a provisional version of Icelandic corpora, prepared in the Faroese PoS tagging scheme, ready for use in cross-lingual NLP applications.

## introduction part of speech (pos) tagging is the process of labelling words and symbols of running text based on their lexical category and morphological features. text corpora that have been pos-tagged in this way serve as a valuable tool in various fields of linguistic research and language technology. the specifics and format of the pos tags used, the tagging scheme, varies greatly between languages and applications. in the current project, we focus on two languages with significant linguistic similarities, icelandic and faroese, and pos tagging schemes for the two which overlap significantly in function; the mim-gold tagging scheme @xcite ) and the sosialurin tagging scheme @xcite , respectively. icelandic and faroese are distinct yet relatively similar languages, with their similarities especially apparent in morphology and syntax. while icelandic has seen significant gains in the field of language technology (lt) over the past few decades @xcite , the same is not true for faroese. due the similarities between the two, there is a real possibility that employing transfer learning, using icelandic data in tandem with faroese, to create effective lt tools and digital language resources for faroese. with the end goal of cross lingual transfer learning in mind, we focus on the task of pos tagging. our goal is to produce an effective way to map between the tagging schemes used for the two languages. this requires some revisions to one of the tagging schemes and assurance that a one-to-one mapping between tagsets is possible. the paper is structured as follows. section 2 discusses the possibilities of cross-lingual transfer learning between faroese and icelandic. section 3 describes the icelandic mim-gold tagging scheme and section 4 the faroese sosialurin tagging scheme. section 5 discusses the current differences between the two tagging schemes and section 6 details the procedure of converting between the two tagsets, while section 7 discusses possible alternatives such a conversion. section 8 concludes. ## faroese, icelandic and transfer learning the fundamental reason that makes icelandic nlp implementations applicable for faroese are the grammatical similarities between the two languages. these similarities are especially apparent in morphology, as both languages retain grammatical categories not apparent in other similar languages, e.g., four grammatical cases for nominals and an extensive conjugation system for verbs, to name a few. furthermore, the similarities also extend to the syntax of the languages and orthographies, although with various systematic differences in both. with this in mind it can be sup-posed that nlp solutions that perform well for icelandic may also perform well for faroese, especially data-driven applications. some data already exists on the efficacy of cross-lingual transfer learning between icelandic and faroese. the farparsald project @xcite focused on using a syntactically annotated corpus of faroese, the faroese parsed historical corpus (farpahc; @xcite , to train a syntactical parser, farparsald, based on the data-driven berkeley parser @xcite . the relatively small farpahc corpus, containing about 40,000 tokens, was supplemented with excerpts from its icelandic counterpart, the one million word icelandic parsed historical corpus (icepahc; @xcite . using this approach, the overall parsing accuracy of farparsald was raised from 75.44% to 78.06%, when 20% of the icepahc corpus, about 200,000 tokens, was added to the faroese training data. in effect, a training set made of mostly icelandic data returned better results than the faroese-only data. a similar approach may be taken in pos tagging faroese. abltagger @xcite , a recent bi-lstm driven pos tagger has shown impressive results in data-driven tagging of icelandic. this implementation might well serve as a platform for further transfer learning between the two languages. ## the mim-gold tagging scheme the icelandic tagging scheme we use in our project is the mim-gold tagging scheme, used in its eponymous corpus @xcite , a one million word, hand corrected corpus which serves as a gold standard for pos tagging icelandic. this tagging scheme is a modified version of the one used in the icelandic frequency dictionary (ifd) corpus @xcite , with various revisions made to the tagset to improve and streamline machine tagging of texts. in this tagging scheme, each token receives one pos tag, constisting of a tag string. each tag string consists of a series of characters, each having a particular morphosyntactic function, e.g., case, number, tense and grammatical gender. this is illustrated in ## the sosialurin tagging scheme the faroese pos tagging scheme we focus on is the one used in the sosialurin corpus, devised by @xcite as part of a larger project to create a pos-tagged corpus for the language and train automatic pos tagging software. this scheme is, to a large extent, based on the tagging scheme used in the ifd corpus for icelandic @xcite . this was possible because of the many similarities between icelandic and faroese in morphology and grammar in general. as in its icelandic counterpart, the faroese tagging scheme assigns each token a tag string, which contains a series of letters, each signifying relevant morphosyntactic information. the languages are not identical, however, and this is reflected in the faroese tagging scheme. furthermore, in a handful of grammatical categories, the sosialurin tagging scheme encodes fewer details than the icelandic one. in short, it is not as fine grained. finally, the tagging schemes use different symbols in the tag strings themselves, rendering the tagging schemes superficially different. an example of the sosialurin tagging scheme in practice is shown in hann he er is grivin buried undir under homrum. hamrar 'he is buried at hamrar.' as discussed in section 3 a number of revisions have been made to the ifd tagging scheme, token tag explanation hann ppmsn p-pronoun, p-personal m-masculine, s-singular, n-nominative er vnps3 v-verb, n-indicative, p-present, s-singular, 3-third person grivin vamsn v-verb, a-past participle, m-masculine, s-singular, n-nominative, undir ed e-preposition, d-governs dative homrum smsdl s-noun, m-masculine, s-singular, d-dative, l-location the revisions applied to the sosialurin tagging scheme include reworked numeral and punctuation tag strings, simplified case governance tagging for adverbs and the removal of a dedicated tag for past participles. furthermore, various new tag strings were introduced, based on features from the original ifd tagging scheme which were omitted from the original faroese scheme, e.g., distinction between different categories of pronouns. in addition to the mim-gold based revisions, we suggest a possible language-specific revision to the faroese taggings scheme. this entails the removal of distinction between person (1st, 2nd or 3rd) from verb tags in the original tagset. in faroese, person is never morphologically distinct in verbal plural forms, and may thus be reduntant in the tagging scheme, in theory. such a revision would improve the accuracy of machine-tagging, but downstream effects, e.g., on syntactic parsing, are not clear. as such, we leave it as an open suggestion and do not apply it in our project. with all revisions applied, the total number of theoretical tags in the sosialurin tagset is about 600. when applied to the original sosialurin corpus, 379 of these tags appear in the corpus, while the original corpus contained 390 unique tags. this is to be expected, mostly due to the simplified punctuation tags in the revised tagging scheme. the revisions applied to the faroese tagging scheme have been shown to positively affect overall pos tagging accuracy. when applied to the sosialurin corpus and evaluated using ten-fold cross validation, a faroese implementation of abltagger achieved an overall error reduction rate of 7.51% (hafsteinsson and ingason, 2021). ## remaining tagging scheme differences with the revisions based on mim-gold, described in section 4, to the faroese tagging scheme, the function of the two tagging schemes has become markedly more similar. the remaining aspect separating the two are language-specific features of the two schemes, specifically concerning verbal pos tags and the interpretation of article tags. both icelandic and faroese make a morphological distinction between two voices for verbs, the active and middle voices. the mim-gold tagging scheme for icelandic treats the verbal voice as a defining characteristic of all verbs. in the tag string, this is shown with the letter g for the active voice, and m for the middle voice. however, in the sosialurin tagging scheme for faroese, the verbal voice is instead treated as a verbal mood. this causes a discrepancy between the two tagging schemes, as the hierarchy of the verbal tag string is fundamentally different. a verb in icelandic, tagged as being in the indicative mood, could either be in the active or middle voice. this is not possible in the faroese tagging scheme, since the middle voice is considered a verbal mood; the hierarchical nature of the tag string doesn't allow two different mood labels. the reason for this difference might be differences in the languages themselves. although both faroese and icelandic exhibit what may be called a grammatical voice in verbs, the faroese form is likely reduced compared to the icelandic. in turn, the distinction between voice in faroese verbs is not as fundamental as in icelandic. with this in mind, the discrepancy as a whole may be tentatively circumvented in the tag conversion. a more significant difference between the two tagging schemes concerns the article word class. the icelandic tagging scheme tags uses a specific tag for definite articles, which reflects conventional analyses of icelandic grammar, in which the free-standing definite article 'hinn' is classified as a distinct word class, with no indefinite article being used. this free-standing article is thought of as a literary device of irregular usage, with the more common suffixed definite article being in more general use. conversely, faroese uses both definite and indefinite free-standing articles; 'tann' and 'hin' as definite and 'ein' as indefinite, along with a suffixed definite article, like @xcite . despite the apparent function of these words as articles within faroese, these words are tagged as indicative pronouns in the faroese tagging scheme, forgoing a distinct article tag altogether. furthermore, this seems to be an inherent difference between the conventional analyses between the two languages, which discourages the approach of simply adding an article tag to the tagging scheme. ## conversion between tagsets we suggest a partial solution to the effect of the inherent differences between the two tagging schemes, when converting between the two. concerning the verbal tags, when converting from the faroese tagging scheme to the icelandic, all verb pos tags not tagged as in the middle voice are mapped to equivalent verbal tags in the indicative mood, active voice. faroese verbs tagged in the middle are, conversely, mapped to the indicative middle voice. the opposite is done when converting from icelandic to the faroese tagging scheme, with the information on mood being overwritten, in the case of verbs that are in the middle voice. this approach produces a one-to-one mapping between the two tagging schemes and mitigates the discrepancy between them. this is especially efficient when only converting from the icelandic to faroese, which suffices use in cross-lingual transfer learning, as described in section 2. regarding the difference concerning the article class, further research is needed before an end result is settled on. the conversion between tagsets itself is not hampered by the absence of a distinct article tag in the faroese tagging scheme, but it may have an effect when applying datasets with converted tagging schemes, e.g., in transfer learning. future work will shed more light on this. with this in mind, we have set up simple python scripts which generate full tagsets for the tagging schemes and convert between the two. furthermore, we have produced preliminary datasets for use in testing of cross-lingual transfer learning, based on the mim-gold corpus for icelandic, the tagset of which was used in the development of the conversion described above. the conversion scripts and training datasets are tentatively made available on github foot_0 as products of this project. ## alternatives to conversion although we the main objective of the current project concerns the conversion between two tagging schemes, we are remain aware of the possibility of alternatives to this approach. one notable possibility would be to simply unify the two tagging schemes. with the modifications described in section 4 applied to the faroese tagging scheme, the two tagging schemes become near identical in function. if the end goal is to align one tagging scheme to the other, it begs the question whether a single tagging scheme would suit the needs of the two languages for use in nlp, e.g, by simply using the established icelandic mim-gold tagging scheme to describe both. the grammatical similarities between the two languages, discussed in section 2 further supports this argument. however, as the remaining discrepancies between the tagging schemes suggests, this approach is at best inopportune. at the moment, the conventional analyses of the two languages differ in such a way that simply applying the icelandic mim-gold tagging scheme on faroese text would be suboptimal. however, experimenting on this could be fruitful, and reconciling these differences in analysis at a future date may also be possible. circumventing the topic of the two tagging schemes discussed here, it should be noted that both faroese and icelandic have been described using the universal dependencies (ud) annotation scheme. three ud corpora are available for icelandic and two for faroese, with considerable overlap in the production of the faroese farpahc corpus and the icelandic icepahc and modern corpora, each being converted to the ud format from existing datasets, as described by @xcite . in this sense the two languages have already been described with a common annotation framework, although the ud annotating scheme is not strictly a dedicated pos tagging scheme compared to the two tagging schemes used in our project. ## conclusion we have described the process of conversion between the pos tagging schemes of two grammatically similar languages, the icelandic mim-gold tagging scheme and the faroese sosialurin tagging scheme. despite the two tagging schemes being functionally similar, they use separate ways to encode fine-grained morphological information on tokenised text. we described the differences between the two, along with revisions made to the faroese tagging scheme, with the goal of streamlining automatic pos tagging. we discussed grammatical differences between faroese and icelandic which result in minor discrepancies between the two tagging schemes and suggested a way to mitigate the effects of this when converting between the two. as a result, we produced a simple way to convert pos tags between the languages. the results of our work have been made available for use, consisting of python scripts for converting icelandic and faroese tagged corpora and preliminary converted training data, ready for application in cross-lingual nlp applications, with the end goal of it being of benefit in cross-lingual transfer learning.