T witt I rish: A U niversal D ependencies Treebank of Tweets in M odern I rish (2022)

Abstract: Modern Irish is a minority language lacking sufficient computational resources for the task of accurate automatic syntactic parsing of user-generated content such as tweets. Although language technology for the Irish language has been developing in recent years, these tools tend to perform poorly on user-generated content. As with other languages, the linguistic style observed in Irish tweets differs, in terms of orthography, lexicon, and syntax, from that of standard texts more commonly used for the development of language models and parsers. We release the first Universal Dependencies treebank of Irish tweets, facilitating natural language processing of user-generated content in Irish. In this paper, we explore the differences between Irish tweets and standard Irish text, and the challenges associated with dependency parsing of Irish tweets. We describe our bootstrapping method of treebank development and report on preliminary parsing experiments.

## introduction irish is a minority language spoken mostly in small communities in ireland called 'gaeltachtaí' @xcite but social media sites, such as twitter, provide a platform for irish speakers to communicate electronically from any location. users may reach a wide audience quickly, unconstrained by the conventions of standard language upheld by editors in publications, revealing the orthographic, lexical, and syntactic variation abundant in informal irish. analysis of up-to-date, real-world language data can provide an insight into how irish is used in everyday communication and how such informal texts compare to prescriptive norms of standardised language to which published texts tend to adhere. user-generated content (ugc), such as tweets, is a valuable, highly available resource for training syntactic parsers that can accurately process social media text. ugc is a genre with features different from those of both spoken language and standardised written language more traditionally found in natural language processing (nlp) corpora. plank (2016) notes the advantages of utilising fortuitous data in order to create more adaptive, robust language technology. given that the accuracy of syntactic parsing tools has been shown to decline when evaluated on noisy ugc data @xcite and that domain foot_0 adaptation has been shown to improve parser performance for dependency annotation of english tweets @xcite and pos-tagging in irish tweets @xcite , the need for genre-specific resources is clear in order to reliably process this variety of data. the prerequisite, therefore, for research in this area is a data set of irish ugc. this research attempts to fill this gap through the development of twittirish, a treebank of irish tweets, within universal dependencies (ud) @xcite , a cross-lingually consistent framework for dependency-based syntactic parsing. twittirish provides linguistic information for irish in a digitally accessible format valuable for linguistic research and the development of nlp tools. open-source projects such as ud facilitate collaboration and rapid evolution of ideas among linguists internationally. in order to maintain optimum consistency with other ud treebanks, the annotation methodology employed in this research closely follows the general ud guidelines and the language-specific guidelines for irish while aiming to incorporate the most up-to-date recommendations @xcite for ugc in this evolving area of nlp. ugc, especially social media text, has recently become a popular focus within ud and nlp research more broadly @xcite @xcite @xcite @xcite @xcite and has encouraged active conversation around how best to represent it within this framework among the ud community. we carry out preliminary parsing experiments with twittirish, investigating the following two questions: how effective is a parser trained on the irish ud treebank @xcite , which contains only edited text and no ugc, when applied to tweets? and what difference do pretrained contextualised word embeddings make? we observe a difference of approximately 23 las points between twittirish and the iudt test set and find that the use of monolingual bert embeddings @xcite improves performance by over 10 las points. the paper is structured as follows: section 2 details the existing irish nlp resources we use for our research, section 3 outlines the development of the treebank, section 4 describes the characteristics of ugc evident in irish tweets, and section 5 presents parsing experiments and error analysis. ## irish nlp resources we use the following resources: indigenous tweets (it) foot_1 this project compiles statistics on social media data of 185 minority and indigenous languages including irish. all tweets in the twittirish treebank were sourced via it. ## twittirish development we combined 700 pos-tagged tweets from the ltc with 166 tweets more recently crawled by it in order to leverage previous linguistic annotations while also including newer tweets. this involved converting the ltc annotation scheme to that of the ud framework and then pos-tagging the new raw tweets. we provide further detail in appendix a. ## parsing experiments we compare the performance of two widely used neural dependency parsers on the twittirish test set, and examine the effect of using pre-trained contextualised word embeddings from a monolingual irish bert model (gabert). we report parsing performance broken down by sentence/tweet length, upos tags, and dependency labels and carry out a manual error analysis. further information is detailed in appendix b. ## conclusion presented in this paper is the novel resource, twit-tirish, the first universal dependencies treebank for irish ugc. analysis of this linguistic genre and anonymised examples of irish tweets are presented. this research facilitates the development of nlp tools such as dependency parsers for irish by providing a test set on which future irish language technology can be tested. future work will involve both further annotation and exploration of semi-supervised techniques.