Named Entity Recognition for the I rish Language (2025)

Abstract: The Irish language has been deemed ‘definitely endangered’ (Moseley, 2012) and has been clas- sified as having ‘weak or no support’ (Lynn, 2023) regarding digital resources in spite of its status as the first official and national language of the Republic of Ireland. This research de- velops the first named entity recognition (NER) tool for the Irish language, one of the essen- tial tasks identified by the Digital Plan for Irish (Ní Chasaide et al., 2022). In this study, we produce a small gold-standard NER-annotated corpus and compare both monolingual and mul- tilingual BERT models fine-tuned on this task. We experiment with different model architec- tures and low-resource language approaches to enrich our dataset. We test our models on a mix of single- and multi-word named entities as well as a specific multi-word named entity test set. Our proposed gaBERT model with the implementation of random data augmentation and a conditional random fields layer demon- strates significant performance improvements over baseline models, alternative architectures, and multilingual models, achieving an F1 score of 76.52. This study contributes to advanc- ing Irish language technologies and supporting Irish language digital resources, providing a basis for Irish NER and identification of other MWE types.

## introduction despite being the first official and national language of the republic of ireland, irish faces a stark reality -it is 'definitely endangered' @xcite . furthermore, it is one of the two european union languages classified as having 'weak or no support' regarding digital resources @xcite . recognising this challenge, the digital plan for irish @xcite outlines a broad strategy aimed at strengthening technologies tailored to the irish language. central to this plan is the recognition of the urgent need for a named entity recognition (ner) tool for irish. such a tool not only facilitates various natural language processing (nlp) tasks but also represents a crucial step in providing much-needed essential digital support for the irish language community @xcite . existing research on irish mwes has also highlighted named entities (nes) as requiring special attention @xcite , as treatment of these constructions mirrors other mwe types, such as noun compounds @xcite . our research aims to address this gap in irish language technology by developing a base ner tool specifically tailored for the irish language, and the construction of the first gold-standard ner-annotated corpus for irish. ner is an information extraction task involving the identification of portions of text that refer to nes and the categorisation of these portions into predefined groups such as location, person, organisation, or other relevant categories. while ner may seem straightforward in its concept, it presents significant challenges. determining the category of a ne relies not only on the entity itself but also heavily on the context it appears in @xcite . state-of-the-art ner tools employ neural models that are pre-trained using language modeling tasks, which mitigates the need to have an abundance of annotated corpora @xcite . for irish, a ner tool represents a pivotal step towards improving digital content and interfaces in the language, leading to an increase in its use across digital environments. in the development of this tool, a small ner-annotated corpus has been constructed from existing contemporary irish text. this corpus is to be utilised with pre-trained language models, such as gabert @xcite for the ner task. due to the size of this corpus and also the size of irish text in the pre-training of multilingual language models, we use data augmentation approaches to enhance and enrich the corpus. furthermore, we add a conditional random fields (crf) layer and a bidirectional long shortterm memory (bi-lstm) crf layer to leverage contextual understandings captured by the models while incorporating the sequential modelling capabilities of crfs and bi-lstms @xcite . in this report, we evaluate several bert (bi-directional encoder representation from transformers) @xcite models for the ner task for irish, both monolingual and multilingual, and compare the above strategies. the models are tested on both a mixed-length ne test set and also a multi-word ne (mw-ne) test set. ## data the data we used for these experiments are comprised of 36,825 tokens and were collected from three sources: the iudt training set, the iudt test set @xcite , and publicly available transcripts from dáil proceedings (houses of the oireachtas, 2024). nes have previously been tagged in the iudt datasets using a designated label in the morphological features; a simple script was used to filter out these sentences for use as data. the domain is balanced, containing text from news, books, websites, and other sources. all sentences gathered from dáil proceedings were published between october 2023 and february 2024 (houses of the oireachtas, 2024). this ensured the data postdated the training completion of gabert in 2021 @xcite , and so was very unlikely to have been included in the training data for this model. as the dáil transcripts are largely english text, annotators manually filtered the text for irish sentences, and identified sentences containing named entities from these for use as data. the dáil text comprises of formal lan-guage often discussing proposed laws, government policies, national issues, and other parliamentary business. while the iudt data was tagged with a general "named entity" label, none of the above data sources had been previously labeled with finegrained ne information. annotation of the data collected was conducted using label-studio (label studio, 2020) by two annotators and carried out following specified annotation guidelines (see appendix a), labelling entities as persons (per), locations (loc), and organisations (org), using an iob2-tagging scheme (where b indicates the initial token of a named entity span e.g. b-loc, i indicates a non-inital token of a named entity span e.g. i-per, and o indicates the token is outside of a named entity span). iob2 tagging was previously implemented in a similar task for irish mwe identification @xcite . the annotators both performed annotation on all 1,249 sentences; all discrepancies were discussed during the annotation process and a decision was made on how to annotate that token/tokens. overall, there were very few discrepancies in the annotation. all datasets were curated carefully to have a balanced spread of named entity types within them. additionally, the validation and test sets each contained a majority of unseen nes (see figure 1 ), with approximately 75% of the nes being unseen in the training data. this enables us to investigate the capability of testing on unseen nes, mirroring the focus on unseen mwes in the parseme shared task edition 1.2 @xcite . altogether there was an 80.79%/8.01%/11.20% train/validation/test split implemented for this task. while mw-nes represent roughly 70% of the nes in training, the number of single and two-word mw-nes in the training set represent the majority of the nes (38% single nes and 30% two-word mw-nes). 4 models for the task we employ three bert @xcite models (gabert, mbert and xlmroberta) to evaluate monolingual and multilingual models on the task of ner for irish. gabert, a monolingual irish bert model, proved valuable to our research as it outperformed multilingual models on downstream tasks @xcite . mbert (multilingual bert) is a multilingual model containing irish training data @xcite . the third model used was xlmroberta which has achieved stateof-the-art performance on sequence labelling tasks and has outperformed mbert on cross-lingual classification on low-resource languages @xcite . irish is contained in the pre-training data for both mbert and xlmroberta, allowing us to evaluate their performances against monolingual gabert by fine-tuning these models on the ner task for the low-resource language irish. ## experimental set-up we utilised the adamw optimiser @xcite to fine-tune all parameters of the models. weight-decay was implemented as a regularisation technique to prevent overfitting due to the small size of the training data. a learning rate of 3e-5 and an epsilon value of 1e-8 were chosen to strike a balance between convergence speed and stability during training. the weight decay rate was set to 0.01 for parameters subject to weight decay. additionally, the maximum gradient norm was set to 1.0 to prevent exploding gradients. this scheduler adjusted the learning rate dynamically throughout the training process, starting with a warm-up phase of 0 steps and gradually linearly increasing the learning rate until reaching the total number of training steps. training sentences were passed to the models randomly so that the sources of the data were shuffled. validation and test sentences were passed to the model sequentially. training epochs were set to 10 with a patience of 2 epochs. if the validation loss increased two times, training stopped and the epoch with the lower validation loss (before the two increases) was used for testing @xcite . the maximum sequence length was set to 256, with training batch size being 32. all models were trained using a t4 gpu in the google colab environment. ## evaluation the main results from our experiments are presented in ## ethical considerations in the current climate of large language models, and massive data resources, the importance of data sovereignty and proper usage cannot be overlooked. the iudt data @xcite was in part selected for the construction of the gold-standard nerannotated corpus as it is under a cc by-sa 3.0 license and comprises publicly accessible textual data sourced from the new corpus of ireland-irish (ncii) @xcite , encompassing content from various sources such as websites, books, news articles, and other media. additionally, it includes supplementary publicly available data available under the open data directive (european parliament and council of the european union, 2019). the dáil proceedings used are also publicly available under this directive (european parliament and council of the european union, 2019). the lower energy demands of smaller bert models is an argument for their continued usage in such experiments, particularly for exploratory studies such as this one. insights from this work can be applied in future studies employing larger energyand resource-hungry models. ## future directions several avenues for advancing the scope and efficacy of ner for irish present themselves after this research work. firstly, acquiring more annotated data remains paramount given the scarcity of labelled corpora for low-resource languages such as irish. expanding the dataset used in this task could significantly bolster the performance of the models. a promising approach to this is self-training @xcite with irish wikipedia (vicipéid). this semi-supervised approach would mitigate the labour-intensive manual annotation employed in this research @xcite . also, improving the data augmentation approach could be a focal point for future enhancement. advanced techniques such as sentence-level resampling @xcite could provide substantial benefits. this approach involves modifying existing sentences to create new training examples, which leads to different syntactic and semantic variations to improve the model's accuracy and generalisation capabilities @xcite . additionally, a hybrid approach leveraging existing parsers should be considered @xcite , using the parser to identify nes in the text while bert-based classifiers could be trained to predict the ne label. newer models such as the uccix @xcite monolingual irish model are recently available for future experiments, and increases in performance can be weighed against the energy costs of training larger models. as mentioned in the european language equality project's report on the irish language @xcite , the gaois database of irish-language surnames @xcite and the national placenames and biographies database @xcite could also be leveraged to build a ner tool. futhermore, the task of ner can be integrated into a larger study on machine translation capabilities of handling these challenging constructions @xcite . additionally, a crf model could be used to provide a baseline for comparison with the models utilised in this work. finally, there is scope to combine this work with ongoing research in irish mwe processing, and in particular research on noun compounds, as both constructions could be treated simultaneously. ## conclusion we presented several architectures and training data setups for the ner for irish task as well as a gold-standard ner-annotated corpus. we proposed and evaluated multiple ner models for irish. of these, the best-performing model is gabert with the implementation of random data augmentation and a crf layer. despite the limited amount of data available for fine-tuning, this model has demonstrated remarkable performance improvements compared to alternative architectures, including baseline gabert and the multilingual models mbert and xlmroberta on the task of irish ner. with an f1 score of 76.52, the gabert rda crf model demonstrates robustness and accuracy in identifying both single-and mw-nes in irish text and generalises well to unseen data. our findings further highlight several noteworthy observations. the incorporation of a crf or bilstm crf layer yielded notable performance improvements. additionally, data augmentation strategies showed promising results at for different entity types and for general ne prediction. we hope that the goldstandard ner-annotated corpus for irish can provide a benchmark for future research and valuable training data. ultimately, our study sets the stage for continued progress in irish nlp, offering a vital step forward in supporting the irish language in the digital age.