Generating I rish Text with a Flexible Plug-and-Play Architecture (2023)

Abstract: In this paper, we describe M-FleNS, a multilingual flexible plug-and-play architecture designed to accommodate neural and symbolic modules, and initially instantiated with rule-based modules. We focus on using M-FleNS for the specific purpose of building new resources for Irish, a language currently under-represented in the NLP landscape. We present the general M-FleNS framework and how we use it to build an Irish Natural Language Generation system for verbalising part of the DBpedia ontology and building a multilayered dataset with rich linguistic annotations. Via automatic and human assessments of the output texts we show that with very limited resources we are able to create a system that reaches high levels of fluency and semantic accuracy, while having very low energy and memory requirements.

## introduction natural language generation (nlg) for tasks including dialogue-turn generation and fact verbalisation is increasingly widely used in commercial systems. despite recent spectacular advances achieved by llms, in application contexts where accuracy and reliability are crucial, many commercial systems continue to use the same old template filler systems that have been around at least since the 1980s. 1 the other two main categories of nlg systems are neural language-model based (nlmb) systems, currently extremely popular in research systems, and rule and grammar based (rgb) systems, currently very unpopular. in contrast to template-based (tb) systems, nlmb systems have very high coverage, while also sharing tb systems' high fluency and robustness. however, the disadvantage of an nlmb system is that it cannot be guaranteed that the output will be free of grammatical errors or even that it will be semantically accurate. the latter is of particular concern as nlmb systems cannot be trusted not to omit essential content, make things up, or even insult users. moreover, such systems also tend not to be built for low-resource languages (lrls) languages because of the large amounts of data needed to build them. finally, nlmb systems often suffer from low variation, and very low energy efficiency, with the best current models having shockingly high carbon footprints. rgb systems on the other hand have become increasingly unpopular since the nlp field switched first to statistical systems, then to neural systems. while rgb systems tend to have low coverage, suprasentential fluency, and robustness as well as having to be built manually, they can be guaranteed to have high accuracy and grammaticality, as well as being efficient in terms of data and energy requirements, and suitable for lrls. according to the european language equality report for irish @xcite , irish is a low-resource language. in a survey of available resources for european languages, on a scale of 1-4, irish was classified as 4 having "weak or no support", and ranked 31st out of the 33 european languages surveyed. the report identifies a range of language technology gaps, mainly due to the lack of underlying data resources, dedicated funding and skill-sets, and finds that to date there has been little or no system development for automatic subtitling, information retrieval, information extraction, natu- the digital plan for the irish language (department of tourism, culture, arts, gaeltacht, sport media, 2022) notes that urgent action is needed if irish is to benefit from the digital revolution and to survive the threat of digital extinction. it notes two complementary approaches, knowledge-based and data-driven machine-learning methods, and states that both are needed and each brings specific advantages. a linguistic knowledge base provides a digital, explicit account of the structure of contemporary irish which is an important goal in itself, while machine-learning approaches can offer a quick and less labour-intensive route to developing certain technologies. both approaches are needed and, especially in the context or lrls, can be combined in specific systems. in this paper, we present a flexible plug-and-play architecture that addresses both knowledge-based and machine-learning-based gaps in irish natural language processing, by releasing a generation system and a rich dataset. while the current (single) multilingual flexible neuro-symbolic (m-flens) system is multilingual -generating text also e.g. in english, french, spanish, and catalan-, we focus here on its instantiation with rule-based modules for the generation of irish texts from dbpedia triple sets. below, we start by describing and motivating our architecture (section 2). next we describe the webnlg dataset, the forge generator and the irish morphology tools (section 3) we use. we present the extension to webnlg data-to-text for irish, and evaluate it via metrics and human assessment; we also present a new irish dataset with rich linguistic annotations produced with our instantiated architecture (section 4). we finish with a discussion of related work (section 5). the generation pipeline, 2 dataset 3 and an interactive demo for the generation of short wikipedia pages in irish or english 4 are all publicly available. ## data and tools in the following subsections, we describe the dataset (webnlg) and tools (forge and irish nlp) we use in our experiments. ## m-flens for irish natural language generation in this section, we describe our pipeline for the generation of irish texts from dbpedia triples, figure 2 : sample predarg template corresponding to the location property. ## related work rule-based nlg. there is a long tradition of rule-based natural language generation systems such as realpro @xcite , @xcite , igen @xcite , simplenlg @xcite , marquis @xcite , openccg @xcite , naturalowl @xcite , gendr @xcite and others. more recently, rdfjsrealb @xcite and forge @xcite were adapted to webnlg, but none were able to generate irish text. note that the idea of decomposing the generation process into steps has been the standard before the emergence of end-to-end systems, and that previous work on nlg already based their modules on the meaning-text theory, going back to realpro @xcite and marquis @xcite . it is however the first time that a plug-and-play architecture is proposed with these modules, and the first time that an irish rule-based nlg system is developed. irish datasets and language resources. there are few freely available monolingual irish corpora, and moreover, domain-specific irish datasets are scarce. resources are mostly targeted towards machine translation and/or language analysis tasks. with the exception of the webnlg 2023 data (and now the data presented in this paper), no datasets exist for text generation tasks @xcite . monolingual corpora. monolingual data include the new corpus for ireland (with fiction, news reports, official documents, etc.) @xcite , the unshuffled irish portion of the 2019 oscar corpus @xcite , the gaois corpus of contemporary irish @xcite , with news media and e-zines, or the irish wikipedia vicipéid, foot_10 which draws directly from fréamh an eolais, an irish-language encyclopedia of science and technology. 16 moreover, a corpus of idioms @xcite and universal dependency treebanks such as irish ud @xcite , pre-standard irish ud @xcite and twittirish @xcite are available. bilingual/parallel corpora. significant advances have been made in the collection and availability of bilingual corpora, including: (i) paracrawl v7 @xcite , a collection of parallel corpora crawled from multi-lingual websites; (ii) the gaois parallel corpus foot_12 of 26m irish words and 24.5m english words; and in particular, (iii) the irish-eu english-irish parallel corpus which was a direct outcome of the european language resource coordination project (elrc 18 ). this resource contains 195k+ parallel sentences, collected from various public bodies and government departments released via elrc-share 19 . in ireland all national translation data is collected by estór. 20 irish tools and models. the european language grid foot_16 catalogue lists a number of multilingual tools and services that support irish (e.g. bitextor, opus mt, systran). irish nlp tools (uí dhonnchadha, 2009) offers the only suite of text analysis in irish. transformer language models (lm) such as multilingual bert (m-bert) @xcite , and the language-agnostic bert sentence embedding @xcite ) support irish. the monolingual irish gabert lm was trained on over 7.9m sentences, and outperforms baselines for tasks such as dependency parsing and multiword expression identification. @xcite . ## conclusions we have presented a high-accuracy, energy and resource-efficient system for generating irish text which achieves a satisfactory quality of output. its modular architecture means that shortcomings can potentially be remedied by training statistical modules, such as a text structuring module for improved fluency, or by including enhanced rule-based modules which can be added to the pipeline. this type of modular rule-based nlg system is particularly suitable for low-resource languages, where large amounts of training data is not available, and can play an important role in generating accurate fact-based online language content, such as wikipedia pages. such systems can be developed incrementally and language documentation is an inherent and valuable by-product of the system. in addition, rule-based systems tend to suffer less from the negative and harmful biases which have been identified in the application of some llms.