R omanian Language Translation in the RELATE Platform (2022)

Abstract: This paper presents the usage of the RELATE platform for translation tasks involving the Romanian language. Using this platform, it is possible to perform text and speech data translations, either for single documents or for entire corpora. Furthermore, the platform was successfully used in international projects to create new resources useful for Romanian language translation.

## introduction translation platforms represent a subset of language technology (lt) platforms, providing services and resources for written or spoken language translation. artificial intelligence (ai) methods are used to implement the platform functionalities. these can be used either online, following a request-response model, or offline for processing large corpora, following an initial upload in the platform. @xcite notes that instead of competing with one another, platforms should be constructed to be interoperable and interact with each other to create synergies toward a productive lt ecosystem. we agree with this observation and consider that one way to achieve interoperability is through standardized formats for both input and output, allowing data to be exchanged between platforms. furthermore, web services can expose internal functionality, allowing for integration into other systems. this paper provides a detailed presentation of the translation functionalities of the relate platform . relate was developed as a modular, state-of-theart platform for processing the romanian language. available functions are provided by modules developed in multiple national and international projects, both in-house and by partner institutions. since its inception, one of its main goals was using standardized and easy-to-use file formats, combined with web apis, thus allowing integration with other systems (pȃis , , 2020) , as needed. component integration is performed directly by consuming the provided apis from a partner's servers or utilizing docker containers hosted on one or multiple servers associated with the platform. thus, it follows the philosophy behind the european language grid foot_1 . relate contains multiple translation functions for both text and speech. furthermore, it allows for development of translation related corpora. the paper is organized as follows: section 2 presents related work, section 3 describes the current architecture of the platform and its evolution. text translation functions are presented in section 4. speech to speech translation is covered in section 5. examples of large corpora, useful for translation, created within the platform in the context of international projects are given in section 6. finally we conclude in section 7. juremy 3 is an intelligent concordance search tool available for all combinations of the 24 eu official languages. it can be used to search legal and technical terminology in documents. in order to display the results, juremy uses eur-lex and iate databases and to reference the source document, juremy provides the user with a series of metadata such as document title, topic, iate evaluation, or work date. a plus of this online service is that it allows the user a customized search, but the services of juremy are only available after registration on the website. @xcite emphasize that numerous ai domains are underdeveloped at the national and international levels. even though ai technologies such as deep neural networks offer significant opportunities for many societal and economic challenges, there is still work to do until lt technologies can be considered viable solutions for all 24 eu official languages. another essential aspect un-3 https://juremy.com/ ) derlined by the authors is the enormous fragmentation of the european ai and lt landscape and consider that efforts should be made to ensure that all these platforms can exchange information, data and services to identify synergies in market capitalization. furthermore, the authors propose implementing standardized ways of exchanging repository entries that enable multiplatform and multi-vendor service workflows. ai4eu foot_2 and elg foot_3 platforms are presented as large european ecosystems that can assure interoperability between language technologies in europe. ## architecture of the relate platform the relate platform was implemented primarily for processing large text corpora @xcite . in addition, it also offers access to state-of-theart (sota) tools for the romanian language on a "per request" use case. recent developments allow speech processing of the romanian language by integrating tools for automatic speech recognition (asr), text-to-speech (tts) and speech-to-speech translation. modules available in the platform include: teprolin @xcite , nlp-cube @xcite , udpipe @xcite , ttl @xcite , mlpla @xcite , romaniantts @xcite , legal-domain ner @xcite , biomedical ner @xcite . also, some of the older, existing tools were exposed as web services and integrated in the platform. these modules account for the following operations: text segmentation (paragraph, sentence, token), phonetic transcription, lemmatization, syllabification, dependency parsing, text classification, term extraction, named entity recognition, diacritic restoration, abbreviation and numeral expansion, speech recording, asr, tts, text translation, and speech translation. additionally, web interfaces are available for querying the representative corpus of contemporary romanian language (corola) @xcite and the romanian wordnet @xcite . from a user perspective, the platform provides two interfaces: document-based and corpus-based. in the document-based interface, the user can work with a single file (text document or speech recording) and obtains the processing results in near realfigure 2: task-based architecture in the relate platform from a technical point of view, the platform uses the same underlying lt services to operate in a single document and task-based modes. for performance purposes, different services can be instantiated multiple times on the same hardware nodes or servers. this methodology allows scaling the platform annotation capabilities with the size of the corpora to be processed. a scheduling component distributes the tasks across the available services. the number of instances associated with each lt service differs based on the service's speed. faster services require fewer instances, while slower services benefit from more instances, thus allowing for increased parallelization of the processing queue. figure 2 depicts the scheduling component with associated task runner processes. the corpus management component provides basic functions such as file uploading (either a file-by-file process or an entire archive with multiple files), processing using the task-based system (task scheduling, task monitoring), visualization of both raw files and resulting annotations, and data export. processed files can be exported in the internal format or converted to project-specific formats available within the platform. the internal platform format is based on the conll-u plus foot_4 specification, which in turn is derived from the basic conll-u format foot_5 , employed in the universal dependencies foot_6 project. this is a tabular format with an additional document or segmentspecific metadata. the file starts with a metadata field ("global.columns") which describes the content associated with each column. for the relate platform, we keep the first ten columns corresponding to the basic conll-u file and add additional columns, as needed, based on the tasks executed on each corpus. therefore, the final annotated format may differ between corpora if different annotation tasks were executed. this can be further changed using format converters. currently, converters are available for exporting in other conll-u plus structures or xml documents. furthermore, due to our interest in linguistic linked data, various romanian language resources (barbu mititelu et al., 2020; pȃis , and barbu-mititelu, 2022; barbu mititelu et al., 2022), some of which were created within the relate platform, were converted into rdf format. examples of such resources are represented by the legalnero (pȃis , et al., 2021) named entity corpus and the robin technical acquisition speech corpus (rtasc) @xcite . figure 3 presents the different components available in the relate platform. the web front-end is the graphical user interface employed to interact with the components. it handles unauthenticated interactions, user authentication and authenticated requests. the back-end exposes platform functionality as web apis that can be consumed from the front end. this layer also allows for potential integration into other applications or platforms. at this level, corpus management functions are implemented, together with the task scheduling component. the other components, implementing specific processing functions, are called directly from the web back-end or task execution processes. rofigure 4: translation flows in the relate platform translation in the relate platform is performed around a text translation component. however, due to the integration of both asr and tts components (for romanian and english), it is possible to translate also speech (by using pre-recorded audio files or by using the integrated speech recording functionality), resulting in new audio files (available for download or direct playback). different translation scenarios are depicted in figure 4 . as described above, depending on the use case, the translation pipeline can be invoked on a request basis or for entire corpora. details on the text translation component are given in section 4 and the speech-to-speech translation component is further described in section 5. ## text translation the "cef automated translation toolkit for the rotating presidency of the council of the eu" action aimed to make the european commission's etranslation platform available to users from eu member states by extending the etranslation platform with a set of custom mts tailored for the eu presidency domain. the romanian-english and english-romanian translation systems were improved by developing high-quality custom mt systems for the eu presidency and dsi domains. for the romanian language, the research institute for artificial intelligence "mihai drȃgȃnescu" contributed to developing the translation system (ro-en and en-ro), a component of a wider system for the presidency of the council of the eu. the current mt platform foot_7 allows users to translate entire documents and local websites, including secure automated translation systems for all eu official languages. using the tilde machine translation api foot_8 , the textual translation component for ro-en and en-ro was integrated into the relate platform so that users can translate documents directly in the platform and also analyse the resulting document using the platform's functionalities. in addition to the full-text translation, a version of the romanian wordnet (tufis , and barbu mititelu, 2015) aligned with the english wordnet @xcite is available for querying. in this case, the user can look up a romanian word and see the equivalent synset from the english word-net. figure 5 shows an example query for the word "european" (in english is written similarly, except with a capital letter "european"). ## speech to speech translation automatic s2st plays a core role in allowing people to communicate more naturally using spoken utterances when they do not share a common language, and nowadays, two methods are usually em4 ) text-to-speech (tts). the available models of each component are depicted in the upper part for romanian and in the lower part for english @xcite . due to the limited amount of romanian resources that are available for directly training an e2e model on this task, it was created a cascaded s2st service for both romanian to english foot_9 and english to romanian foot_10 speech translation that contains four components in their respective pipeline @xcite , as depicted in figure 6 . each component incorporates at least one configurable model for the romanian and english languages, enabling simple integration of new models into the system and improved flexibility in choosing a specific configuration given a potential requirement. the first component of the cascaded s2st system is the asr to transcribe the audio input. the romanian version has two models: robin asr and robin asr dev. the former used the deep-speech2 architecture @xcite and was trained on approximately 230 hours of public speech data, obtaining a 9.91% word-error-rate (wer) on a customized dataset that was created by randomly extracting 5,000 samples from the training set @xcite . the robin asr dev model is a specialized speech recognition system developed to better recognize utterances from the technical domain, specific to the robin project 13 . robin was a user-centred project designing software systems and services to use robots in an interconnected digital society. it also included a component for human-machine dialogue in specific microworld scenarios @xcite . the robin asr dev model @xcite was trained only on the rtasc corpus and, in order to leverage the benefits of transfer learning on small datasets, we started from a wav2vec2 @xcite model that was pre-trained on the whole unlabeled audio data from voxpopuli foot_12 @xcite that is publicly available on huggingface foot_13 . robin asr dev achieved 13.93% wer on the rtasc test set. the english version of the asr component also contains two models: mozilla deepspeech, which is based on the deepspeech @xcite architecture and that contains the latest speech-totext system offered by mozilla foot_14 , and en deep-speech2 which, as robin asr, is based on the deepspeech2 model and was trained only on lib-rispeech @xcite . both models were evaluated on the clean test set of librispeech and obtained 7.06% wer and 9.19% wer, respectively. this difference in performance comes from the training set used for each model, mozilla deepspeech being trained on more data than en deepspeech2 which is not available for public usage. the relate platform offers, at the time of writing this paper, a single model for textual correction on romanian -robin correction that applies two postprocessing algorithms to the incoming transcriptions. firstly, the component capitalizes the first character of the words that are found in a list of known named entities and then, in the second part, it replaces the unknown words from the transcription with known words from a vocabulary. this component is optional and can be removed from the cascaded system if needed (e.g. when working with uncased text or with an open vocabulary). a new neural punctuation restoration component for the romanian language is still under active development and will become available in the future @xcite . a prototype is avail-able for testing foot_15 , but is not yet fully integrated and is not available in the speech translation pipeline. the romanian to english and english to romanian machine translation components use the api introduced in section 4. the english language comes with one model for the tts component in our cascaded system -mozilla en tts, a pretrained tacotron2 with dynamic convolution attention @xcite , that was developed by mozilla foot_16 using the ljspeech dataset 19 . the model obtained a median opinion score (moc) of 4.31±0.06 using a 95% confidence interval. for the romanian language, we offer two models that are not based on deep neural networks but use the classical hidden markov models (hmm) to generate the audio output for a given sentence: romanian tts @xcite and racai ssla @xcite . the difference between these two versions is that the former model outputs a synthesis of higher quality than the former model but at a slower rate, thus allowing a user to trade off computational speed for a better synthesis, or vice-versa, depending on the requirements. the cef telecom project multilingual resources for cef.at in the legal domain (marcell) foot_18 had as the primary goal the enhancement of the etranslation system developed by the european commission. within this project, seven legislative corpora have been created that contain the total body of national legislative documents in effect for seven countries included in the consortium: bulgaria, croatia, hungary, poland, romania, slovakia and slovenia. all the corpora were tokenized, lemmatized and morphologically annotated, dependency parsed, named entities were also added, nominal phrases were identified together with iate foot_19 terms and eurovoc foot_20 descriptors. interactive terminology for europe (iate) has been the eu's eurovoc is a multilingual thesaurus developed and maintained by the publications office of the european union. the main purpose for which it was built is to help process the information contained in documents issued by the eu institutions. the current version, eurovoc 4.4, was released in 2012 and includes 6,883 unique ids for thesaurus concepts, organized in 21 top-level domains, which are further refined in 127 micro-tesauri. it serves as the basis for the main domains of the iate database. all the corpora are in conll-u plus format with fourteen columns in each file, the first ten columns keep the standard conll-u values (id, form, lemma, udpos, xpos, feasts, head, de-prel, deps and misc), while the following four columns (ner, np, iate and eurovoc) are specific to the marcell project. since texts from each corpus came from different sources, metadata harmonization was necessary to create a homogeneous resource in file format. therefore, many fields were established, some mandatory for each language and others optional. the obligatory keys that assure the harmonization of the data are: id -unique identifier of the document, date -date of the document in iso 8601 format, title -the title of the document in the original language, type -the legal type of the document in the original language, entype -the legal type of the document in english. the optional keys are: url -the address of each document, keywords -several keywords in the original language, and topic -the human-readable topic of the document in the original language. in @xcite are presented all the available metadata keys and attributes in source archives for each language. the marcell romanian language corpus contains approximately 144k processed legislative documents that can be classified into five main categories: governmental decisions (25%), ministerial orders (18%), decisions (16%), decrees (16%) and laws (6%). in terms of document length, most of them contain more than 1,000 words per document, and only 6,000 can be considered short documents because they contain less than 100 words per docu-ment. a general overview of the romanian legislative corpus can be seen in ## conclusion in this paper, we described the usability of the re-late platform in the context of machine translation of the romanian language. it provides options for text and speech translation using a modular architecture. additionally, the platform successfully created considerable language resources relevant for machine translation. sections 6.1 and 6.2 described the creation of two large comparable corpora in 7 official eu languages, including the romanian language. the platform is designed to be highly customizable and easily extensible, while the standardized file formats ensure interoperability with other systems. furthermore, the service-oriented architecture (soa), based on rest apis, allows for additional integration options with external applications or other language platforms. the relate platform is available open source on github 25 . its current form resulted from integration of components developed in many research projects. we aim to continue the development, both in terms of translation capabilities and more generally with regard to language technology for romanian language.