A New E nglish- D utch- NGT Corpus for the Hospitality Domain (2023)

Abstract: One of the major challenges hampering the development of language technology which targets sign languages is the extremely limited availability of good quality data geared towards machine learning and deep learning approaches. In this paper we introduce the NGT-Dutch Hotel Review Corpus (NGT-HoReCo), which addresses this issue by providing multimodal parallel data in English, Dutch and Sign Language of the Netherlands (NGT). The corpus contains 283 hotel reviews in written English, translated into written Dutch and into NGT videos. It will be made publicly available through CLARIN and through the ELG platform.

## introduction as stated in rivera @xcite , "the emergence of new technological approaches such as deep-learning neural networks, based on increased computational power and access to sizeable amounts of data, are making human language technologies (hlt) a real solution to overcoming language barriers." nevertheless, these very promising advances mainly concern hlt which focuses on spoken languages only, while hlt which targets sign languages is severely limited and strongly lagging behind @xcite . this discrepancy between what has been achieved for spoken languages and what is available for signed languages is due to a number of challenges which are limiting the development of lt for signed languages (e.g. the lack of standardised data format, the lack of a standardised writing and annotation systems, etc.). for more details, see @xcite . the biggest bottleneck limiting the performance of new technological approaches for sign languages is the quantity of high quality data. to give an example, on average the data available for a relatively well resourced sign language is roughly ten times smaller than data available for a so-called low resource spoken language @xcite . besides the quantitative bottleneck, there is also an issue with data quality. besides data scarcity, most of the parallel datasets which are available consist of spoken language news broadcasts interpreted into a sign language (in most cases by a hearing interpreter) @xcite . this affects the authenticity and the quality of the sign language data, since the interpreting process interferes with its accuracy (interpretation takes place simultaneously, which means that the interpreter needs to be quick and sometimes has to sacrifice accuracy for efficiency), and most hearing interpreters are not l1 users of the sign language (an exception being interpreters who are coda -children of deaf adults -and other specific cases). the goal of the compilation of the ngt-dutch hotel review corpus (ngt-horeco) described in this paper is to contribute to reducing the scarcity of good quality sign language data by providing a multimodal parallel corpus of written english reviews and their translations into written dutch and into ngt videos. the quality is ensured with respect to the authenticity of the ngt by the fact that translations were performed by deaf pro-fessional translators. the accuracy of the translations is ensured by the fact that it concerns actual translations, performed in an offline modus without the constraints which are custom in an interpreting context. the availability of a corpus such as ngt-horeco targets the stimulation of advancements in the field of sign language technology through both high-quality data for training models as well as a gold standard data for evaluation. ngt, together with german sign language (dgs), represent the richest sign languages in europe in terms of available resources. nevertheless, data available even for relatively well-represented sign languages are far from being sufficient for the development of language technologies. the main source of data for ngt is the corpus ngt @xcite , which is available for download at the language archive ( https://archive.mpi.nl/tla/ ), in the form of separate files, and as a single file through the clarin infrastructure ( http:// hdl.handle.net/10032/tm-a2-u5 ). it contains 72 hours of dialogues between native users of ngt. 104 signers took part to the recordings. one limitation of the corpus is that only 25% of the data have been annotated @xcite ; this is due to the fact that to date annotation is a manual and very-time consuming task @xcite . as a consequence, only part of the corpus ngt can be employed for mt tasks. a different type of resource is constituted by lexicons. the lexicon of the corpus ngt (crasborn et al, 2020a) was made available by global signbank and is downloadable per sign. it consists of 3.645 short video files. another available ngt lexicon downloadable per sign is https: //www.lerengebaren.nl/ , which consists of 2.993 videos. ## methodology: preparation of the corpus the creation of ngt-horeco required preparation of data for both dutch and ngt. after gathering the publicly available english texts, these were translated into written dutch; subsequently, the dutch texts were translated offline into ngt videos by professional deaf translators. ## results: ngt-horeco the corpus comprises 297 hotel reviews (roughly 1.680 sentences) in written english, their translation into written dutch and into ngt videos. the limited domain of the data, namely, hospitality, allows to have recurrent topics and signs in different possible combinations and to account, to a certain extent, for inter and intra signer variation. the total amounts of words contained in the corpus is 21.464 for english and 22.274 for the dutch text. the word length of the written reviews varies from around 15 to 400 words. the distribution of lengths in the post-edited translation is presented in figure 1 , where the x-axis is the length of the post-edited text, in bins of 10 words. the y-axis is the ratio of files with a certain length. the ngt translations consist of almost 4 hours of videos (230,54 minutes). the duration of the ngt videos ranges from around 10 seconds to around 4 minutes. the distribution of lengths of the videos is presented in figure 2 , where the yaxis is the ratio of files and the x-axis is the duration in seconds, in bins of 10 seconds. the corpus is publicly available through the elg platform at https://live . european-language-grid.eu/ catalogue/corpus/21566, and is also made available through the clarin platform. the permanent identifier for corpus download is http: //hdl.handle.net/10032/tm-a2-w2 . ngt-horeco is available under a cc by-nc license, however, the written english text does not have availability restrictions, being fully publicly available in a kaggle dataset. ## conclusion and future steps in this paper we introduced a new available multimodal parallel corpus of written english, written dutch and ngt videos. the corpus contains 297 hotel reviews in written english which were translated into written dutch and into ngt videos. the dutch-ngt translations were performed by deaf professional translators. parallel data such as ngt-horeco support further developments of sign language technology, including but not limited to sign language machine translation. a current limitation of the corpus is that there is no alignment between written sentences and video fragments. to date, there are no tools to automatically generate such alignment; consequently, a fur-ther implementation of the corpus would include manual alignment. in addition, the size of the corpus is still quite limited, due to time and cost restrictions of the ngt-horeco project. nevertheless, the advantage of the availability of parallel data such as ngt-horeco is that similar parallel corpora have the potential to be implemented with additional features and languages. for instance, having the same reviews translated by more ngt translators coming from different parts of the netherlands would account for language variation. we have considered this option but decided to first focus on having as many reviews translated as possible. nevertheless, this would be a valuable direction for an implementation of the corpus. currently we have initiated a further development of ngt-horeco to also include flemish sign language (vgt). adding vgt is of particular interest because ngt and vgt, despite not being closely related languages, both base their mouthing on dutch and are generally used in countries where dutch is (one of) the official language(s). additionally, ngt-horeco is going to be enriched with different types of annotations, such as pose estimates, etc.