Microservices at Your Service: Bridging the Gap between NLP Research and Industry (2023)

Abstract: This paper describes a collaborative European project whose aim was to gather open source Natural Language Processing (NLP) tools and make them accessible as running services and easy to try out in the European Language Grid (ELG). The motivation of the project was to increase accessibility for more European languages and make it easier for developers to use the underlying tools in their own applications. The project resulted in the containerization of 60 existing NLP tools for 16 languages, all of which are now currently running as easily testable services in the ELG platform.

## introduction universities and other research institutes in europe, and sometimes companies, are nowadays often publishing open source natural language processing (nlp) software on various platforms, primarily github. this software is often associated with research papers and, in the best case, also linked to other sharing platforms, such as clarin 1 or meta-share 2 . github is, however, often the only place in which the tools are available. if a user finds a tool with a suitable license, it may still be difficult to determine if the tool works as intended. the threshold for trying out these nlp tools can also be high due to the reliance on various dependencies that may not be 1 http://clarin.eu/ 2 http://www.meta-share.org/ compatible with other desired tools or the tools are simply not up to date. reproducibility of results is important in nlp but currently many results cannot be reproduced, even if the code is available. for example, @xcite were only able to reproduce the same results in 1 out of 10 experiments. in this paper, we describe a collaborative european project, microservices at your service: bridging the gap between nlp research and industry foot_0 (hereafter simply referred to as the microservices project), carried out by four partners: lingsoft, a private company from finland, university of tartu from estonia, reykjavik university from iceland, and gradiant, a non-profit organisation from spain. the main aim of the project was to increase accessibility of nlp tools for more european languages by: the project has deployed services in the elg for 16 languages (see section 4). for many languages, there is a distinct lack of resources in the current academic nlp research (maria @xcite . highlighting the efforts made for lowresource languages is paramount to foster the development and usage of these resources by both the academic community and the industry, and our project targeted several of these low-resource languages. each underlying open source tool was implemented as a microservice (see section 2.1) using docker (see section 3.3) for containerization. this allows developers, who need functionality from the various tools, to design their nlp applications as a collection of loosely coupled running services, as opposed to building the application using sources from various github repositories, which, notably, may be written using various programming languages and depend on various external libraries. in total, our project has resulted in the containerization of 60 existing nlp tools, all of which are currently running as services accessible through the elg. ## background nowadays, software is often distributed to the end users via the internet, rather than having the users install the software on their local machines. this method of distribution is called software-asa-service or saas. many large commercial organisations offer cloud platforms for distributing software, e.g. ai and nlp as saas, to the end users, and on some platforms it is possible for other organisations than the platform provider to upload their own tools for further distribution. in this section, we provide the reader with basic information on the concept of microservices, the elg cloud platform, and elrc-share. ## project execution our two year project started in march 2021. the goal of the project (described in section 1) included several stages. in the first stage, we sought out open source tools that might be of potential interest. we prioritized those that are actively maintained or developed. this was carried out both by bottom-up search on the software sharing platforms (primarily github), and by contacting research institutions in the targeted regions. in parallel, we also collected standard or available test data sets for the tools. this initial phase was followed by testing the set of collected tools on the existing test data. if many tools existed for the same task, a selection was made based on metrics performance and language coverage. after all tools were tested and selected, we started containerizing the tools and expose a web service api for each of them on the elg. finally, we stored metadata information of each tool in elrc-share. our dissemination activities ran parallel to making the tools available: we held workshops on different themes of the project, ranging from dockerization of the tools to demonstrating their functionality and use case integration. ## getting the tools into use there is a risk that new tools made for lowresource languages might not be known by the community. a tool might be created as a one time release for an academic publication, or it might not have gathered the attention needed for a continued development. for the purpose of both stimulating researchers to share their tools and promote the tools we made available, we held three types of workshops: first we had an early awareness workshop, in which we provided hands-on guidance on how to release available tools as docker images. during the second year, we held two workshops focusing on how to make tools available in the elg platform. finally, at the end of the project, we held workshops which summarized our work and demonstrated how the tools we provided can be integrated into lt applications. all workshops are made available on the project webpage. in what follows, we describe some of these pilot integration cases. in each of these cases, it was easy to "plug in" a container with a well defined api, and then handle the input and output in the process pipeline. a language identification (lid) tool was utilized in two different cases. in a translation process, we utilized it to make sure the training data for a neural machine translation (nmt) model was actually in the correct language. the original texts contained sentences in other languages, causing an in-production nmt model to occasionally produce english instead of swedish translations. the previous lid tool had a 98.9 % precision and 96.4% recall for swedish, whereas the new tool, heli ots @xcite , had a 99.9 % precision and 99.6% recall. when there are hundreds of millions of words in the training material, one percentage point yields millions of words tagged in wrong language. the new lid tool alleviated this problem to a sufficient extent. this lid tool was also found useful in an online library platform foot_8 , where publishers provide large amounts of e-books. sometimes the metadata provided by the publisher does not match the language of the actual e-book, yielding erroneous behavior, for example, in screen readers. at the online library platform, we also piloted aligning audio books and e-books, to allow seamless switching from text to audio and back, using an audio alignment tool. this tool was not designed for this kind of task originally, but, nevertheless, it allowed testing potential new features for the platform. furthermore, we also tested ner and linking to ontologies to further improve the findability within an e-book or audio book. ## limitations in the previous sections, we have argued that it can be beneficial to dockerize nlp tools for the purpose of making them accessible as running microservices. however, this approach can have some practical limitations. first, changes to a tool do not automatically become available in the dockerized version. thus, the running microservice in the elg might become outdated. however, if the developer of the underlying tool is keen on making the newest version running as a microservice, the developer can easily build the docker image again (the code for building it is open source) and then ask elg to pull the new image from the associated docker hub. most of that process can also be automated. second, due to resource constraints, elg services are not guaranteed to be constantly running. if a user calls the api of a service, which is not running, the user will probably experience considerable initial delay (associated with the first api call) before the requested service has started. with regard to both of the above mentioned limitations, it is worth noting that anyone can use a given docker image to expose an api for the underlying tool on some web server. in other words, elg is not the only option for providing access to a running service. ## conclusion in this paper, we have described a collaborative project which succeeded in making 60 nlp tools covering a total of 16 languages available as microservices in the elg platform. we also described the microservice principles and the european platforms that record or host these microservices, and the steps to get the tools into these platforms. we recommend that researchers continue this work by providing their tools as docker images and as compatible services in the elg platform. this requires just a little more effort from the researchers, but substantially lowers the threshold for testing the tool for new researchers/developers. hence, lowering the threshold for integrating the tool in new services and raising the potential impact of the initial research.