TULAP - An Accessible and Sustainable Platform for T urkish Natural Language Processing Resources (2023)

Abstract: Access to natural language processing resources is essential for their continuous improvement. This can be especially challenging in educational institutions where the software development effort required to package and release research outcomes may be overwhelming and under-recognized. Access towell-prepared and reliable research outcomes is important both for their developers as well as the greater research community. This paper presents an approach to address this concern with two main goals: (1) to create an open-source easily deployable platform where resources can be easily shared and explored, and (2) to use this platform to publish open-source Turkish NLP resources (datasets and tools) created by a research lab. The Turkish Natural Language Processing (TULAP) was designed and developed as an easy-to-use platform to share dataset and tool resources which supports interactive tool demos. Numerous open access Turkish NLP resources have been shared on TULAP. All tools are containerized to support portability for custom use. This paper describes the design, implementation, and deployment of TULAP with use cases (available at https://tulap.cmpe.boun.edu.tr/ ). A short video demonstrating our system is available at https://figshare.com/articles/media/TULAP_Demo/22179047 .

## introduction and motivation there is a growing interest in the field of natural language processing (nlp) due to the recent advances in deep learning-based approaches, such as transformer architectures and pretrained language models. this interest shows itself in the demand both for applications solving nlp tasks and for high-quality corpora that can be used in machine learning models. although there are plenty of such resources for well-studied languages such as english, there is a scarcity of these resources in most of the other languages. in addition to developing such resources, there is a need for building software environments that collect these resources within a platform and offer them to the nlp community with easy-to-use interfaces. this paper introduces tulap (turkish language processing platform) that aims to provide a variety of turkish nlp resources (datasets and tools). tulap provides a user-friendly user interface (ui) where users can easily discover and examine nlp resources. tools can be explored without the hassle of having to build and install them. all tools are containerized apis (application programming interface) to facilitate easy installation and use. the tools served on tulap are provided as open-source to facilitate further research and development. this is particularly important in educational institutions where students tend not to package their code appropriately, leading to to loss of time and effort for those who wish to use their work. the main motivation of tulap is to develop a platform that supports the continuous contribution of nlp resources generated by our research lab, tabilab 1 . for the initial deployment, we gathered previously developed resources, created apis, and containerized them. hereafter, the resource developers will be able to make their contributions with an easy-to-use interface. the aim is to provide an up-to-date nlp platform with resources that are easy to discover and explore as well as to contribute new resources produced by researchers. the main contributions of this work are summarized as follows: ## related work the goal of collecting nlp resources under a unified framework and thus making implementing various types of applications easier has led to the development of nlp platforms in several languages. in this section, we briefly review a few of the widelyused nlp platforms with different functionalities including those specific to the turkish language. nltk @xcite is one of the earliest nlp libraries consisting of tools as a hierarchy of modules. it supports a wide range of tasks as well as corpora that can be used in various nlp tasks. al-lennlp @xcite is a library for deep learning-based nlp research. the users can build their own models using common deep learning operations. stanford corenlp toolkit @xcite provides a pipeline of preprocessing operations and downstream nlp tasks in various languages. it is one of the most widely used nlp platforms partly due to its simple design. users can also integrate new nlp tasks to the pipeline. the hugging face platform (hugging face, inc., 2023) is one of the most popular platforms that is actively used in current nlp research. it includes a wide range of libraries that consist of different types of software such as machine learning models, datasets, demos, and evaluation tools. the transformers library @xcite includes open-source implementations of state-of-the-art transformer-based models. the users can easily share the resources they have developed in the platform via the hugging face hub. among the platforms specific to turkish, i̇tü turkish nlp web service @xcite offers a variety of tools that can be used in turkish nlp studies. the tools in the platform can be used either as a pipeline or as stand-alone components. since the platform has been designed for turkish, it includes tools that are specific to turkish such as diacritic restorer. a recently-developed platform is mukayese @xcite , which is a benchmarking platform that provides a set of datasets and benchmarks for seven different types of turkish nlp tasks. the mukayese platform is also a part of the turkish data depository (tdd) project foot_4 for building a repository of turkish nlp resources. the platform that we introduce in this work, tu-lap, bears commonalities with these earlier works as well as different functionalities. unlike most other platforms, we offer both nlp datasets and software tools. the tools can be used as ready-touse stand-alone applications, while the datasets can be utilized for training machine learning models or for other purposes. with respect to the tool set, tulap is similar to the stanford corenlp framework in the sense that both preprocessing and downstream operations are supported, and the platform is extendable with new tools. there are several differences of tulap from the currently available turkish platforms. while these platforms mostly provide preprocessing operations, tulap also supports a wide range of more complex tasks like text summarization and question answering. furthermore, all the resources are made accessible with open-source or creative commons licenses and all tools are containerized for portability purposes. finally, the platform is extensible with support for end-user contribution of new tools and datasets. ## tulap deployment at the time of the writing of this article, tulap includes 12 datasets and 16 tools which are shown in ## using tulap this section describes how tulap can be used through use cases. the landing page of tulap nlp tools tokenizer @xcite sentence splitter @xcite deasciifier @xcite lemmatizer @xcite morphological analyzer @xcite morphological disambiguator @xcite @xcite verbal multiword expression identifier @xcite named entity recognizer @xcite question answerer @xcite relation extractor @xcite sentiment analyzer. i) binary (aydın and @xcite ; ii) ternary @xcite text summarizer @xcite grammar annotation tool. i) standalone @xcite ; ii) web ( akkurt ## discussion and conclusions this work's primary focus was to create an accessible and sustainable platform for turkish nlp resources produced at bogaziçi university. for this, we developed a research output platform (tabi-rop) where open access to datasets and tools that result from research activities related to nlp or other fields can be shared. tulap was developed using tabi-rop and hosts numerous resources produced by our research lab. one of the most challenging tasks was the collection and packaging of previously developed resources. this fact validated the need for such a platform -a sentiment also expressed by the research lab alumni when reviewing tulap. we continuously monitor tulap as well as maintain tabi-rop foot_9 . currently, we are upgrading the repository component to benefit from dspace v7 @xcite improvements. moving forward, we plan to include preprocessing tasks, new tasks, and improved versions of present tools. we are pleased that new resources have been contributed subsequently to the initial release. other future work includes platform maintenance and improvements to the presentation of and interaction with resources. while many platforms have been emerging to @xmath0 docker build -t ner . // docker console feedback omitted @xmath1 curl -x post http :// localhost :8080/ ner / predict / -h ' content -type : application / json ' -d ' {" text ":" i̇ stanbul barosu ' ndaki yapay zek â , robotlar ve hukuk konferans ı ' nda pirimiz alan turing 'i anmadan olmazd ı. "} ' {' tagger_output ': { '0 ': ['i̇ stanbul ', 'b -org '] , '1 ': [" barosu ' ndaki ", 'i -org '], '2 ': [ ' yapay ', 'o '], '3 ': [ ' zek â ' , 'o ' ] ,...} listing 1: the commands to acquire, build, run the ner tool and an api request. the output of the docker build and api response are truncated due to space limitation. we approached the development of this platform as a software project starting from requirements elicitation, design, implementation, deployment, and maintenance. the project was managed with weekly meetings, version management, and issuetracking tools. we believe that this approach was instrumental to achieving our goals. we note that the research output platform (tabi-rop) which we designed and developed for tulap supports sharing computational research outputs in general and stands as a valuable contribution in its own right. it is not restricted to the deployment of tulap and it can serve as an underlying platform in other domains and languages. our hope is that tulap facilitates research and development efforts in turkish nlp with information, demos, open-source resources, and easily accessible/usable reference versions of data and tools that we have provided.