T urkish D elight NLP : A Neural T urkish NLP Toolkit (2022)

Abstract: We introduce a neural Turkish NLP toolkit called TurkishDelightNLP that performs computational linguistic analyses from morphological level to semantic level that involves tasks such as stemming, morphological segmentation, morphological tagging, part-of-speech tagging, dependency parsing, and semantic parsing, as well as high-level NLP tasks such as named entity recognition. We publicly share the open-source Turkish NLP toolkit through a web interface that allows an input text to be analysed in real-time, as well as the open source implementation of the components provided in the toolkit, an API, and several annotated datasets such as word similarity test set to evaluate word embeddings and UCCA-based semantic annotation in Turkish. This will be the first open-source Turkish NLP toolkit that involves a range of NLP tasks in all levels. We believe that it will be useful for other researchers in Turkish NLP and will be also beneficial for other high-level NLP tasks in Turkish.

## introduction turkish is one of the low-resource languages with a rich morphology. although still limited, there has been an increasing interest in turkish nlp in the last decade. being a morphologically productive language is the main drawback of the turkish nlp research. current deep learning models are notoriously data-hungry. when it comes to morphologically productive languages, the data requirement substantially increases compared to other morphologically poor languages. this is due to the number of different word forms that can be generated via inflection and/or derivation. although current word embedding models such as bert @xcite rely on tokenization that considers sub-word tokens rather than word tokens, the recent research @xcite still shows that the performance of such models degrades with novel words. we introduce a new neural turkish nlp toolkit that involves the following linguistic and nlp tasks in turkish: stemming, morphological segmentation, morphological tagging, part-of-speech tagging, dependency parsing, semantic parsing, and named entity recognition. morphological segmentation, morphological tagging, part-of-speech tagging, and dependency parsing are learned jointly using a multi-task learning approach. most of the previous work on turkish morphology and syntax considers morphological and syntactic tasks as independent problems. however, syntax is strongly defined by morphology and vice versa, especially in agglutinative languages. therefore, in this study, we benefit from the mutual interaction between morphological and syntactic layers in the language. all components apart from semantic parsing model are built on lstms that are capable of learning especially long distance relations. the models also utilise a bahdanau @xcite attention mechanism in various layers for an efficient learning of the valuable contextual information within the sentence/word. moreover, we investigate cross-level information flow between the layers by incorporating information between words in different time steps. as distinct from the other components, semantic parsing model adopts an encoder-decoder model where the encoder is based on self-attention mechanism @xcite . turkishdelightnlp is available at http: //rgcl.wlv.ac.uk/turkishnlp/ and the source codes of all components are also publicly available, which are specified in each section below. we also provide api to allow users to process their data using http requests 1 . in addition to the nlp toolkit, we provide few datasets; that are an ucca-based semantic annotation for turkish, a turkish stemming training set based on metu-sabanci turkish treebank @xcite , and a word similarity test set along with the human judgements for assessing morphologically rich turkish word embeddings. ## related work and tools on turkish nlp despite being a low-resource language, turkish has been one of the actively studied languages among other low-resource languages especially in the last decade. numerous models have been recently released for turkish. however, most of them were not released publicly available, and they were not shared as tools that facilitate generating a result in real time. the earlier studies on turkish morphology include morphological analyzers such as the two-level description of turkish morphology @xcite , the stochastic morphological analyzer based on finite state transducers @xcite , paradigmatic approaches @xcite @xcite , and few other open source analyzers such as @xcite , trmorph @xcite , and the syntactically expressive morphological analyzer by @xcite . the earlier studies also involve dependency parsers such as the probabilistic and deterministic dependency parser by @xcite , the two-phase statistical parser based on conditional random fields (crfs) by durgar @xcite , and the recent neural parser by @xcite . there are a couple of turkish stemmers introduced such as the probabilistic stemmer by @xcite , and the finite state machine-based govde-turk by yücebas and tintin (2017); a few partof-speech taggers were also proposed such as the hidden markov model-based pos tagger by @xcite , the deterministic tagger using the two-level morphological description by oflazer and @xcite , and unsupervised bayesian approaches @xcite . the first semantic parsing annotation for turkish @xcite has been presented for abstract meaning representation (amr) @xcite and there is not any other semantic parser introduced for turkish yet, to our knowledge. as seen, most of the linguistic analysis tasks on turkish are based on either statistical or deterministic approaches. currently, the turkish nlp research focuses more on nlp applications such as named entity recognition @xcite @xcite ), text classification @xcite , sentiment analysis @xcite , offensive language identification (ozdemir and yeniterzi, 2020), text summarisation @xcite ), text normalisation @xcite with especially the availability of the large pretrained neural word embeddings in almost any language. most of the nlp tools in turkish were released before the deep learning era and they still have not been replaced by the neural network approaches and the researchers in the field still use the oldfashioned statistical and deterministic models for morphological or syntactic processing. we aim to fill this gap with our turkish nlp toolkit by introducing better performing neural-based methods for turkish linguistic analysis and nlp. the most similar one to our toolkit is itu nlp toolkit @xcite that also involves a wide range of nlp tools such as normalization, spell correction, morphological analysis, dependency parsing, and named entity recognition. however, all of their models are independent from each other and they are built on either deterministic or statistical machine learning algorithms. our toolkit deviates from theirs by adopting neural models and analysing morphology and syntax jointly by considering the interaction between them. moreover, their toolkit does not involve any semantic parsing as ours. ## about turkish turkish is an agglutinating language with a rich morphology. the morphological rules are quite regular in turkish that define the order of the morphemes in a word, as well as the morphophonemic processes such as consonant mutation and vowel harmony, which lead the suffix and the final consonant and vowel in a word to be harmonised with each other mutually. therefore, a morpheme can have tens of different surface forms in turkish, which are allomorphs of the same morpheme. in turkish, syntactic information is encoded in inflectional morphemes. for example, the word ' yapabilecegim' (in english, ' i will be able to do') involves the following inflectional morphemes that each correspond to a syntactic role: ' -abil' ('be able'), ' -eceg' ( ' will'), and ' -im' ( ' i'). in this paper, we propose to process every word considering its left and right context through a cross-level information from morphological segments up to dependencies in a moving window, so that morphological information of the contextual words help to analyse the pos tags, and the pos information of the contextual words help to analyse the dependency relations in a sentence. ## a neural turkish nlp toolkit the introduced toolkit involves different components that are all described thoroughly below. ## web interface turkishdelightnlp is a streamlit application that provides a simple user interface for producing predictions for different tasks. we selected streamlit since it is a low-code web framework that enables researchers to easily create a data-driven app. streamlit has a relatively simple application programming interface and it is specifically designed for data science applications. in turkishde-lightnlp, in the backend, query and model are cached to avoid repeated calculations of the same input. docker is used to increase portability and to be deployed in different operating systems and hardware platforms. figure 4 shows the user interface. in the left panel, there is a menu for the models. whenever the user selects a model and populates a sentence, the result is displayed on the right panel. we also provide a rest api that allows users to access the toolkit with http requests. to be able to use the api, we provide an api token, so a user can access it from clients such as curl and postman. moreover, with the help of swagger and redoc documentation, users can see how to consume api endpoints. ## conclusion and future work we introduce a new neural turkish nlp toolkit that performs different levels of linguistic analysis from morphology to semantics, as well as other nlp applications such as ner. all source codes and relevant datasets are publicly available and we believe that this framework for turkish nlp will be beneficial for other researchers in the area, and will eventually expedite the turkish nlp research.