A Diverse Set of Freely Available Linguistic Resources for T urkish (2023)

Abstract: This study presents a diverse set of freely available linguistic resources for Turkish natural language processing, including corpora, pretrained models and education material. Although Turkish is spoken by a sizeable population of over 80 million people, Turkish linguistic resources for natural language processing remain scarce. In this study, we provide corpora to allow practitioners to build their own applications and pretrained models that would assist industry researchers in creating quick prototypes. The provided corpora include named entity recognition datasets of diverse genres, including Wikipedia articles and supplement products customer reviews. In addition, crawling e-commerce and movie reviews websites, we compiled several sentiment analysis datasets of different genres. Our linguistic resources for Turkish also include pretrained spaCy language models. To the best of our knowledge, our models are the first spaCy models trained for the Turkish language. Finally, we provide various types of education material, such as video tutorials and code examples, that can support the interested audience on practicing Turkish NLP. The advantages of our linguistic resources are three-fold: they are freely available, they are first of their kind, and they are easy to use in a broad range of implementations. Along with a thorough description of the resource creation process, we also explain the position of our resources in the Turkish NLP world.

## introduction in recent years, with the development of transformers, natural language processing has experienced a dramatic breakthrough. previously, learning architectures offered state-of-art solutions to many nlp tasks, such as sequence tagging and text classification. accordingly, data-driven approaches have become the dominant technique used to process language data. this has made availability of large and high-quality language data an essential resource for the development of nlp models. turkish is spoken by over 80 million people, both in turkey and across europe, cyprus, and asia foot_0 . however, despite this abundance of turkish speakers, the number of available turkish linguistic resources does not compare to the corresponding amount of resources available for well-studied languages such as english. turkish is an agglutinative language with complex morphology @xcite , and its morphosyntactic characteristics are challenging to handle in nlp applications. similar challenges arise in creating linguistic resources for turkish. in this paper, we present a new set of turkish linguistic resources, including corpora, pretrained spacy models, and education material. the corpora comprise named entity recognition datasets of diverse genres, including wikipedia articles and supplement products customer reviews. we also compiled several sentiment analysis datasets of different genres created via crawling e-commerce and movie reviews websites. the key characteristic of our corpora is their availability, as all corpora are easily accessible via their github repos. with regard to spacy pretrained models, to the best of our knowledge, our models are the first of their kind. each spacy pretrained language model includes a pos tagger, a dependency parser, a lemmatizer, a morphological analyzer, and a named entity recognizer as pipeline components. although some webbased solutions have previously been provided, our pos taggers and dependency parsers are the first ones implemented in pure python and are freely accessible. our resources also include education materials. specifically, we offer relevant information on the corpora building process, including all necessary details on web scraping, text cleaning, file formatting, and training the spacy language models. along with detailed tutorials about using pretrained spacy models in python, we also provide tutorials on turkish linguistics, dataset formats and the general dataset compilation process. all in all, this paper presents a comprehensive collection of resources to the turkish nlp community. ## background in this section, we review available corpora and pretrained models to better contextualize the contributions of our work. ## corpora in this section, we present the corpora part of our set of resources. we start with our named entity and span corpora (section 3.1), followed by sentiment analysis corpora (section 3.2) and a small corpus of covid-19 symptoms (section 3.3). ## pretrained models in this section, we introduce our spacy turkish language models. overall, spacy is an industrialstrength open-source nlp library offering state-ofthe-art performance with an order of magnitude exceeding other available nlp libraries @xcite . spacy also comes with a well-structured api, detailed documentation, support for issues, and an immense user community. moreover, along with being easy to install and deploy, spacy fits well into the python machine learning ecosystem. each spacy language model is a pipeline of pretrained components @xcite . trainable components include a statistical lemmatizer, morphologizer (statistical morphological analyzer), ner, pos tagger, and dependency parser. spacy offers "spacy projects" for end-to-end training, packaging and sharing custom pipelines (honnibal, jul 2020). accordingly, we trained our models with spacy projects; the project template and the configuration files of each component along with the corresponding training hyperparameters are available on our github page. 14 we provide the following 3 pretrained models: tr_core_news_trf, tr_core_news_lg and tr_core_news_md. all these models include a vectorizer, lemmatizer, morphologizer, ner, pos tagger, and dependency parser components. the only difference among these models lies in the vectorization: while tr_core_news_lg and tr_core_news_md include static vectors, tr_core_news_trf is a transformer-based pipeline, meaning that word vectors for tokens are calculated and passed to the downstream components by the underlying transformer @xcite . to train tr_core_news_trf,we used the dbmdz turkish bert model. 15 tr_core_news_lg and tr_core_news_md are packaged with "static" word vectors; here, "static" means that these vectors are not learned parameters of the statistical models, and spacy itself does not feature any algorithms for learning word vector tables @xcite . in order to be included in the model training, static vectors have to be separately trained and packaged. accordingly, we trained floret vectors that support a vast number of subwords in a compact way @xcite . for turkish morphology, representing subwords is a critical issue. for two packages -namely, tr_core_news_lg and tr_core_news_md -we prepared two floret vector packages: one medium-sized and the other large-sized, respectively. training configuration of these two packages can be found in their github repos 16 ; the packaged vectors can be found in their huggingface repo. 17 figure 4 : dependency tree of an example sentence generated by transformer based spacy turkish pipeline. the sentence includes an adjectival clause "sizin yaşadıgınız" that modifies the noun "ilden". like adjectives, most adjectival clauses in turkish precede the noun that they modify. the dependency relation between the clause and the modified noun is marked by acl. the clause has its own nominal subject "sizin", the relation between this token and the clause verb "yaşadıgınız" is marked by nsubj. the clause head "ilden" functions as the oblique object of the main sentence and "kimse" is the nominal subject. all relation types come from universal dependencies @xcite . the visual is created by displacy. all 3 models were trained on the same corpora: universal dependencies turkish boun treebank @xcite was used to train the morphologizer, lemmatizer, pos tagger, and dependency parser components; the ner component was trained using our turkish wiki ner dataset (3.1.1) and panx @xcite . we trained mediumand large-sized floret vectors which are 300dimensional. however, vocabulary sizes were different: while medium-sized vectors include 50k keys, large-sized vectors include 200k keys. the flloret vectors were trained on the mc4 corpus @xcite . all our spacy turkish lan-guage pipelines are available for download in our huggingace repo. 18 ## education material finally, we prepared a number of video and code tutorials to provide relevant information on the dataset collection process and dataset formats, as well as to demonstrate python and bash scripting for cleaning and manipulating text, show possible use cases of the spacy turkish language models, and provide general information about turkish linguistics. our video tutorials, with each tutorial coming as a youtube playlist consisting of several videos, include the following: ## conclusion in this paper, we presented a diverse set of opensource linguistic resources for turkish language processing. our resources include corpora, pretrained spacy language models, and education materials such as code and video tutorials. the importance of our resources for the turkish nlp community is three-fold. the first important aspect of our resources is their accessibility.