U niversal D ependencies for S indhi (2025)

Abstract: Sindhi is an Indo-Aryan language spoken primarily in Pakistan and India by about 40 million people. Despite this extensive use, it is a low resource language for NLP tasks, with few datasets or pretrained embeddings available. In this work, we explore linguistic challenges for annotating Sindhi in the UD paradigm, such as language-specific analysis of adpositions and verb forms. We use this analysis to present a newly annotated dependency treebank for Universal Dependencies, along with pretrained embeddings and an annotation pipeline specifically for Sindhi annotation.

## introduction developing a universal dependencies (ud) treebank for sindhi presents unique challenges due to the language's complicated linguistic features. despite being spoken by approximately 40 million people in pakistan and india, sindhi is a lowresource language compared to other languages of similar population, with only a few tools and datasets available (see section 2). sindhi is a split-ergative indo-aryan language, written right to left. 1 it is closely related to punjabi and saraiki. both languages have ud datasets as works in progress @xcite ), but they are not yet publicly released, and many related languages are also low resource, meaning difficult issues in sindhi annotation have few references from which to work. annotating sindhi for the increasingly widespread universal dependencies framework @xcite is complicated by its complex case system, where case markers themselves undergo further inflection, alongside rich inflectional morphology with nominal and verbal elements. additional complexities arise from pronominal suffixation with nouns, postpositions, adverbs, and verbs, and a partially free word order that introduces syntactic ambiguity. furthermore, the language employs intensifiers across various word classes. the multifunctional use of pronouns as determiners also poses challenges for establishing consistent dependency relations. these characteristics require a tailored approach to create an accurate and robust sindhi ud treebank, with some of the more difficult topics explained in section 4. to annotate plain text, current annotation pipelines almost universally use neural methods with static or contextual embeddings. as a lowresource language, sindhi is most commonly represented in such embeddings as part of a multilingual collection, such as the fasttext project @xcite , or as part of a collection of related languages @xcite . however, embeddings specifically trained for the language in use typically produce better results, a result this work demonstrates applies to sindhi. in order to train new embeddings, we collect data from a variety of text sources, described in section 5. we use the annotated dependency treebank and newly trained word embeddings to train an annotation pipeline using the stanza package @xcite . to facilitate annotation, we build pipelines from partially annotated data and use these labels as silver annotations for the annotators to use as a baseline. the main contributions of this work are: ## related work there are existing datasets for building components of sindhi annotation pipelines, although none of them are complete solutions. a partially complete analysis of 665 sentences for sindhi, named mazhar dootio (md) after its primary author, is available at universaldependencies.org @xcite . this dataset is small and lacks dependency trees. nevertheless, it provides a useful baseline, especially for features and lemmatization. in this work, we reanalyze these sentences, adding dependencies and updating the tagging and featurization standards. @xcite presents an ner dataset for sindhi, with 26,000 tagged sentences, available on github. 2 we use it here to build an ner model for the annotation pipeline. the same group also produced a pos dataset of 293k words @xcite . as static and contextual word embeddings are essential for training neural models, there have been multiple works which provide embeddings. for static embeddings, fasttext included sindhi vectors trained on common crawl and wikipedia @xcite . later, uestc crawled a larger corpus of 61m words to produce sindhispecific word vectors @xcite . for contextual embeddings, xlm-r @xcite uses 40m tokens of sindhi text as part of its training collection. muril-large @xcite uses the same amount of sindhi text, but perhaps because it is focused on indic and dravidian languages, tends to perform better on downstream tasks. prior work aside from the mazhar-dootio dataset also addresses lemmatization, including @xcite and @xcite . we use a different lemmatization standard in this work, though; see section 3.2. this work uses stanza @xcite to build and evaluate models. similar projects which could build a sindhi annotation pipeline using the data include udpipe @xcite and spacy @xcite . ## annotation process the ud dataset described here was annotated in partnership with mltwist, an nlp annotation company. mltwist recruited two native speakers of sindhi for annotation, listed here as authors. for annotation platforms, we used datasaur for dependencies and upos, followed by kili for features and xpos. individual sentences that needed revision were edited via the free tool conllueditor @xcite , with batch edits executed with semgrex and ssurgeon @xcite . ## linguistically adapting ud for sindhi several constructions in sindhi are challenging to annotate as part of a universal dependencies framework, or are seen only in the context of other low-resource languages. ## embedding pretraining as has been typical in the last decade of nlp work, before building the annotation pipeline, we pretrained static word embeddings, a character model @xcite , and a transformer using a corpus of raw text. the corpus collected is 372m words, significantly less than the original bert, which used 3,300m words for english @xcite , but sufficient for pretraining models for a new language. this is significantly larger than that used in muril and xlm-r, each with less than 40m. source m tokens wikipedia 5 oscar cc 11 (abadji et al., 2022) books corpus 2 (ali et al., 2020a) adabi forums 0.5 sindhi la journals 0.8 sindhi la encyclopedia 5 various newswire 38 sindh salamt forums 24 sindh salamt books 57 sangraha 229 (khan et al., 2024) ‫آﻫﻦ‬ ‫اﻃﻼع‬ ‫ﺟﺎ‬ ‫ﻣﺮڻ‬ ‫ﭔڏي‬ ‫ﭴﮣﺎ‬ 2 ‫ﺳﻮڌو‬ ‫ﻋﺒﺪاﻟﻠﻪ‬ ‫ﻣﺎﺳٽﺮ‬ are reports of dying drown persons 2 including abdullah master root nsubj obl advcl nmod figure 3 : in this non-projective tree, ‫ﻋﺒﺪاﻟﻠﻪ‬ attaches to ‫ﭔڏي‬ despite the intervening subject "two people, including master abdullah, are reported to have drowned." we experimented with glove @xcite , fasttext @xcite , and word2vec @xcite , the latter two using the gensim implementation @xcite . based on these results, we use the glove model, to be distributed with the stanza pipeline for sindhi. other pretrained models will be available along with the default stanza pipeline. stanza @xcite implements a contextual character model. we used huggingface's roberta implementation to finetune muril and to train two candidate sindhi transformers from a random initialization. interestingly, and unfortunately, each of the models produced were less accurate on the three tasks compared with the original muril-large (see ## annotation pipeline having annotated a ud dataset and built some pretrained models, we then built a pipeline using stanza annotation software. for ner, we used the existing siner dataset @xcite . lemma and ner scores are high. surprisingly, feature tagging is presently the lowest, performing below dependency and ner. continued improvements to the annotations and the embeddings should improve scores. task emb score lemma seq2seq w/ charlm 99.08 accuracy upos muril 98.44 f1 features muril 85.56 f1 depparse muril 91.83 las ner muril 88.13 entity f1 ## conclusion this project explores the more challenging structures for analyzing sindhi grammar using universal dependencies. we then analyze roughly 6000 sentences of sindhi, building a dataset for release in the 2.16 and 2.17 releases of ud. in parallel, we collect a larger corpus of sindhi text than previously collected, facilitating more accurate static embeddings for the language. combining each of these tasks with stanza results in the first end-toend annotation pipeline for the sindhi language.