How to Create Treebanks without Human Annotators – An Indigenous Language Grammar Checker for Treebank Construction (2025)

Abstract: Creating treebanks for low resource languages is an important task. However, low resource Indigenous language contexts have not only limited resources in terms of text data, but also limited human resources that are available for linguistic annotation. We suggest a work-around by applying a Constraint Grammar operated rule-based dependency parser to do the work of creating a marked-up treebank. However, due to a lot of noise, meaning spelling and grammatical errors in South Sámi written texts, this tool often fails to create complete and correct trees. As a fix to this, we created a grammar checking tool for the most common South Sámi grammatical error types, which improves the quality of the dependency parser significantly. As both literacy and normative standards for most Indigenous languages are much more recent than for majority languages, spelling and grammatical variation and errors are a common source of noise, and the application of a correction tool like ours can be useful in the construction of treebanks for these languages.

## introduction in an extremely low resource language context, treebanks are an important link to developing high level tools that other languages consider standard. machine-learning based language technology can utilise the treebanks for training and testing new models, and rule-based systems can use them as a gold standard to strive for. in addition they can be used for language comparative tasks, evaluation, etc. low resource languages like south sámi, however, are not only low resource in terms of data (< 2 million words) but also lack human resources, which makes manual linguistic annotation of big text corpora impossible. for creating a south sámi treebank, we therefore applied a constraint grammar based dependency annotation tool that can annotate unlimited amounts of text automatically using existing morphological and syntactic tools as their basis. when dealing with low resource indigenous languages we need to keep in mind that language standards are often still in the process of being developed, and language contact with the majority language influences the way people use their language. south sámi texts contain a lot of noise in each sentence in terms of typos and non-standard forms, code-switching and sentence structures that resemble literal translations from the majority language rather than using authentic south sámi syntax. this type of noise is not comparable to the noise in a majority language corpus. it rather reflects the relatively large amount of l2 writers (second language users) in the south sámi text corpus. as we want a treebank that can also be used for teaching purposes, we would like it to represent mostly l1 language. some of these errors and non-standard forms disrupt the sentential dependency structures and prevent our tool from working properly. especially noun phrase internal errors, case errors and agreement errors lead to broken dependency trees. we therefore suggest the usage of a spelling and grammar error correction (gec) tool as part of the pipeline to create a treebank. all our tools are part of a multi-lingual language resource platform (giellalt) which provides a common infrastructure for over 150 languages, most of them lowresource and/or indigenous languages. 1 we manually marked-up error corpora, which we used to identify relevant and frequent errors and created a grammar checking tool that corrects these morphosyntactic structures. the corrected sentences are then fed into the dependency tool, which create our treebank for south sámi. south sámi is an indigenous language with about 500 speakers, and about 10 percent of these writes the language. this work has been made within a language technology group that started as an initiative of the sámi parliament 20 years ago, which is why we combine both native language and engineering competence. our main goal is to develop tools for and together with the language community, especially those that are needed in administration and education. this is self-determination in practice, which is also central principle in sámi endeavors. south sámi is a uralic language with interesting syntactic features, such as copula drop, which leaves many sentences without a finite verb, an interesting matter for dependency parsing. this work is a contribution to creating both proofing tools and a treebank for further research and tool creation. south sámi did not previously have an annotated treebank, thus our contribution in this work is also that of a new treebank. our goal was to create in the most efficient way given limited resources, also making sure that language presented therein is authentic but error free. the treebank follows the written standard that is backed by the south sámi standardization body gïelegaaltije creating a valuable annotated corpus resource. we will in the following present the grammar checking tool, and show how it is integrated into automatic treebank construction of south sámi. ## a treebank for south sámi our visl cg 3 dependency analyzer for south sámi @xcite maps dependencies between word forms that have received a morphological analysis and a syntactical label. each of these rules builds a partial tree, and combined with each other ideally a full tree is created. however, the tool is also able to construct partial trees, which is useful for atypical sentences, ellipses, headlines, in particular sentences without finite verbs. this is also relevant for south sámi as copula-drop is a typical feature of the language. @xcite it also means that the tool can construct partial trees for sentences that contain spelling and grammatical errors or ommitted words. we ran the dependency parsing tool on 481 sentences and 7,266-token sample corpus to see how many complete trees it is able to construct. 188 of 481 sentences produce complete parse trees. one of these complete trees is displayed in figure 4 showing the dependency structure of ex. (1). it includes a finite verb and three coordinated infinitives. the vislcg3 output of the dependency analysis is displayed as graphical trees for the purpose of visualization. the original output can be seen in figure 5 , where dependency structures are expressed by absolute numbers after the hashtag for the position of each word pointing to the number of the word they are dependent on. in the case of the finite verb daarpesjibie its position in the sentence is 2 and it points to the root 0 (#2->0) it creates a full tree despite the orthographical error in jih (should be: jïh) as this the morphological analyzer accounts for some of the typical orthographical errors. the object dam should be analyzed as dependent on the infinitive guarkedh 'understand' instead of daarpesjibie 'need'. the dependency tree for ex. (2-a) is also complete. however, the dependency structure in figure 6 shows several errors. the adjective veaksehke and the demonstrative pronoun gaajhkh should be dependent on the noun gielen instead of the finite verb leah. the reason for the partial errors in the dependency structure is one grammatical error in the ad-"<@xmath0:0.0> @@xmath1:0.0> @fmv #2->0 "<@xmath2:0.0> @cvp #3->4 "<@xmath3:0.0> @fs-imv #4->2 "<,>" "," @xmath4:0.0> #5->3 "<@xmath5:0.0> @imv #6->4 "<@xmath6:0.0> @cnp #7->6 "<@xmath7:0.0> @imv #8->6 "<.>" "." @xmath8:0.0> #9->2 jective form veaksehke (correct: veaksehks) makes it appear a subject in nominative singular instead of an attribute to gielen. gaajhkh can therefore not be identified as adverb dependent on the adjective. the morphological analyzer is robust enough to compensate for several spelling errors as the long 'i' in three words and misspelled aepien (correct: aerpien). they still receive a morphological and syntactical analysis. spelling errors and grammtical non-standard forms are overdimensionally represented in south sámi written texts. for most majority languages, spelling errors and non-standard forms are filtered out by some kind of proofreading. in addition, writers of majority languages have typically undergone a lot of training and their writing has undergone a lot of proofreading in their respective languages school systems. figure 7 gaerjagåetie gaerjagåetie n tjööngkie tjöönghkedh err/orth subj 0 fmv jïh jïh cc cnp vaarjele vaarjelidh v fmv gaajhkide gaajhke pron tjoejide tjoeje n n obj advl , , clb "," guvvieh guvvie n subj jïh jïh cc cnp trygkesovveme trygkesovvedh ex/v subj aamhtesh aamhtese n obj mah mij pron vadta vedtedh v subj fs-n_ sveerje sveerje n subj olkese olkese adv advl . . clb "." gaerjagåetie tjööngkie jïh vaarjele gaajhkide tjoejide , guvvieh jïh trygkesovveme aamhtesh mah sveerje olkese vadta . 123 morphosntactic errors 334 syntactic errors 259 real-word errors 147 lexical errors 216 non-word spelling 3,263 the coordinated demonstrative phrase does not have consequent case agreement, the nominative plural nouns guvvieh and aamhtsesh should be in accusative case just as their coordinated predecessor tjoejide. the parsed tree in figure 7 therefore interprets guvvieh as a new subject to vaarjele and does not make it a daughter of tjoejide as it should be. in addition, the nominative plural relative pronoun mah has a case error. it should be accusative mejtie in order to be identified as the object of the finite verb vadta. ## creating a preprocessing tool for dependency structure in order to create a smoother dependency analysis for south sámi and facilitate treebank building, we decided to preprocess the text by means of a handwritten spelling and grammar checker for the most common error types. we added a grammatical error annotation layer to sikor (sikor, 2025). we chose a 182,759-token part of the corpus that had been marked up for spelling errors already, and classified the grammatical error types on top of those. a demonstrative phrase error as explained in ex. (3-a) is marked as a unit. the error is then classified with its morpho-syntactic properties -in this case the nominative plural noun should be in accusative plural -and then the whole phrase is repeated in its corrected form as below. based on our annotation we decided to write rules for the most frequent error types that would potentially affect the dependency analysis of the sentences. south sámi demonstrative phrase and numeral phrases differ from germanic structures and follow complex rules, which is why errors are common. in demonstrative (and indefinite) phrases typically pronouns and nouns agree in number and case. in numeral phrases, on the other hand, only nominative agrees in number and case. in all other cases, the noun is in singular after all numbers above one. in ex. (4-a), the indefinite pronoun nominative plural gaajhkh 'all' needs to be changed to accusative gaajhkide ' to all' because of the subsequent accusative noun we also need to account for exceptional use of numerals such as in the following sentence (5), where nulle 'zero' is actually used as part of a compound 'zero-object' and not as a quantifier. voestes aejkien manne nulle objeekten bïjre govlim utnim luste goerehtidh maam ij våajnoes aktene raajesisnie. 'the first time i heard about the zero object, i thought it was fun, which wasn't in a sentence.' apart from demonstrative phrase, numeral phrase and nominal phrases involving adjectives, also postpositional phrases can alter the dependency structure in parts of the tree. ex. (6-a) displays a typical case error in dependents of postpositions. in south sámi, the correct form is genitive case. however, a frequent error is to use accusative case as dam 'the' instead of genitive dan 'the'. these errors can also involve coordinated noun phrases such as in ex. (7-a). the following constraint grammar rules in figure 8 add errortags to (multiple) demonstrative/indefinite pronouns noun combinations and relate them to each other (addrelation) to create a unified error that will be visualized as one error. figure 9: copula drop dependency analysis of ex. (11) ## evaluation we chose a 100 sentence test corpus, part of sikor, to manually evaluate the post spell-and grammar checking dependency analysis and got the following results. 73 of 100 sentences received a correct dependency analysis (73%). of 633 dependencies distributed to word forms -excluding punctuation -55 human edits were needed to fix the dependencies. this means that 91.3% of the dependencies are correct. 24 of these edits were necessary because the sentence contains copula drop as shown in the dependency analysis of example (11) in figure 9 . both the non-finite verb barkeminie 'working' of the relative clause and the finite verb of the main clause lea 'is' go to the root of the sentence, where only the latter should do so. copula drop is a known issue in south sámi describe thoroughly in @xcite , and it appears in different forms -the sentence can drop the auxiliary in periphrastic verbal constructions as the one in the previous example, leaving only the non-finite verb form (past participle, gerund etc.). it can also be dropped in copula constructions, leaving only the subject and the predicate. when there are complex sentences with main-and subclause, where the mainclause has copula drop, while the subclause has a finite verb form, the automatic analyzer often analyses the finite verb form of the subclause as the daughter of the root, instead of making it the daughter of the non-finite verbform of the main clause. south sámi syntax poses challenges to machinebased dependency analysis, which languages with required finite verbs do not, and new solutions need to be carefully investigated. other reasons for failing dependencies are remaining spelling and grammar errors (6), and shortcomings in the analysis regarding coordination (7) and finding the correct verbal mother (12). ## conclusion low resource languages like south sámi need language resources and treebanks like all other languages. our approach has taken into account that south sámi lacks human resources to mark up large amounts of texts to create a treebank by applying a rule-based tool to do so. instead, we have used our human resources to create and improve rule-based grammar checking and dependency tools so that we can post-edit our treebank with much less effort than creating it from scratch. we have further identified one of the causes of noise in the creation of such resources -spelling and grammatical errors. we therefore enhanced a marked-up error corpus to systematically identify the most frequent grammatical errors that can get into the way of automatic dependency annotation. these include both, errors on the noun phrase and the verb phrase level -demonstrative phrases, numeral phrases, adjectival forms, case errors in habitive constructions and postpositional phrase being a few of them. based on this analysis we have written rules for all the previous error types to automatically identify and correct these errors and preprocess the input text for the dependency analyzer. we can see that the number of full and partial trees increases with the correction of these grammatical errors, and our current dependency tool gives us 91.3% of correct dependency relations. we were also able to identify the main reasons for remaining flaws in our system. they are related to south sámi being a copula drop language, which makes it more challenging to identify the roots of these sentences, which can either be a non-finite verb or a nominal phrase. this pecularity of south sámi will also be interesting when comparing its treebank with the one of other languages. as a next step, we plan to improve our dependency tool and with some human post-editing create the first south sámi treebank. we have seen that our method is an efficient way of creating a treebank, a dependency tool and a grammar checker at that same time, all of which can be used as language resources and proofing tools by the south sámi language community.