Supporting Language Users - Releasing a Full-fledged L ule S ámi Grammar Checker (2023)

Abstract: We present the first rule-based L1 grammar checker for Lule Sámi. Releasing a Lule Sámi grammar checker has direct consequences for language revitalization. Our primary intention is therefore to support language users in their writing and their confidence to use the language. We release a version of the tool for MS Word and GoogleDocs that corrects six grammatical error types. For the benefit of the user, the selection of error types is based on frequency of the errors and the quality of our tool. Our most successful error correction, for a phonetically and syntactically motivated copula error, reaches a precision of 96%.

## introduction we release a new l1 grammar correction tool for lule sámi that can be integrated in ms word and googledocs. gramdivvun is the first grammar checker for lule sámi and has been released may 31st 2023. 1 the underlying purpose is to provide a tool that can give language users the security that their language is right in the absence of a strict norm -a paradox we face in our daily work. speakers and writers of a language are confident and carefree when they feel secure in their language use. 2 however, minority languages often face loss of language arenas and at the same time have less resources for language teaching than majority languages. the consequence is that (new) language users get insecure in their use of language and are often left to criticism by the language experts when speaking or writing. this can lead to frustration and resistance to use the language among the ones that are not considered language experts. the notion of the language barrier -where older generations take the role of the 'language police' -has also been reported in other indigenous language contexts, for example when learning the cree language as an adult. @xcite as one of the authors of this work is herself a member of the lule sámi speech community she is familiar with general attitudes, one of which is that the ones that know the language have a clear feeling of how the language should be, even if there is not a written norm. this creates a gap between these experts and the language learners. at the same time there are few contexts/opportunities to improve one's grammar skills and avoid being criticised so that speaking can lead to anxiety and speakers can feel discouraged to use the language. especially in writing, lule sámi text production differs from its coexisting majority language text production. even official texts and texts written by highly proficient users contain a lot of spelling and grammar errors @xcite . this is due to lower written language proficiency in minority languages, and also a lack of written norms. a norm and someone enforcing this norm is necessary to teach language competence to the younger generation and pass on expert language knowledge in all its richness. in the absence of sufficient l1 teachers, now many l2 speakers are becoming teachers that need support to teach the language in all its details. there are no books that explain grammatical phenomena in all their details, including contrasting examples and frequent mistakes. existing grammar books only have text book examples and focus on morphology, rather than syntax. where sufficient human feedback on our language production is missing, we need a tool that can evaluate the correctness our language on the fly. our language technology tools already have a wide user base including official domains such as the sámi parliament, sámi media and schools that use our proofreading tools. the grammar checker will be included in the automatic updates of future versions of the spellchecker to provide better tools for the users. divvun has been established to provide language technology tools for the sámi language community, and has an ownership agreement with the sámi parliament, which unequivocally states that what divvun develops belongs to the sámi people through the sámi parliament. the construction of a lule sámi grammar checker started in october 2020 with a general error categorization and smaller experiments with rules. in 2022 we did intensive work to collect regression tests and reported first results @xcite . the main motivation for making proofing tools are the needs of the language users and the tools' usability. that means that we want to make the tools available at an early stage, even if they do not include all the functionalities yet, and at the same time ensure their quality (i.e. especially good precision). ensuring the quality means that only those error types that give a certain precision are included. the tools are meant to support teachers, proof readers and individuals by finding errors that are hard to detect because of orthographic similarities. they are also meant to help enforcing the (mostly orthographic) language norm proposed by the normative organ giellagálldo foot_2 in a consistent way. ## language situation for lule sámi lule sámi is an indigenous language spoken in northern norway and sweden. the language is classified as a severely endangered language by unesco and has an estimated 800-3,000 speakers @xcite @xcite @xcite . lule sámi is a morphologically complex language, for more details see @xcite . the current orthography of lule sámi was approved in 1983, and the first spell checker for the language was launched in 2007. lule sámi lacks a long written tradition. according to kuoljok (1997) most of the speakers can barely read and even fewer write. this situation has changed since 1997. in the education system, lule sámi is taught and used as the language of instruction. in nor-way, lule sámi was for the first time taught as first language in primary school in 1992, and from 2012 it was possible to take a bachelor's degree in lule sámi at nord university. lule sámi is also to a greater extent used in public administration, in 2000 the jåhkåmåhke/jokkmokk municipality became one of the municipalities in sweden with a sámi-language administration and in 2006 the municipality divtasvuona/tysfjord was included in norways sámi-language administration municipalities. this development means that lule sámi is also used in writing to a greater extent than before. however, the written tradition is not very established, and the elderly heritage speakers master the written language only to a smaller extent. in 2013, a lule sámi corpus of writing errors was created to test the spell checker's effectiveness. today this corpus consist of 39,892 words, written by native lule sámi speakers, and it has all together 4,784 writing errors. 2,055 are nonword errors identified by the spell checker, while the remaining 2,729 errors are morpho-syntactic, syntactic and lexical errors that only a grammar checker can detect and correct @xcite . the mark-up of this corpus shows an error rate of 11,9% in written texts, which indicates that lule sámi speakers struggle when writing the language. to fully master a written language one must read a lot @xcite , minority language users therefore have a greater need for help in the writing process, since they do not experience their language in written form as much as majority language speakers. with lule sámi classified as a severely endangered language by unesco, it is important to increase the use of lule sámi to revitalize the language. a grammar checker for lule sámi would make it easier for people to write in the language, thus increasing its written use. to develop a functional lule sámi grammar checker, we opted to focus on errors made by proficient speakers instead of second language learners. this approach allows us to create a grammar checker that can handle sentences with fewer errors and gradually introduce more complex errors. a grammar checker for texts written by second language learners would require a different approach as they tend to have more and different types of errors, including more complex errors. errors made by high proficiency speakers often arise when the written norm deviates from the spo-ken dialectal variation or the "errors" might express an ongoing language change. ## technical background all tools described in this article are part of a multilingual infrastructure for 130 languages @xcite . lule sámi has a morphological analyser and lexicon, which are both publicly available foot_3 . the morphological analyser was originally imported with all rules and set specifications from north sámi and then adapted to lule sámi. gramdivvun takes input from the finite-state transducer (fst) to a number of other modules, the core of which are several constraint grammar modules for tokenisation disambiguation, morpho-syntactic disambiguation and a module for error detection and correction. the full modular structure is described in @xcite . we are using finite-state morphology @xcite to model word formation processes. the technology behind our fsts is described in @xcite . constraint grammar is a rule-based formalism for writing disambiguation and syntactic annotation grammars @xcite . in our work, we use the free open source implementation vislcg-3 @xcite . the challenge consists in writing rules that are as general as possible so one rule can cover many different erroneous forms at once. most lule sámi grammatical errors can be referred to as a combination of morphological features that is confused with another combination, rather than a confusion pair of two lemmata as is typical for languages with less morphological complexity like english (e.g. theirs-there's). this allows for a higher degree of abstraction. the syntactic context is specified in hand-written constraint grammar rules. the add-rule below adds an error tag (&real-negsg3-negsg2) to the negation auxiliary ij '(to) not (do)' as in example (1) if it is a 3rd person singular verb and to its left there is a 2nd person singular pronoun in the nominative case. the context condition further specifies a barrier for the rule to apply. subjunctions, conjunctions, or finite verbs -typically indicating a new clause -stop the scanning of the rule. each add-rule is accompanied by a copyrule that exchanges relevant morphological tags in order to produce the correct sequence for the fst morphological generator to generate the correct form. in this case sg3 is exchanged for sg2. at the same time, we add a tag, &suggest to mark that this is not the erroneous form anymore, but the correction. dån you.2sg ittjij neg.past.3sg boade come guossáj. guest.ill 'you didn't visit.' at first, we wrote test sentences for yaml tests ourselves and also searched sikor (sikor) manually for sentences with similar errors. after having written rules, we automatically harvested test sentences corrected by gramdivvun in the developer-corpus foot_6 , and used these to improve the rules. at first, we wrote test sentences for yaml tests ourselves and also searched sikor manually for sentences with similar errors. after having written rules, we automatically harvested test sentences corrected by gramdivvun in the developercorpus foot_7 , and used these to improve the rules. yaml is a mark-up language with a simple syntax that makes writings of the tests convenient and co-operation with programmers and linguists easier foot_8 . we chose to use the yaml format for grammar testing because of positive experiences with the use of the same format for spell checker testing. foot_9 ## evaluation for the evaluation of our tool, we use a part of sikor, the lule sámi corpus, containing administrative, law, religious, non-fiction, fiction, and science texts. sikor consists of a freely available corpus, freecorpus, and a corpus that is restricted by copyright, boundcorpus. we distinguish between three different parts: 1. the gold corpus for evaluation, marked-up for spelling and grammar errors, 2. the unmarked testing corpus and 3. the development corpus for developing rules. for simplicity, we will refer to the error marked-up gold corpus as freecorpus and boundcorpus. this work includes testing for inconsistencies and improvement of the manual grammar error mark-up the first time. since the goldcorpus consists of text that has not been proof read, there are a lot of grammatical errors. the goldcorpus and its mark-up is described in @xcite . the testcorpus is not manually marked-up, but put aside for future evaluation and quality assurance as mark-up as the current goldcorpus is still fairly small, and needs enhancement to cover all different grammatical error types sufficiently. the development corpus on the other hand, is being used to test and improve the grammar checker rules on the fly. it is therefore not marked-up. a preliminary evaluation on boundcorpus in based on the results of tables 2 and 3 , and keeping the quality assurance for the users in mind, we have released functionalities for errors regarding copula form and relative pronoun agreement, the second of which we reduced to errors regarding inessive case relative pronoun agreement. the first two error types have a good precision and perform well in regression testing. all of them have a precision above 70%. in addition, we have released error correction for error types with few instances in boundcorpus, which are based on good regression test results and knowledge about high frequency of the errors from experience as a manual proof reader. these error types are: adverbial use of the modal verb in third person singular, soajttá 'maybe s/he does'; use of attributive adjective forms instead of predicative forms; lexical confusion of the verbs @xmath2; and reflexive pronoun errors. after fine-tuning the existing error mark-up on a bigger corpus that includes more fiction texts, and therefore other error types (freecorpus), we evaluated the wellperforming rules on both boundcorpus and freecorpus, cf. copula errors are by far the most frequent ones. in both corpora together, we found as many as 498 copula errors, four times as much as only in boundcorpus. all error types except for two have a precision above 85%. the low precision of reflexive and @xmath3. in both cases, false positives are due to other errors in the text which lead to wrong corrections, but not detection. gramdivvun finds the error in the sentence, but fails to correct the error in the whole sentence structure based on other errors. altogether these are six general error types that have been released with functionalities in the first version of the lule sámi grammar checker. many of the rule types involve several rules. for example, copula correction includes three different rules: one for correcting from sentence initial to correct sentence internal forms, one for correcting the sentence internal form to the correct sentence initial form and one for choosing between the sentence internal forms la and l. the benefit of our work has been twofold, we have improved both our tools and our marked-up data. firstly, we have used rule development for automatic grammatical error detection, and secondly, we have improved grammatical error markup after running the grammar checker. this shows that consistency in manual error mark-up can be assisted by automatic grammar checking. the evaluation shows despite good precision for the six rule types that were released, there are a number of false alarms and cases where gram-divvun does not find the error. in ex. (8) and ( 9 ), the sentences are more complex than what we thought of when writing rules. in ex. (8) the grammar checker erroneously changes the attributive adjective buosjes 'tough' to predicative buossje. in this example there are two attributive adjectives connected with the conjunction ja meaning 'and'. adding coordination conditions to the rules is fairly simple to fix. some of false alarms are due to combinations of errors. in ex. (10), gramdivvun erroneously changes the plural relative pronoun ma 'that' to singular mij. therefore the subject is singular and the verb guosski 'regard' is also corrected by the grammar checker. here gramdivvun changes ma to singular which is a false positive because of a wrong referent. consequently it also tries to change the verb guosski to singular to correct the agreement with the relative pronoun. lav have.pres.1sg válljim choose.pst.ptcp teoritevstajt text.pl.acc kompendijis compendium ma that.pl.nom guosski regard.pres.3pl álggoálmmukmetodologijav. indigenous.methodology.ill 'i have chosen texts from the compendium that regard indigenous methodology.' we also have similar examples where the erroneous correction by the grammar checker is due to a combination of errors, but here it is the writer who has made two different errors. in ex. (11) the grammar checker corrects the attributive adjective váges 'reliable' to singular váhke, where it should have been corrected to plural váge. the writer has made two errors, one of which is a number error in the verb viertti 'must' (present 3.sg) which should be present 3.pl vierttiji. gramdivvun misses this subject-verb agreement error and therefore the adjective attribute form is corrected to predicative singular form. adding an agreement error rule to gramdivvun will lead to a correction of the second error. (11) moralla moral subttsasin story de then máhttá might liehket be rádna friend.pl.nom *viertti must.pres.2sg liehket be *váges honest.adj.attr nubbe each nuppijn other jus if rádnastallam friendship galggá will bissot. remain. the same problem with a combination of errors happens in ex. ( 12 ), where the writer has misspelled the indefinite pronoun iehtjádijn 'with another'. because of the typo the grammar checker erroneously corrects oahpástuvvat 'get to know' to oahpásmuvvat. ietja self dahki do.pres.3pl majt what hálidi, want.pres.3pl, ja and dan that båttå moment máhtá can.pres.2sg buorebut better *ietjadijn non.word oahpástuvvat, get.to.know.inf, javllá says inga inga lill. lill 'everyone does what they want, and at the same time you can get to know someone better, says inga lill.' there are also examples where the rules of the grammar checker work fine, but where the grammar checker erroneously corrects because of problems with disambiguating homonymies. in ex. (13) the disambiguator construes jage 'year' to be nominative plural, when it actually is genitive singular. because of the grammar checker construes jage to be the subject of the sentence it corrects the sentence-initial present copula form le 'is' to the 3pl form li instead of the correct 3sg form la. (13) badjel over guoktalåk twenty jage years *le be.pres.3sg.sent.init duodje sámi.handcraft munji me årrum be vájmoássjen heart.case ja and oasse part iehtjam my identitehtas. identity. 'for over twenty years sámi handcraft has been close to my heart and a part of my identity.' the evaluation shows that even though the grammar checker works well with six rules, there are still complex issues that cause the grammar checker to fail even for these types of errors. more errors in the same sentence makes it harder for the grammar checker. it is therefore important that the users know that this grammar checker is predominantly meant for l1 users and that upon its release, it does not work very well for second language learners' texts, yet. the evaluation shows that building a grammar checker for l1 users before l2 users is a good way to go, as the tool performs better with only one error in the sentence, and proficiency writers are assumed to make less errors. ## conclusion and future plans we have released a tool for grammatical detection and correction of lule sámi (gramdivvun) to support the lule sámi language community in writing. we evaluated our tool and based on the evaluation, we chose six general error types that met our quality requirements and were ready to be released. these are corrections regarding copula forms, lexical confusion of oahpásmuvvatoahpástuvvat, number agreement for reflexive pronouns, the use of the modal verb soajttá as an adverb, confusion of attributive and predicative adjective forms, and finally number agreement of inessive relative pronoun forms. while our evaluation corpus is still a bit too small to have a good representation of all errors, it was evident that copula errors are very frequent, and the other error types were also represented. copula errors also show the best precision with 96% and recall of 84%. in other error types, we rely on our manual proof-reading experience to know about their frequency. this goes hand in hand with our wish to focus on user demands. in the future we will improve precision and recall for the correction of existing error types by testing on more syntactic contexts. this means we will we need to enhance the corpora with error mark-up. in addition we will improve the quality of error rules that have not been included in this version of gramdivvun with the goal of releasing them. we can also conclude that even l1 language users typically make several errors in a sentence. this is due to low literacy in lule sámi, and interference errors caused by bilingualism. our focus must therefore be a tool that can handle these types of sentences.