I talian Language and Dialect Identification and Regional F rench Variety Detection using Adaptive Naive B ayes (2022)

Abstract: This article describes the language identification approach used by the SUKI team in the Identification of Languages and Dialects of Italy and the French Cross-Domain Dialect Identification shared tasks organized as part of the VarDial workshop 2022. We describe some experiments and the preprocessing techniques we used for the training data in preparation for the shared task submissions, which are also discussed. Our Naive Bayes-based adaptive system reached the first position in Italian language identification and came second in the French variety identification task.

## introduction language identification (li) of digital text still poses difficulties for text classification methods when performed in more complex situations @xcite . one of the problematic contexts is the closeness of the languages to be identified. in this article, we tackle the problem of close language identification for languages or dialects traditionally used in italy and distinguishing between regional french varieties used in europe and canada. the research and experiments were conducted while participating in the language identification shared tasks organized in connection with the ninth edition of the vardial workshop for nlp for similar languages, varieties, and dialects @xcite . the french cross-domain dialect identification (fdi) and the identification of languages and dialects of italy (itdi) shared tasks were organized for the first time. however, they were following a long line of vardialrelated li shared tasks from the discriminating between similar languages (dsl) tasks in 2014-2017 @xcite @xcite to newer, more specialized ones such as romanian dialect identification (rdi) and uralic language identification (uli) in 2020 and 2021 @xcite . the itdi shared task focused on 11 living romance languages or dialects: emiliano-romagnolo (eml), friulian (fur), ladin (lld), ligurian (lij), lombard (lmo), neapolitan (nap), piemontese (pms), sardinian (srd), sicilian (scn), tarantino (considered a dialect of neapolitan by iso 639-3), and venetian (vec). the shared task was a closed one; hence, no other data besides that indicated and provided by the organizers were to be used. the organizers also stated that the test set would contain only a subset of the 11 languages. the fdi shared task featured four regional varieties of written french from news sites in france, belgium, switzerland, and canada. the organizers of the task provided all the data. in section 2, we present previous work on language identification of the languages that are the targets of these two shared tasks. in section 3, we describe the data provided and allowed in the tasks, and, in section 4, we present our method. our experiments and the preprocessing done to the training data are explained in section 5. in section 6, we present and discuss the results of the submitted runs. ## previous work to our knowledge, there is no previous li research focusing specifically on the languages of italy. however, previous language identificationrelated research has featured the rare romance languages that make up the itdi repertoire. emiliano-romagnolo, friulian, and sardinian were part of the 372 languages featured in the research by @xcite . @xcite automatically created a phylogenetic-like tree for languages based on more than 50 versions of the universal declaration of human rights, including the friulian and the sardinian editions. lombard, piemontese, sicilian, and venetian were featured in the ex-periments leading to the development of the helimethod @xcite . lombard, neapolitan, piemontese, sicilian, and venetian were included in the li experiments conducted by @xcite . @xcite and @xcite investigated word-level language identification in multilingual documents, including mixed lombard -english, among other combinations. @xcite mention the lombard -italian pair as one of the top 10 most frequently confused pairs in the uli-178 track of the uralic language identification shared task. the uli-178 track was a general language identification task between 178 languages, among them lombard, piemontese, sardinian, sicilian, and venetian @xcite . neapolitan, piemontese, and sicilian were part of the altw 2010 multilingual language identification shared task dataset @xcite . @xcite investigated language identification in the context of web crawling and mention neapolitan, sicilian, and venetian as part of the lowest-resource languages in their research. all the languages of the task except lombard were included in the language identifier featuring more than 900 languages developed by @xcite . also, lombard was added to his version with more than 1300 languages @xcite . 1 emiliano-romagnolo, lombard, neapolitan, piemontese, sicilian, and venetian are part of the repertoire of the fasttext off-the-shelf language identifier foot_1 and lombard, piemontese, sardinian, and sicilian are included in the heli-ots off-the-shelf language identifier @xcite . 3 distinguishing between french regional varieties from france and canada was part of the overall aims in the 2016 and 2017 editions of the discriminating between similar languages (dsl) shared tasks @xcite . 4 the 2016 edition of the dsl was won by the tubasfs team using svm and character n-grams from one to seven @xcite . they managed to achieve 95.8% recall for the canadian variety and 94.0% recall for the french variety. in 2016, we came second with a heli method-based identifier using words and character n-grams from one to six @xcite . the dsl 2017, the last one of its kind, was won by the cecl team using svm with character n-grams from one to four in the first stage to detect the language group and another svm with a variety of features in addition to character n-grams such as pos tag n-grams, the proportion of capitalized letters and punctuation marks to detect the language within the group @xcite . ## method we used the same system we had developed for and used in the winning submission of the 2021 edition of romanian dialect identification (jauhiainen et al., 2021). 6 the system uses a naive bayes-based method using the observed relative frequencies of multiple-size character n-grams as probabilities. we first used the method as a baseline for the cuneiform language identification (cli) shared task @xcite and later with adaptive language models @xcite to win the discriminating between the mainland and taiwan variation of mandarin chinese (dmt) @xcite and the rdi 2021 (jauhiainen et al., 2021) shared tasks. the naive bayes type method adds together logarithms of the relative frequencies of character n-gram combinations f i in the training data c g as defined in equation 1 the system uses an adaptation technique to learn from the test data @xcite . there is also a possibility to perform iterative adaptation, in which the test data is processed several times from the beginning of the adaptation process. the exact range of the used character n-grams is optimized using the development data. after optimizing the basic method, the parameters for the adaptive version are determined. in the adaptation technique, the test data is first identified with the basic method, and a confidence score is calculated for each identified instance. the confidence score is the difference between the scores of the best and the second-best language. the test instances are sorted according to the confidence scores and then divided into a certain number of splits. the number of splits is determined using the development data. the character n-gram frequency information from the most confident split is added to the respective language models, and the rest of the material is reidentified with the adapted models. then the rest of the material is again sorted and divided into equally sized splits, and the information from the most confident split is added to the models and the rest re-identified. the previous process is repeated until all the material is added to the language models. in the iterative version, the adaptation process is restarted from the beginning. the number of possible iterations is also determined using the development set. ## experiments this section presents the details of the experiments and various preprocessing techniques we used when participating in the shared tasks. ## results in this section, we describe the results of the submitted runs and the conclusions we can derive from them.