GlotLID: Language Identification for Low-Resource Languages
Authors: Amir Hossein Kargaran
Ayyoob Imani
François Yvon
Hinrich Schuetze
Venue: EMNLP 2023 Findings (2023)
Keywords: Language Identification
Low-Resource Languages
TL;DR: The paper introduces GlotLID, an open-source language identification model with support for more than 1600 languages.

Abstract: Several recent papers have published good solutions for language identification (LID) for about 300 high-resource and medium-resource languages. However, there is no LID available that (i) covers a wide range of low-resource languages, (ii) is rigorously evaluated and reliable and (iii) efficient and easy to use. Here, we publish GlotLID-M, an LID model that satisfies the desiderata of wide coverage, reliability and efficiency. It identifies 1665 languages, a large increase in coverage compared to prior work. In our experiments, GlotLID-M outperforms four baselines (CLD3, FT176, OpenLID and NLLB) when balancing F1 and false positive rate (FPR). We analyze the unique challenges that low-resource LID poses: incorrect corpus metadata, leakage from high-resource languages, difficulty separating closely related languages, handling of macrolanguage vs varieties and in general noisy data. We hope that integrating GlotLID-M into dataset creation pipelines will improve quality and enhance accessibility of NLP technology for low-resource languages and cultures. GlotLID-M model, code, and list of data sources are available: https://github.com/cisnlp/GlotLID.

GlotLID: Language Identification for Low-Resource Languages
Amir Hossein Kargaran" Ayyoob Imani"o, François Yvon' and Hinrich Schütze
*Center for Information and Language Processing, LMU Munich, Germany Munich Center for Machine Learning (MCML), Germany Sorbonne Université, CNRS, ISIR, France amirêcis. 1mu. de
Abstract
Low-quality datasets cause poor performance for downstream applications. They can also give rise to a misleading perception of progress when cov- erage of a low-resource language is claimed based on noisy data. NLP for low-resource languages requires high-quality datasets and high-quality datasets require high-quality LID (language iden- tification). For this reason, high-quality LID for low-resource languages is paramount. To address this need, in this paper we present GlotLID-M, a high-quality LID that covers 1665 languages. We use ISO 639-3 to individuate languages.
Several recent papers have published good so- lutions for language identification (LID) for about 300 high-resouree and medium-resource languages. However, there is no LID available that (i) covers a wide range of low-resource lan- guages, (ii) is rigorously evaluated and reliable and (iii) efficient and easy to use. Here, we pub- lish GlotLID-M, an LID model that satisfies the desiderata of wide coverage, reliability and ef- ficiency. It identifies 1665 languages, a large increase in coverage compared to prior work. In our experiments, GloiLID-M outperforms four baselines (CLD3, FT176, OpenLID and NLLB) when balancing Fl and false positive rate (FPR). We analyze the unique challenges that low-resource L.ID poses: incorrect cor- pus metadata, leakage from high-resource lan- guages, difficulty separating closely related lan- guages, handling of macrolanguage vs varieties and in general noisy data. We hope that integrat- ing GlotLID-M into dataset creation pipelines will improve quality and enhance accessibil- ity of NLP technology for low-resource lan- guages and cultures. GlotLID-M model, code, and list of data sources are available: https: / //gite thub. com/ cisnIp/GlotLID
When expanding the scope of LID from few hundred 1665 languages, the problem of granu- larity becomes severe. In real-world settings, LID needs to support both macrolanguages and their varicties; it also needs to be robust against out-of- model cousins (Caswell et al., 2020; Kreutzer et al., 2022). We pay particular attention to this issue. While low-resource is our main focus, Blevins and Zettlemoyer (2022) point out that low-quality LID also affects high-resource corpora through contamination, resulting in claims of successful crosslingual transfer that are due to unrecognized coverage of low-resource languages. We also ad- dress this issue, e.g., we improve English FI on the "Universal Declaration of Human Rights" corpus (UDHR) to .85 compared to .43 for OpenLID.
1 Introduction
The NLP community should create technology that covers many languages as possible, not only medium-resource and high-resource languages. This goal can only be achieved if corpora for low-resource languages are available. Web-mined datasets - including CC100 (Wenzek et al., 2020), mC4 (Xue et al., 2021) 1 and OSCAR (Abadji et al., 2021; Ortiz Suárez et al., 2019) have made impor- tant contributions to low-resource NLP. In particu- lar, they lay the ground for multilingual neural mod- els like XLM-R (Conneau al., 2020), mT55 (Xue al., 2021) and Glot500 (ImaniGooghari al., 2023). However, existing web-mined datasets have systematic quality issues (Kreutzer et al., 2022) and insufficient coverage of low-resource languages.
Contributions. (i) We curate GlotLID-C, com- prehensive dataset covering 1665 languages, most of them low-resource, from a diverse set of do- mains. (ii) We train GlotLID-M on GlotLID-C, an open-source LID covering these 1665 lan- guages. (iii) In our experiments, GloULIDMM out- performs several baselines by more than 12% ab- solute Fl UDHR, which we take the best benchmark for our focus on low-resource lan- guages. (iv) When balancing F'l and false posi- tive rate (FPR), GlotL.ID-M also outperforms base- lines on FLORES 200, which dominated by high- /medium-resooe resource languages.

---

would like to rely on uncertainty measures to distin- guish cases where the highest-probability language is certain from those where it is not. This would low-resource LID because we want address the need for high-quality corpora for low-resource lan- guages. Line-by-line LID filtering is an effective method for achieving high corpus quality. Re- liable LID can eliminate various types of noise (see (Caswell et al., 2020; Kreutzer et al., 2022)) including data from other languages and non- linguistic data that is frequent, especially in web-crawled content. By adjusting the confidence threshold, users will have control over the level of quality of the corpora they create.
2 Requirements for low-resource LID
Main use case: Corpus creation. Corpus cre- ation and cleaning is the main use case for our
allow us to choose a level of confidence for the resulting corpus. For example, we may want to retain only sentences identified with high con- fidence (say, 70%). This is essential to produce high-quality low-resource corpora.
Because of this requirement, we do not consider Frane (Wormer, 2014) as a baseline. While has many desirable properties, it generally does not provide well-calibrated probabilities. It usually returns several classes, giving 1.0 to the top class and values close to 1.0 to several others.
Efficiency. LID is easy to run in parallel, but we still need an efficient solution to make it applicable to large corpora, not least for ecological reasons. Lack of efficiency is the reason why we do not use AfroLID (Adebara et al., 2022) as a base- line, despite its excellent coverage of African lan- guages. AfroLID is a transformer architecture and less efficient than its competitors.
Broad coverage of languages, minimize out-of- model cousin errors. We strive for as broad a cov- erage as is possible given available datasets. This has two benefits. First, it reduces "fut-of-model cousin" errors (Caswell et al., 2020; Kreutzer et al., 2022), i.c., reduces the risk that language not covered is misclassified as closely related covered language. Second, having LIDs that discriminate many low-resource languages is a pre-requisite for developing NLP technologies for the largest possi- ble number of languages. Yet many existing LIDs only cover few hundred languages. In this study, we therefore focus LIDs having broad cover- age. excluding CLD2 (McCandless, 2010), Equi- lid (Jurgens et al., 2017), Langdetect (Shuyo, 2010) and ly (Lui and Baldwin, 2012). These LIDs cover less than 100 languages or are outper- formed by the models we compare with.
Granularity flexibility, When scaling LID from few hundred languages more than 1500, it is hardly practical restrict the set of labels to a sin- gle level of the language hierarchy (e.g., using re- sources like https: /ttss///ison ////ss669-3333 s.1....... org). This is due to the complexity of defining and de- limiting languages, including the coexistence of macrolanguages and their varieties. In many cases, we want to keep both the macrolanguage and the va- rieties in our label set because the varieties we have data for important languages in their own right. But for other varieties, we do not have variety- labeled data, so the only way to include them is through the macrolanguage. For example, FLO- RES covers the macrolanguage aka (Akan) and its variety twi (Twi), but not its variety fat (Fanti), Keeping both aka and twi gives flexibility to LID users: they can either differentiate aka and twi or they can consolidate the two labels to the single label aka, depending on what makes more sense in their setting.
Open-source. LIDs should be open-source to encourage open collaboration and conform to best research practices. Some LIDs that meet our other requirements are not source, e.g., those pub- lished by Caswell et al. (2020), Bapna et al. (2022) and Kudugunta et al. (2023). CLD3 (Botha et al., 2017: Salcianu et al., 2018) is freely available, but its training code is not open-source.
Ease of use. LIDs should be easily deployable across platforms and programming environments without having to worry about dependencies, com- patibility and lack of maintenance.
Dataset curation
Because of this case-of-use requirement, we do not consider whatlang (Brown, 2014b,a) nor idNet (Dunn, 2020), two broad-coverage LIDs that meet many other requirements, but are hard to use in many practical scenarios due to software issues and lack of maintenance.
We now deseribe GlotLID-C, corpus for LID training that covers 1665 languages.
Source selection. We choose sources that we deem trustworthy (i.c., high chance of correct lan- guage label). To address the domain sensitivity of
Uncertainty assessment, In our use cases, we
It has no coverage of other low-resource languages.

---

LID and broaden language coverage, we curate diverse set of text domains.
ing any sentences from GlotLID-C, there lit- tle contamination of UDHR for low-resource lan- guages. We follow here most prior work which has the problem of contamination of UDHR for high-resource languages. We will however remove from GIotLID-C train the sentences causing con- tamination as part of our next release.
We review sources referenced by ImaniGooghari et al. (2023); Burchell et al. (2023); Blaschke et al. (2023); Adebara et al. (2022); Adebara and Abdul- Mageed (2022). In each case, consider the collection methodology, selecting sources whose language labels are trustworthy. We generally do not use web-crawled sources to avoid the associated problems (Kreutzer et al., 2022). Most selected sources are derived from Wikipedia, religious texts, collaborative translations, storybooks, and news sites. This gives us a coverage of 1 1832 languages, more than any other public LID. For list of data sources, see $A.
GlotLID-M
We select FastText (Joulin et al., 2017) as the ar- chitecture for GlotLID-M, because it satisfies all requirements outlined in $2 will explain now.
We train our FastText model GlotLID-M on GlotLID-C train with 1832 languages. FastText can easily handle the large number of languages in the corpus. Because of this broad coverage, out- of-model cousin errors are reduced. Although we restrict the number of classes to 1665 for some experiments (c.g., in Table 1), GloULID-M's classi- fication always uses all 1832 languages to mitigate out-of-model cousin errors. This satisfies the first requirement from $2: GlotLID-M is a useful tool for corpora creation because has a broad coverage of languages that can occur in raw data.
Preprocessing. We ensure that each sentence is written in the correct script, based on the writing system databases of Kargaran et al. (2023) and van Esch et al. (2022). We use the GlotScript (Kargaran et al., 2023) Python library to determine scripts.? We also eliminate duplicate sentences. Statisties. Our final corpus, GlotLID-C, com- prises 289 million sentences (i.e., lines of data) totaling 40GB and spans 1832 languages (identi- fied by their ISO 639-3 code). 1677 languages have more than 1000 sentences. Refer to $D for the total number of sentences per language.
FastText provides an open-source codebase for training, which supports customization and exten- sion of GlotLID-M.
Train/test split. We designate 85% of the data as GlotLID-C train. Let ny be the number of sen- tences from language / the remaining 15%. Then we sample min(1000, ni) sentences from it. We refer to the resulting dataset as GlotLID-C test. Contamination. To make sure evaluation data (especially UDHR, refer to 1).1) do not over- lap with our sources, we compute contamination of UDHR GlotLID-- train.
FastText is easy to use: It offers a number of lan- guage bindings, making it compatible with multiple programming languages (including C++, Python, Java, Node.js, Rust, Ruby, R) and reducing depen- dency, incompatibility and other software issues. FastText meets the requirement of uncertainty assessment because provides confidence scores that can serve as thresholds to effectively mitigate noise in the data. For the same reason, FastText also supports granularity flexibility: we can accu- mulate probabilities over language varieties to get good estimate of the probability of the macrolan- guage. To this end, we simply add to the macrolan- guage probability the probabilities of its varieties. This way, the system can return appropriate esti- mates at various levels of granularity.
We count UDHR test sentence as occurring in the training set if all of its word four-grams occur in one sentence of GlotLID-C. Most of these contam- inations are due to resources: Wikipedia and Tatoeba.3 3 GlotLID-C train shares 374 languages with UDHR.
For 292 languages, we find that none of the UDHR test sentences occurs in the training data. For 57 languages, less than 10% of UDHR test sen- tences oceur in the training data. The remaining 25 languages with contamination rate over 10% are all high/medium resource languages.
As a professionally designed and implemented linear classifier, FastText is efficient: it had the best throughput of the candidate solutions tested and can process large corpora with high speed. As a linear model, FastText has the additional advantage of delivering explainable classification decisions. FastText a multinomial logistic classifier. The in- put sentence represented as an average of n-gram
In our experiments, we decided against remov-
Phttps: thub. http:///.t...ccmccca///a/tttasa/ htt.aataabbb..ogge//////llo..o.a

---

embeddings. This allows us visualize how much each n-gram contributed the final prediction. See NLLB Team et (2022), Fig. 8, for details.
The level of granularity across language (sub)families varies in FLORES; e.g., includes nine varieties of Arabic. On the other hand, some languages (e.g., est:Estonian) are only available as macrolanguage. In some cases, FLORES in- cludes both macrolanguage and varieties, c.g., aka (Akan) and its variety twi (Twi), and zho (Chi- nese) and its variety yue (Yue Chinese). Although some issues have been reported (see $C.1) with FLORES, we do not have the resources investi- gate them, so we use it as is.
Taking all these requirements together (and its good LID performance demonstrated in $6 and acceptable calibration in $F), GlotLID-M, based FastText, is, in our opinion, excellent tool for supporting our use case, the creation of high- quality low-resource corpora.
5 Experimental setup
UDHR consists of more than 500 translations of the "Universal Declaration of Human Rights". 419 translations available from the "UDHR in Uni- code" project have iso-639-3 code that is not "und" (undetermined)). We discard short sentences (e.g., consisting of just an article number or the single English word 'missing') by discarding the 35% shortest sentences for each language.
We train GlotLID-M on GlotLID-C train using the hyperparameters (NLLB Team et al., 2022: Burchell et al., 2023) and otherwise FastText defaults (see $B). Following Arivazhagan et al. (2019), NLLB Team et al. (2022) and Burchell et al. (2023), perform up-sampling for low resource languages. Sentences from language represent- ing Pt of the dataset are sampled proportionally to Pf where T the temperature. Following NLLB Team et al. (2022) and Burchell et al. (2023), we set + = .3
In some cases (e.g.. Zulu and Quechua). UDHR contains both macrolanguage and one of its va- rieties. We have also seen some issues in UDHR (see $C.2), but we have not extensively investigated these potential problems.
5.1 Evaluation data
5.2 Baselines
We evaluate GloMLID-M on GlotLID-CC test, FLORES-200 (NLLB Team et al., 2022) and UDHR* (Universal Declaration of Human Rights). While testing on data unseen in training is stan- dard in NLP, the results have to be taken with grain of salt because there is often domain mis- match in real-world applications of LID (Caswell al., 2020; Dunn, 2020). FLORES-20 and UDHR address this concern: they are not part of our training set (however, see discussion $3) and do not draw on our sources. Many other benchmarks share sources like Wikipedia with us (Thoma, 2018; Haas and Derezynski, 2021; Ah- madi et al., 2023). FLORES-200 and UDHR are also the benchmarks with the broadest available language coverage.
Our baselines are FT176, CLD3, NLLB (NLLB Team et al., 2022) and OpenLID (Burchell et al., 2023). The first two were used for filtering the resources OSCAR and mC4 (Kreutzer et al., 2022). CLD3. CLD3 uses an n-gram (I<n<33 based neural network model. CLD3 sometimes deviates from established metadata conventions. For exam- ple, ISO-639-1 ku refers to kur (Kurdish), but in CLD3 ku refers to its variety kmr (Northern Kur- dish). It refers to Hebrew as iw, but the ISO code for Hebrew has changed to he and heb.
FT176. FT176 FastText model that uses Wikipedia (WP) codes as labels. The documenta- tion of language metadata sometimes unclear; e.g., FT176 refers to Alemannic German as als al- though ISO-639-3 als is Tosk Albanian. It refers to the Malay macrolanguage as ms, but unlike ISO- 639-3, this does not include ind (Indonesian).
FLORES-200 is a collection of 842 articles ob- tained from English-language Wikimedia projects. Each sentence in the articles was translated into 204 distinct language-script combinations, correspond- ing 196 distinct languages, and human- human-verified verified. It provides 997 sentences for development, 1012 for dev-test and 992 for test. FLORES-200 test is not publicly available. Following prior work, we use dev-test as our FLORES test set.
NLLB and OpenLID. NLLB and OpenLID are Fast models. Their language label sets are mostly taken from FLORES, so granularity and coverage are similar to FLORES.
Language metadata matching, Matching the
'https: h/tp::://faaa fast text. ec/ doca/////// language identi ficatioon html
'http:/ //www////wwww/// uni

---

Decision rule
Given an LID classifier m, base set / of languages and a threshold 6, we assign label 6(8, H, to sentence as follows:
undetermined a (1])))
if maxics P..((ls
6(8 m, B,0)
otherwise
We distinguish two scenarios: SET! and SET?.
In sccnario SET!, the set of languages covered by the evaluation benchmark is known. We restrict model's predictions to those languages that occur in the benchmark. This means that B is (proper or improper, see table captions for detuils) subset of the languages occurring in the henchmark.
In scenario SET?, the set of languages covered by the evaluation benchmark is not known. We do not restrict model m's predictions: the model considers the entire set of languages it was trained on. This means that H is the set of languages that m was trained on.
Figure I: Decision rule for assigning classes (i.e., languages) in language identification
metadata of the models to the metadata of the benchmarks (FLORES, UDHR, GlotLID-C) is not easy. First, models do not consistently adhere to standard language codes. In addition, differences in granularity require matching rules. For example, if benchmark only covers macrolanguage and none of its varieties, then we consolidate classi- fication decisions for the macrolanguage and its variations into the macrolanguage label. See ŞE for details on metadata matching.
scenario. Instead of hundreds of languages that are not supported by all models, we have more than a thousand. We therefore run evaluations on the data for all languages not just for M1 n M2 n (TT). evaluations the entire benchmark
That is, T. not the subset in M1 O M2 B(T). This is the SET? setting in Table where SET? signifies that the LID is not given prior knowledge about which languages occur in T. For example, for the comparison of CLD3 and GloMLID-M on FLORES in the top part (SET?) of Table 2, both CLD3 and GlotLID-M are run on the entire FLORES test set. We do not exclude the languages that are present in T, but not part of MCLDS O M Ma i.e., the languages outside of the set of 95 languages common CLD3 and GlotLID-M.
Confidence thresholds. For CLD3, we use .5 and .7, the two preset thresholds in Google's CLD3 repository. For the other three baselines and .3 as the second threshold value because .7 severely reduces the number of positive predictions for the FastText models, resulting in low Fl.
GlotLID-M, also use .5, but
Macro average. For fair comparison to prior work, we restrict the macro average over languages to a subset of languages in order to replicate the experimental setup of this prior work. This subset is indicated in the tables.
Prior work has not systematically investigated the effect of confidence thresholding. However, it is of key importance for our use case of creating high-quality corpora for low-resource languages. See $5.3 and $6 for discussion of this point.
Realistic evaluation for low-resource scenar- ios. We believe that our new evaluation setup SET? better approximates real world situations. In clean- ing pipelines, LID models are often presented with an unknown of languages without prior knowl- edge. Therefore, it crucial for an LID to have the capacity to handle unknown languages. This can be achieved by setting a threshold on the confidence scores. If the confidence score for predicted label falls below the threshold, the model should label the input text "undetermined". This reduces the risk of languages unknown the model being incorrectly categorized as a known language (the out-of-moda. problem). Consequently, when com- paring LIDs, it is necessary to apply each model to the entire benchmark.
5.3 Decision rule
Figure defines decision rule.
SET! scenario. When comparing LIDs mi and m2 (trained on the of languages M, and M2) on a benchmark T (supporting the set of lan- guages B(T)), many evaluations create a subset M₁ n M2 n B(T) and remove all sentences in the benchmark that are labeled with languages outside of M1 n M2 n BTT). SET! evaluation replicates this standard way of evaluating LIDs.
SET? scenario. We believe that the SET! sce- nario makes the LID task unrealistically easy: portion of the data that could give rise to false pos- itives (data not in Mj n M2 n B(T)) is removed. It is particularly unrealistic for our low-resource

---

GioLLI M. 8=.0
GloiLID M. 5
probability is) and 0 .5 (i.e., we only assign a language label if its probability exceeds .5). See Figure 1 for the definition of our decision rule.
Benehmark
[[]]
F11
FPR]
F11
FPR.
GletLDD C
all
1832
.940
.0055
.338
D0033
GlatLID C
subset
1665
.977
,0033
.933
.0022
UDHR
.734
.0007
Focusing on the "subset" results for 0- .5, Fl is .973 on GlotLID-C and .924 on FLORES; and FPR is ,0002 on GlotLID-C and ,0010 on FLORES. This is a very good performance, in particular for the use case of low-resource corpus creation be- cause low FPR means that the resulting corpora will be less contaminated. On UDHR. again for the "subset" results for 0 = 5, F1 is .770 and FPR .0006. This is again an encouragingly low FPR, but Fi is quite a bit lower than for GlotLID-C and FLORES. The reason is that we have a domain shift (compared to GlotLID-C) and many more lan- guages (compared to FLORES), resulting in lower FI. Although the UDHR results should be im- proved further, we will now show that they outper- form the state of the art.
UDHR
subset
342
.784
.0144
.770
.0006
FLORES-200 200
all
196
917
.042
.887
.0013
FLORES-200
subset
177
.957
.029
.924
.0010
Table 1: Performance of GlotLID-M on GlotLID-C, UDHR and FLORES-200 test sets. Subset: restriction to an "operational" subset of languages that are either high-resource or for which GlotLID-M achieves F0 and F.R5 on GlotLID-C test. L: intersection of
GloMLID- languages (all: 1832 or subset: 1665) and languages present in benchmark. Referring to Figure 1, the size of the base set B is either 1832 (all) or 1665 (subset). L is the set of languages over which the macro average is computed. For example, for the last line (FLORES-22) 200 subset), B consists of 1665 languages and the reported macro averages are computed over 177 languages.
Table 2 compares GlotLID-M with four base- lines. We consider two evaluation settings (SET? and SET!) and three thresholds 9. The top part of the table (SET?) corresponds to the case where the set of languages in the benchmark is not known, i.e., the LID makes predictions for all languages it was trained on. In contrast, in the SET! setting (bot- tom part), the set of languages in the benchmark is known, and cach LID only makes predictions for those languages. SET? is more realistic setting, as we usually do not know which languages occur in corpus that needs be cleaned.
5.4 Evaluation measures
Unlike some older prior work (Jauhiainen al., 2019b), we do not use accuracy because classes are highly imbalanced. Instead, we follow recent prior work (NLLB Team et al., 2022; Burchell et al., 2023) and use Fl and false positive rate (FPR). Fl is an aggregate measure of precision and recall, both of which are important: we want accurate classifications decisions (precision) and we do not want to lose too much data (recall). FPR is defined as FPR- FP+ where FP is the number of false positives, and TN is the number of true negatives. FPR helps us assess the potentially fatal effect of an even low false positive rate when the negative class is huge which is the case in our scenario. For example, an FPR of .01 (which prima facie may seem ok) for language 1 with base frequency .01 can result in corpus for / that contains 50% noise, an unacceptably high level.
For the SET? setting, GlotLID-M consistently outperforms CLD3 by large margin. Taking into account that F1 and FPR should be balanced, we also take it outperform FT176. Even though GlotLID-M's FPR is slightly higher in some cases, its Fl is better by a large margin, so that it is clearly the better performing system.
On UDHR, GlotLID-M also clearly outperforms OpenLID and NLLB for F1 and FPR by large mar- gins. On FLORES, F1 is slightly worse and FPR slightly better compared with OpenL.D and NLLB. We point out that this comparison is not entirely fair since OpenLID and NLLB were designed with FLORES in mind. More importantly, our use case is the creation of low-resource corpora for which UDHR is the more appropriate benchmark.
6 Results
Table gives results on GlotLID-C test, UDHR and FLORES-200. GlotLID-M does not perform well on some languages. In particular, there are 167 (1832- 1665) low-resource languages for which ei- ther Fl Fl1 or FPR>00005, often due to very small GlotLID-C training sets. The table gives results for "all" 1832 languages as well as for the "sub- set" of 1665 well-performing languages. We run GlotLID-M in two settings: 8=.0 (i.c., we choose the highest probability class no matter how low its
Comparing results for different thresholds, we observe that increasing 0 lowers F1 (because re- call is hurt) and lowers FPR (because precision is increased). This suggests that a higher thresh-

---

FLORES-200
UDHR
CLD3
FT176
OpenIID
NLLB
CLD3
FT176
OpenI.ID
NLLB
[r] - 96
F.
108
[T_ - 195
[1| 188
[T] - 100
[.] -
124
[7]
159
[T]]
172
LID Model
F1
FPR.
F11
FPR.
F1
FPR
F11
FPR
F1
FPR
F11
FPR
F11
FPR.
FIt
FPR.
beselines
.0
.753
.0088
.775
.0090
.923
.0051
.947
.0053
.544
.0099
.666
.0079
.645
.0056
.641
.0051
boselines
6,
.779
.0081
816
.0050
.948
.0051
.576
.0255
.676
.0046
.0040
SET?
baselines
9g
.799
.0060
79%
.0021
.923
.044
.947
0047
.618
0060
6677
.0114
.718
0004
.217
.0030
GlodLID M .0
-978
.0051
9887
.0042
.916
.0043
.947
.0035
.68
0003
8688
00030
.848
.0020
.847
.0019
GloILID-M
.3
.980
0042
987
00.37
898
0000
927
.019
881
0028
879
0026
846
.015
.844
.0155
GloiLID-M
5
.980
.0031
-987
0029
886
.0014
916
.0013
.903
.0033
0021
847
.0012
846
.0111
baselines .0
.952
.0104
881
.500
.0053
.81
.0063
.844
.0588
GloiLID-M
.0 .983
.0104
.991
.0093
.92
.0051
.954
.0053
.952
.0100
.927
.0081
.96
0064
.925
.000
Table 2: Evaluation of LID performance. Top ("SET?")) The set of languages not known, i.e., each LID makes predictions for all languages it was trained on. Bottom ("SET!"): The set of languages is known: each LID only makes predictions for languages that occur in the benchmark. For the more realistic "SET?" setting, GlotLID-M - outperforms the baselines on UDHR (which we take to be the best benchmark for the low-resource case) assuming a good tradeoff between FPR and Fl is desired; it either matches or outperforms them on FLORES. Let M, be the set of languages model m, was trained on and (T) the set of languages covered by benchmark T. Then Fl and FPR are averages over L M1 M2 n B(T') when comparing models my and m2: this is indicated in the third row of table, e.g., [L] 96 for m CLD3, m2 = GIotLID. 0 for CLD3, =1=.3 for FT176, OpenLID and NLLB. 02=.7 for CLD3, 02 for FT OpenI.ID and NLLB. Referring to Figure I, the base set B in SET? has size 103 for CLD3, 176 for FT176, 195 for OpenLID, 211 for NLLB and 1832 for GlotLID-M (i.e., the languages the LID was trained on). For scenario SET!, B = I., i.e., B = M1 n M2 n B(T). For example, B = 96 (for both CLD3 and GIotLID) for the four cells in the the SET! rows and the CLD3 columns in the lower left corner of the table. The best result in each column is bolded, and the second-best result is underlined.
old should be used since lower FPR will result in low-resource corpora with less contamination from high-resource languages.
a high number of errors. Table (top, "most er- rors") gives for each of the three benchmarks the five languages that have the highest number of er- rors (column "language"). "FFP" is the number of false positives, "cl" the ratio of true positives to all positives (that is the "cleanness" of the corpus), "top FP source" the language that contributed most of the errors and "%" the portion of these false positives as percentage of all false positives. We use the cl measure in our analysis because is ultimately the measure we want to optimize to pro- duce high-quality low-resource corpora. Note that cl (the denominator is the total number of posi- tive sentences) is not directly related to FPR (the denominator is the number of sentences that do not belong to the language). cl is more direct measure of the utility of the resulting corpus of a low-resource language (e.g., for training a language model) than FPR.
For the less realistic SET! setting, GlotLID-M performs better than CLD3 and FT176 and compa- rably to OpenI.I and NILB. Overall, GlotLID-M clearly outperforms all baselines for the low- resource corpus creation use case.
To analyze variance of results, we ran three GlotLID experiments with different initial seeds on the 200 languages with the most data, splitting the data into 80% train and 20% test. The Fl score was .991 each time. This indicates that the variance of FastText in this task (and by extension GIotLID)
is negligible.
7 Analysis
In this section, we analyze the GlotLID-M results summarized in Table (0=.0, "all") for our main use case, the creation of high-quality corpora. We address four questions.
Most of the fifteen pairs of "conflated" lan- guages shown in the table are closely related languages: varieties of Arabic (Standard, Na- jdi, Egyptian and Levantine), Persian (Tranian, Dari), Chinese (Mandarin, Yue, Wu, Hakka), En- glish (Standard, Liberian), Quechua (Huallaga Huánuco, Huamalíes-Dos de Mayo Huánuco), Finnic (Finnish, Karelian), Slavic (Russian, Church Slavic), Bihari (Bihari, Bhojpuri) and Hindi (Stan- dard, Awadhi). In many of these cases, speakers of
(i) For which languages do we get a high number of false positives? (ii) For which languages do we produce a corpus with a high contamination rate? (iii) For which languages does learning completely fail? (iv) Is it more re- alistic evaluate LID a balanced test set (as in prior work) or on one that is skewed in favor of high- regh-resource languages?
Most errors. We first analyze languages with

---

FLORFS-220
UDHR
longwage
FT
<l top ET source
AFT
%
longwage
FF
<l top FP source
4FP %
langunge
FT
cl top ET source
AF1
%
Sr Arabic
5787
arxiNajii Anati
524
.22
Mundaria Ca
36
chrrCheccee
Spenish
1952
.34
PidiParoa
156
apeil accntne
qpttllummalee-
englinggish
lirt
postt: Persian
1895
prstuan
.01
BicHimiish
22
kattKazelian
135
Russussia
1057
churtuurc
661.
cmn MmnnMandarn Ch
1008
(0) your Chinese
100R
694
Wo Chinese
24
Hakka Chine
26
882
50 bih-Bkhaari Lgs
854
his Hindi
997
51 owa-Awedii
093
.71
rus Fus-Rassion
157 28 Golykkk
44
24
LieiRei En
712
.77 din-Diiinn
174 .24
mon 200g
St Acabic
5787
18 arx.Nijii Anabi
529
.22
Eveaki
36 25
caacOrolc
Russiisiaan
1057
49
ChurChurcla
661
arrtigypeun
1726
apeiLevantine
quacCussco Quech
quucArecuipa La
eng lnglish
254
po:Dect
338
pbc.S Puslito
300
-22
lav:Cioatiann
42
bos, Bosaian
spa Spanish
1952
pid Plaros
dyu:
255
.2 bam:Bambbara
235
.99
tane Atlas Tamn
52 122 zeh St Mocoocan
52 99
cog:boo "wujaa
547
.47
cititrojaz' ja
547
.99
qpe Levustine A
161
.42 ajus Leventine
70
.43
uzn.N Uzbek
72
46 ab Cundoshi-Sh
16 .22
ert Jyojwa ja C
608
A48 eig Iyo wujwa C
697 .99
10 no posiiies
lct Texum
Sadii
.00
Nann Chine
Chagaai
.00
00
Bel.iv
.00
vep Vaps
X00
.00
Gilyak
00
Imw: Tomman
.00
St Arshic
3787
Nijiii Anéi
629
.22
Mandarin
che Chenokee
Mansein
567
.09
won/Wia Chinose 208
Dzongkiia
bod. Tibetas
10300
.9
Finiish
Kut/Kaseliat
eng Englishh
1267
254
hin Hindi
977
60%
76 99
MaiiMaibilli
hinifindh
488
bheeRhogpuii
FasdRissia
.99 buil Bulgariun
.99
rus rus.Rassia
256 09 eng English
100 .3
nu Russian
1156
.99 churChunds Slev 161 .57
10 .99 sad Asturian
7 700
62. 99 agr Agunnnn
20 32
SpeSphnnish
1952
(94 pid-Panna
156 085
Table 3: Analysis of the GloMLID-M runs with settings 0=.0, SET? from Table and Table 2. "most errors": languages with the most false positives. "most noisy": sample of languages with cleanness between 0 and .5. "no positives": sample of languages without positives. "hi resource": more realistic setting in which the distribution is skewed in favor of regoures languages. For each "language". we give the number of false positives ("FP"), the cleanness of the resulting corpus ("cl": ratio true positives to all positives), its most conflated language ("top FP source"), FP contributed by that language and the ratio of the two FP numbers ("%"". To save space, we write .99 for 1.00.
one variety of the pair also have good knowledge of the other; e.g., many speakers of Arabic varieties know Standard Arabic. The two Quechua varieties are spoken in neighboring areas of Peru. The quan- titatively largest use of Church Slavic (which may be reflected the size of our corpora) is in Russia by Russian speakers.
pora will need to be improved: they most likely contain many sentences labeled as Gilyak/Piaroa that are partially or completely Russian/Spanish. We leave it to future work to revisit and improve our corpus selection and preprocessing methodol- ogy to address this data quality problem.
GlotLID-M confuses Mandarin and Cherokee because our Cherokee training data do not cover the Cherokee syllabary script. Sentences written in this script are returned with a close to uniform distribution over several other scripts, including Chinese, Japanese and Thai, which explains the confusion. The Dinka test set is noisy. In a man- ual inspection, we found 377 sentences that are clearly English, not Dinka. Because GD- did not learn very well to discriminate English and Liberian English, 174 of these 377 sentence were classified as Liberian English.
Arabic, Chinese and English (and perhaps also Hindi, Persian and Bihari) are diglossic linguistic communities. There may be lack of clear sepa- ration between the two conflated varieties in the available corpora because speakers switch back and forth between more formal and less formal ways of speaking depending on factors like context, au- dience and subject. This type of fluid switching between languages often occurs in single sentence or conversation, i.e., it manifests as code switching. As a result, much of the text (and speech) produced in one language may be mixed with the other lan- guage. New methods will have be developed to deal with these quite complex challenges of creat- ing training corpora for language identification; see also (Aguilar et al., 2020).
Most noisy corpora. The second part of Ta- ble ("most noisy") gives, for each benchmark, a random selection of five languages whose clean- ness score cl (ratio of true positive to all posi- tives) is in the range 0<cl<<... The total number of languages in this range is 9 for FLORES, 27 for UDHR and 6 for GlotLID--C Again, most of the conflated pairs are closely related languages as in the last section. Additional pairs that oc- cur here are Dyula/Bambara, Evenki/Orok, Croat- ion/Bosnian, Berber languages (Standard Moroc- can Tamazight, Atlas Tamazight) and two varieties of Chorote (Iyo' wujwa, Iyojwa'jal ja). The resulting
Apart from these related languages, at least four conflated language pairs in Table 3 are clear er- rors: Mandarin/Cherokee, Russian/Gilyak, Span- ish/Piaroa and Liberian English/Dinka. Similar to the situation we described for the closely related languages, Gilyak (resp. Piaroa) is spoken in an area where Russian (resp. Spanish) is the dominant official language. This means that our training cor-

---

corpora are noisy, an issue that we will have to address in future work.
contributed by the low-resource language Piaroa (156) are the same. But since the size of Spanish is increased 100x, its cleanness improves from .34 for the unrealistic uniform distribution to .99 for the realistic skewed distribution. Thus, as we would expect, LID for high-resource languages is a rela- tively easy problem and this does not change much if we run a broad-coverage LID like GlotLID-M.
No positives. Part 3 of Table 3 ("no positives") gives five random examples from languages for which there was not a single positive classification. There were no such languages for FLORES.
For UDHR, we identified two reasons. (i) Perfor- mance on GlotLID-C is good, but poor on UDHR. Tetum is an example. The most likely cause is a domain shift or some other big train/test difference. (ii) The training set is too small (less than 30 sen- tences): hsn (Xiang Chinese), abk (Abkhazian), vep (Veps) and niv (Gilyak) are in this class.
Conversely, LID numbers for low-resource lan- guages can be negatively affected. The Dzongkha corpus generated from FLORES in the uniform set- ting has 103 false positives and a cleanness of .91 (not shown). In the skewed setting, making Tibetan a high-resource language causes 10,300 false posi- tives from Tibetan to Icak into Dzongkha, reducing its cleanness to an unacceptable .09.
For the five GlotLID-C random examples with no positives, the reason is also that the training sets were too small (less than 40 sentences); sek (Sadri), chg (Chagatai), liv (Liv), gbm (Garhwali) and tmw (Temuan). We should have set a higher threshold for minimum size of the training corpus. Note that the number of 1665 languages that we use throughout the paper already reflects this insight. Even though we train on 1832 languages, we claim reasonable performance for only 1665 (Table 1).
This discussion suggests that the established evaluation methodology for LID is unsatisfactory. We recommend that future work considers both unifom and skewed test sets to better assess how LID is expected to perform in the real world.
This analysis demonstrates how much harder LID becomes when we represent as large and di- verse sets of languages we do. What we have shown is that there is a real danger of creating cor- pora that are badly contaminated. To address this, we need to develop methodologies and resources that better handle low-resource languages.
Test set skewed in favor of high-resource. FLORES and UDHR test sets are balanced: high- resource and low-resource languages have about the same size. Following this model, we con- structed the test set of GloLLID-C in the same way. Fl is independent of this distribution, but FPR and cleanness ("cl") are strongly dependent on it. The Spanish corpus generated by GlotLID-M GlotLID-C test has a dismal cleanness of only .34. Is this a problem for Glo!LID-M?
Based on the analysis described in this section we created and open-sourced a much improved ver- sion of the UDHR test set for evaluation of LID. 6 All UDHR results in this paper are based on the version of the UDHR test set descibed in $5.1.
We believe the answer is no, as the corpora run LID on will have distribution skewed in fa- vor of high-resource languages. To simulate this more realistic scenario, the last part of Table ("hi resource") gives five selected languages for each benchmark where we have inflated the subsets for high-resource languages by factor of 100. For ex- ample, instead of a single copy of the English part of FLORES, the test set now contains 100 copies.
Conclusion
We create GlotLID-C, an LID resource that cov- ers 1832 languages, several times more than prior work. We introduce GlotLID-M, an open-source LID that covers 1665 languages with good results. The comparison of GlotL.ID-M against four LID baselines shows superior performance for the low- resource use case. In future research, we would like to improve quality of our training corpora and add more low-resource languages in to GlotLID. We hope GlotLID will be a valuable resource in creating higher-quality corpora for low-resource languages.
We see in Table 3 that this results in clean corpora (cI=.99) for each of the fourteen high- resource languages shown: Standard Arabic, Hindi, Russian, Spanish (FLORES); Mandarin, Finnish, Hindi, Russian, Spanish (UDHR); Mandarin, En- glish, Hindi, Russian. Spanish (GlotLID-C). As an example, looking at Spanish for GIoLLID-C (the first and last lines in the table), the number of false positives (1952) and the number of false positives
Phttps://huge i cis 1e-1mmuudhr-1id

---

Limitations
Ethics Statement
(I) We publish list of GlotLID-C data sources as part of this work. There is no other LID bench- mark available that covers many languages GlotLID-C does. GlotLID-C, FLORES and UDHR all have drawbacks as evaluation datasets for LID. An LID trained on GlotLID-C train and tested on GlotLID-C test will often find the same domain in the test set as in the training set. It is well known that this results in overly optimistic evaluation numbers. FLORES and UDHR con- sist of data that were not originallly produced in each language. Rather, they were translated from high-resource languages. The same is true lesser extent for GlotL.ID-C. Translated language is only an imperfect evaluation benchmark because it can differ greatly from natural language data, i.e., translationese is often not good model of natural language data.
We here highlight key ethical considerations for GlotLID.
Data. The data used in our study comes from openly available (but not necessarily freely redis- tributable) datasets, including resources previously published by researchers, publishers, and transla- tors. We ensured that the data collection process complied with licensing of each dataset. Bias. We recognize potential biases towards higher resource languages. We conducted a com- prehensive analysis of errors and evaluated their impact on our results.
Inclusivity. We acknowledge the challenges associated with low-resource languages and have taken steps to include diverse range of languages in our study.
Ethical Use. We have demonstrated both posi- tive and negative outcomes of applying GlotLID-M as an LID tool. We acknowledge that GlotLID-M has a high error rate for some low-resource lan- guages. This means that there is potential risk of excluding low-resource languages during the collection and processing of NLP corpora.
(2) Many corpora for the lowest resource lan- guages are derived from religious sources. It should be noted that many Bible translations do reflect actual language use.
(3) We do not conduct hyperparameter search and instead use the hyperparameters employed by previous studies. However, conducting such a search can make our findings more robust, con- sidering the difference in the number of languages included in our study compared to the prior work. (4) Although we tried our best to select the most suitable LIDs as the baseline. We could not compare against all of the LID models. This includes CLD2 (McCandless, 2010), Equi- lid (Jurgens et al., 2017), Langdetect (Shuyo, 2010), langid.py (Lui and Baldwin, 2012), what- lang (Brown, 2014b,a), idNet (Dunn, 2020). Franc (Wormer, 2014), AfroLID (Adebara al., 2022), HeLI-OTS (Jauhiainen et al., 2022). transliterate', whatthelang whatlang-rs' lingua"0 Google/Bing Online, LanideNN (Koemi and Bo- jar, 2017), Paasaal, Q-LID (Ren et al., 2022), UDLDI (Goswami et al., 2020), PALI (Ahmadi et al., 2023), SS-LID (Caswell et al., 2020; Bapna et al., 2022; Kudugunta et al., 2023) and TextCat (Cavnar et al., 1994).
Transparency. We provide detailed descriptions of our methodology. model architecture, and evalu- ation process. Additionally, we make our research artifacts, including model, code, and list of data sources openly available to foster collaboration and reproducibility.
Acknowledgements
We would like to thank anonymous reviewers. This work was funded by the European Research Coun- cil (grant #740516).
References
Julien Abadji, Pedro Javier Ortiz Suárez, Laurent Ro- mary, and Benoît Sagot. 2021. Ungoliant: An opti- mized pipeline for the generation of a very large-scale multilingual web corpus. Proceedings of the Work- shop on Challenges in the Management of Large Corpora (CML.C-9) 2021. Limerick, 12 July 2021 (Online-Even Event), pages - 9, Mannheim. Leibniz- Institut für Deutsche Sprache.
'https://gitt htt.g//hbb..cc/bbreegegaaaattttt transliterats. ittrtte
Kathrein Abu Kwaik, Motaz Saad, Stergios Chatzikyr- iakidis, and Simon Dobuik. 2018. Shami: A cor- pus of Levantine Arabic dialects. Proceedings of the Fleventh International Conference on Language Resources and Evaluation (LREC 2018). Miyazaki, Japan. European Language Resources Association (ELRA).
"https://gi thub. com/ wtwp://wwww/ttttth/laaaacc
whatlang-rs
"https://gs http://gggtaaabccoo..eeoe wtt.b..com/mmmsttatl///// iinggg
"https://gt thub. www.hb...o...aaooommmmmm..a....e wttp://www/iiiiiii//o//mccc

---

Ifc Adebara and Muhammad Abdul-Mageed. 2022. To- wards afrocentric NLP for African languages: Where we are and where we can go. In Proceedings of the 60th Annual Meeting of the Association for Compu- lational Linguistics (Volume 1: Long Pupers), pages 3814-3841, Dublin, Ireland. Association for Compu- tational Linguistics.
and Evaluation (LREC 2018), Miyazaki, Japan. Eu- ropean Language Resources Association (ELRA).
Rosana Ardila, Megan Branson, Kelly Davis, Michael Kohler, Josh Meyer, Michael Henretty, Reuben Morais, Lindsay Saunders, Francis Tyers, and Gre- gor Weber. 2020. Common voice: A massively- multilingual speech corpus. In Proceedings of the Twelfth Language Resources and Evaluation Confer- ence, pages 4218 12-422 Marseille, France. European Language Resources Association.
Ife Adebara, AbdelRahim Elmadany, Muhammad Abdul-Mageed, and Alcides Inciarte. 2022. AfroLID: A neural language identification tool for African lan- guages. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 1958-1981, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
Naveen Arivazhagan, Ankur Bapna, Orhan Firat, Dmitry Lepikhin, Melvin Johnson, Maxim Krikun, Mia Xu Chen, Yuan Cao, George F. Foster, Colin Cherry, Wolfgang Macherey, Zhifeng Chen, and Yonghui Wu. 2019. Massively multilingual neural machine translation in the wild: Findings and chal- lenges. CoRR, abs//190.0080
David Ifeoluwa Adelani, Marek Masiak, Israel Abebe Azime, Jesujoba Oluwadara Alabi, Atnafu Lam- bebo Tonja, Christine Mwase. Odunayo Ogun- depo, Bonaventure F. P. Dossou, Akintunde Oladipo, Doreen Nixdorf, Chris Chinenye Emezue, Sana Sabah al azzawi, Blessing K. Sibanda, Davis David, Lolwethu Ndolela, Jonathan Mukiibi, Tunde Oluwaseyi Ajayi. Tatiana Moteu Ngoli, Brian Odhiambo. Abraham Toluwase Owodunni, Nnac- meka Obiefuna, Shamsuddeen Hassan Muham- mad, Saheed Salahudeen Abdullahi, Mesay Gemeda Yigezu, Tajuddeen Gwadabe, Idris Abdulmumin, Mahlet Taye Bame, Oluwabusayo Olufunke Awoy- omi, Iyanuoluwa Shode, Tolulope Anu Adelani, Habiba Abdulganiy Kailani, Abdul-Hakeent Omo- tayo, Adetola Adeeko, Afolabi Abeeb, Anuoluwapo Aremu, Olanrewaju Samuel, Clemencia Siro, Wan- gari Kimotho, Onyekachi Raphael Ogbu, Chinedu E. Mbonu, Chiamaka I. Chukwuneke, Samuel Fanijo. Jessica Ojo, Oyinkansola F. Awosan, Tadesse Kebede Guge, Sakayo Toadoum Sari, Pamela Nyatsine, Freedmore Sidume, Oreen Yousuf, Mardiyyah Odu- wole, Ussen Kimanuka, Kanda Patrick Tshinu, Thina Diko. Siyanda Nxakama. Abdulmejid Tuni Johar, Sinodos Gebre, Muhidin Mohamed, Shafie Abdi Mo- hamed, Fuad Mire Hassan, Moges Ahmed Mehamed, Evrard Ngabire, and Pontus Stenetorp. 2023. Masakhanews: News topic classification for african languages. ArXiv.
Ankur Bapna, Isaac Caswell. Julia Kreutzer, Orhan Fi- rat, Daan van Esch, Aditya Siddhant, Mengmeng Niu, Pallavi Baljekar, Xavier Garcia, Wolfgang Macherey, et al. 2022. Building machine translation systems for the next thousand languages. arXiv preprint arXiv:2205.0333 003903.
Loic Barrault, Magdalena Bicsialska, Ondrej Bo- jar, Marta R. Costa-jussà, Christian Federmann, Yvette Graham, Roman Grundkiewicz, Barry Had- dow. Matthias Huck, Eric Joanis, Tom Koemi, Philipp Koehn, Chi-kiu Lo, Nikola Ljubešić, Christof Monz, Makoto Morishita, Masaaki Nagata, Toshi- aki Nakazawa, Santanu Pal, Matt Post, and Marcos Zampieri. 2020. Findings of the 2020 conference on machine translation (WMT20). In Proceedings of the Fifth Conference on Machine Translation, pages 1-55, Online. Association for Computational Linguis- tics.
Loic Barrault, Ondrej Bojar, Marta R. Costa-jussä, Christian Federmann, Mark Fishel, Yvette Gra- ham, Barry Haddow, Matthias Huck, Philipp Koehn, Shervin Malmasi, Christof Monz, Mathias Müller. Santanu Pal. Matt Post, and Marcos Zampieri. 2019. Findings of the 2019 conference on machine trans- lation (WMT19). In Proceedings of the Fourth Con- ference on Machine Translation (Volume 2: Shared Task Papers, Day 1), pages 1-61, Florence, Italy. As- sociation for Computational Linguistics.
Gustavo Aguilar, Sudipta Kar, and Thamar Solorio. 2020. LinCE: A centralized benchmark for linguis- tic code-switching evaluation. In Proceedings of the Twelfth Language Resources and Evaluation Confer- ence, pages 1803-1813, Marseille, France. European Language Resources Association.
Verena Blaschke, Hinrich Schuetze, and Barbara Plank. 2023. A survey of corpora for Germanic low- resource languages and dialects. In Proceedings of the 24th Nordic Conference Linguistics (NoDaLiDa), pages 392-414, Tórshavn, Faroe Islands. University of Tartu Library.
Sina Ahmadi, Milind Agarwal, and Antonios Anasta- sopoulos. 2023. PALI: A language identification beuchmark for Perso-Arabic scripts. In Tenth Work- shop on NLP for Similar Languages, Varieties and Dialects ( (VarDial 2023), pages 78-90, Dubrovnik, Croatia. Association for Computational Linguistics.
Computational
Terra Blevins and Luke Zettlemoyer. 2022. Language contamination helps explains the cross-lingus -innsllingual capa- bilities of English pretrained models. In Proceedings of the 2022 Conference on Empirical Methods in Nat- ural Language Processing, pages 3563-3574, Abu Dhabi, United Arab Emirates. Association for Com- putational Linguistics.
Israa Alsarsour, Esraa Mohamed, Reem Suwaileh, and Tamer Elsayed. 2018 DART: A large dataset of di- alectal Arabic tweets. In Proceedings of the Eleventh International Conference on Language Resources

---

Ondfej Bojar, Christian Buck, Chris Callison- Gallison-Burch. Christian Federmann, Barry Haddow, Philipp Koehn, Christof Monz, Matt Post, Radu Soricut, and Lucia Specia. 2013. Findings of the 2013 Workshop on Statistical Machine Translation. In Proceedings of the Eighth Workshop Statistical Machine Truns- lation, pages 1-44, Sofia, Bulgaria. Association for Computational Linguistics.
feed-forward networks. In Proceedings of the 2017 Conference on Empirical Methods in Natural Lan- guage Processing, pages 2879-2885, Copenhagen, Denmark. Association for Computational Linguis- tics.
Houda Bouamor, Sabit Hassan, and Nizar Habash. 2019. The MADAR shared task on Arabic fine-grained di- alect identification. In Proceedings of the Fourth Ara- bic Natural Language Processing Workshop. pages 199-207, Florence, Italy. Association for Computa- tional Linguisties.
Ondiej Bojar, Christian Buck, Christian Federmann, Barry Haddow, Philipp Koehn, Johannes Leveling, Christof Monz. Pavel Pecina, Matt Post, Herve Saint- Amand, Radu Soricut, Lucia Specia, and Ales Tam- chyna, 2014. Findings of the 2014 workshop on statistical machine translation. In Proceedings of the Ninth Workshop on Statistical Machine Translation, pages 12-58, Baltimore, Maryland, USA. Associa- tion for Computational Linguistics.
Ralf Brown. 2014a. Language- aware string extractor.
Ralf Brown. 2014b. Non-linear mapping for improved identification 1300+ languages. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 627- 632, Doha, Qatar. Association for Computational Linguistics.
Ondfej Bojar, Rajen Chatterjee, Christian Federmann. Yvette Graham, Barry Haddow, Shujian Huang, Matthias Huck, Philipp Koehn, Qun Liu, Varvara Logacheva, Christof Monz, Mattco Negri, Matt Post, Raphael Rubino, Lucia Specia, and Marco Turchi. 2017. Findings of the 2017 conference machine translation (WMT17). Proceedings of the Second Conference on Machine Translation, pages 169-214, Copenhagen, Denmark. Association for Computa- tional Linguisties.
Ralf D Brown. 2012. Finding and identifying text in 900+ languages. Digital Investigation, 9:S34 S43,
Laurie Burchell, Alexandra Birch, Nikolay Bogoychev, and Kenneth Heafield. 2023. An open dataset and model for language identification. of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Popers), pages 665-879, Toronto, Canada. Association for Computational Linguisties.
Proceedings
Ondfej Bojar, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Matthias Huck, An- tonio Jimeno Yepes, Philipp Koehn, Varvara Lo- gacheva, Christof Monz, Matteo Negri, Aurélie Névéol, Mariana Neves, Martin Popel, Matt Post, Raphael Rubino, Carolina Scarton, Lucia Specia, Marco Turchi, Karin Verspoor, and Marcos Zampieri. 2016. Findings of the 2016 conference on machine translation. In Proceedings of the First Conference on Machine Translation: Volume 2, Shared Task Pa- pers, pages 131-198, Berlin, Germany. Association for Computational I Linguistics.
Isaac Caswell, Theresa Breiner, Daan van Esch, and Ankur Bapna. 2020. Language ID in the wild: Unex- pected challenges on the path to a thousand-languae, web text corpus. In Proceedings of the 28th Inter- national Conference on Computational Linguistics. pages 6588 6608, Barcelona, Spain (Online). Inter- national Committee on Computational Linguistics.
William B Cavnar, John M Trenkle, et al. 1994. N-gram- based text categorization. In Proceedings of SDAIR- 94, 3rd annual symposium document analysis and information retrieval, volume 161175. page 14. Las Vegas, NV.
Ondřej Bojar, Rajen Chatterjee, Christian Federmann, Barry Haddow, Matthias Huck, Chris Hokamp, Philipp Koehn, Varvara Logacheva, Christof Monz, Mattco Negri, Matt Post, Carolina Scarton, Lucia Specia, and Marco Turchi. 2015. Findings of the 2015 workshop on statistical machine translation. In Proceedings of the Tenth Workshop Statistical Machine Translation, pages 1-46, Lisbon, Portugal. Association for Computational Linguistics.
Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Oit, Luke Zettle- moyer, and Veselin Stoyanov. 2020. Unsupervised cross- lingull representation learning scale. In Pro- ceedings of the 58th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 8440- 8451, Online. Association for Computational Lin- guistics.
Ondiej Bojar, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow, Mathhia Huck, Philipp Koehn, and Christof Monz. 2018. Findings of the 2018 conference on machine translation (WMT18). In Proceedings of the Third Conference Machine Translation: Shared Task Papers. pages 272-303, Belgium, Brussels. Association for Computational Linguistics.
Morris H DeGroot and Stephen E Fienberg, 1983. The comparison and evaluation of forecasters. Journal of the Royal Statistical Society: Series D (The Statisti- cian), 32(2
Jan A. Botha, Emily Pitler, Ji Ma, Anton Bakalov, Alex Salcianu, David Weiss, Ryan McDonald, and Slav Petrov. 2017. Natural language processing with small
Jonathan Dunn. 2020. Mapping languages: The corpus of global language use. Language Resources and Evaluation, 54:999-1018.

---

Mahmoud El-Haj. Paul Rayson, and Mariam Aboelezz. 2018. Arabic dialect identification in the context of bivalency and code-switching. In Proceedings of the Eleventh International Conference on Language Rexources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Association (ELRA).
Proceedings of the 14th Workshop on Building and Using Comparable Corpora (BUCC 2021). pages 60-67, Online (Virtual Mode). INCOMA Ltd.
Ayyoob ImaniGooghari, Peiqin Lin, Amir Hossein Kar- garan, Silvia Severini, Masoud Jalili Sabet, Nora Kassner, Chunlan Ma, Helmut Schmid, André Mar- tins, François Yvon, and Hinrich Schüizze 2023. Glot500: Scaling multilingual corpora and language models to 500 languages. In Proceedings of the 6 Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1082- 1117. Toronto, Canada. Association for Computa- tional Linguisties.
Dirk Goldhahn, Thomas Eckart, and Uwe Quasthoff. 2012. Building large monolingual dictionaries at the Leipzig corpora collection: From 100 to 200 lan- guages. In Proceedings of the Eighth International Conference on Language Resources and Evaluation 12), pages 759-765, Istanbul, Turkey, Euro- pean Language Resources Association (ELRA).
Abderrahmane Issam and Khalil Mrini. 2022. Goud.ma: news article dataset for summarization in moroccan darija. In 3rd Workshop on African Natural Lan- guage Processing.
Santiago Góngora, Nicolás Giossa, and Luis Chiruzzo. 2022. Can we use word embeddings for enhancing Guarani-Spanis machine translation? In Proceed- ings of the Fifth Workshop on the Use of Compu- kational Methods in the Study of Endangered Lan- guages, pages 127-132, Dublin, Ireland. Association for Computational Linguistics.
Heidi Jauhiainen, Tommi Jauhiainen, and Krister Lin- den. 2019a. Wanca in korp: Text corpora for un- derresourced uralic languages. In Proceedings of the Research data and humanities (RDHUM) 2019 conference. University of Oulu.
Koustava Goswami, Rajdeep Sarkar, Bharathi Raja Chakravarthi, Theodorus Fransen, and John P. Mc- Crac. 2020, Unsupervised deep language and di- alect identification for short texts. In Proceedings of the 28th International Conference on Computational Linguistics, pages 1606-1617, Barcelona, Spain (On- line). International Committee on Computational Lin- guistics.
Tommi Jauhiainen, Heidi Jauhiainen, and Krister Lindén. 2022. HeLI-OTS, off-the-shelf language identifier for text. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 3912 -9222, Marseille, France. European Lan- guage Resources Association.
Tommi Jauhiainen, Marco Lui, Marcos Zampieri, Timo- thy Baldwin, and Krister Lindén. 2019b. Automatic language identification in texts: survey, Journal of Artificial Intelligence Research, 65:675-782.
Thamme Gowda, Zhao Zhang, Chris Mattmann, and Jonathan May. 2021. Many-to-English machine translation tools, data, and pretrained models. In Proceedings of the 59th Annual Meeting of the Asso- ciation for Computational Linguistics and the 11th International Joint Conference Natural Language Processing: System Demonstrations, pages 306-316. Online. Association for Computational Linguistics.
Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. 2017. Bag of tricks for efficient text classification. In Proceedings of the 15th Con- ference of the European Chapter of the Association for Computational I Linguisticx: Volume 2, Short Pa- pers, pages 427-431, Valencia, Spain. Association for Computational Linguistics.
Hendrik J Groenewald and Liza du Plooy. 2010. Pro- cessing parallel text corpora for three south african language pairs in the autshumato project. AflaT 2010, page 27.
David Jurgens, Yulia Tsvetkov, and Dan Jurafsky, 2017. Incorporating dialectal variability for socially equi- table language identification. In Proceedings of the 55th Anmual Meeting of the Association for Compu- tational Linguisties (Volume 2: Short Papers), pages 51-57, Vancouver, Canada. Association for Compu- tational Linguistics.
René Haas and Leon Derczynski. 2021. Discriminating between similar Nordic languages. In Proceedings of the Eighth Workshop on NLP for Similar Languages. Varieties and Dialects, pages 67-75, Kiyy. Ukraine. Association for Computational Linguisties.
Tahmid Hasan, Abhik Bhattacharjee, Md. Saiful Is- lam, Kazi Mubasshir, Yuan- Li, Yong-Bin Bing-Bin&o Kang, M. Sohel Rahman, and Rifat Shahriyar. 2021. XL- sum: Large-scale multilingual abstractive summariza- tion for 44 languages, In Findings of the Association for Computational Linguistics: ACI-IJCNIP 2021, pages 4693-4703, Online. Association for Computa- tional Linguistics.
Amir Hossein Kargaran, François Yvon, and Hinrich Schütze. 2023. Glotscript: A resource and tool for low resource writing system identification. arXiv preprint arX
Omid Kashefi. 2018. Mizan: A large persian-english parallel corpus. arXiv preprint 1 007
Tom Koemi and Ondiej Bojar. 2017. LanideNN: Multi- lingual language identification on character window. In Proceedings of the 15th Conference of the Euro- pean Chapter of the Association for Computational
Rudali Huidrom, Yves Lepage, and Khogendra Khom- dram. 2021. EM corpus: a comparable corpus for a less-resourced language pair Manipuri-Engliist In

---

Linguisties: Volume 1, Long Papers, pages 927-936, Valencia, Spain. Association for Computational Lin- guistics.
Resources and Evaluation (LREC"14), pages 3158- 3163, Reykjavik, Iceland. European Language Re- sources Association (ELRA).
Julia Kreutzer. Isaac Caswell, Lisa Wang, Ahsan Wahab, Daan van Esch, Nasanbayar Ulzii-rshikh, Allah- sera Tapo, Nishant Subramani, Artem Sokolov, Clay- tone Sikasote, Monang Setyawan, Supheakmungkol Sarin, Sokhar Samb, Benoît Sagot, Clara Rivera, An- nette Rios, Isabel Papadimitriou, Salomey Osei, Pe- dro Ortiz Suarez, Iroro Orife, Kelechi Ogueji. An- dre Niyongabo Rubungo, Toan Q. Nguyen, Math- ias Müller, André Müller, Shamsuddeen Hassan Muhammad, Nanda Muhammad, Ayanda Mnyak- eni, Jamshidbek Mirzakhalov, Tapiwanashe Matan- gira, Colin Leong, Nze Lawson, Sneha Kudugunta, Yacine Jernite, Mathias Jenny, Orhan Firat, Bonaven- ture F. P. Dossou. Sakhile Dlamini, Nisansa de Silva, Sakine Çabuk Balli, Stella Biderman, Alessia Bat- tisti, Ahmed Baruwa, Ankur Bapna, Pallavi Baljekar. Israel Abebe Azime, Ayodele Awokoya, Duygu Ata- man, Orevaoghene Ahia, Oghenefego Ahia, Sweta Agrawal, and Mofetoluwa Adeyemi. 2022. Quality at glance: An audit of web-crawled multilingual datasets. Transactions of the Association for Compu- tational Linguistics, 10:50-72.
Michael McCandless. 2010. Accuracy and performance of google's compact language detector. Blog post.
Salima Medhaffar, Fethi Bougares, Yannick Estève, and Lamia
2017. Sentiment analy- sis of Tunisian dialects: Linguistic ressources and experiments. In Proceedings of the Third Arabic Natural Language Processing Workshop, pages 55- 61, Valncia,, Spain. Association for Computational Linguistics.
Karima Meftouh, Salima Harrat, Salma Jamoussi, Mourad Abbas, and Kamel Smaili. 2015. Machine translation experiments on PADIC: A parallel Arabic Dialect corpus. In Proceedings of the 29th Pacific Asia Conference on Language, Information and Com- putation, pages 26-34, Shanghai, China.
Jamshidbek Mirzakhalov, Anoop Babu, Duygu Ataman, Sherzod Kariev, Francis Tyers, Otabek Abduraufov, Mammad Hajili, Sardana Ivanova, Abror Khaytbaev, Antonio Laverghetta Jr., Bekhzodbek Moydinboyev, Esra Onal, Shaxnoza Pulatova, Ahsan Wahab, Orhan Firat, and Sriram Chellappan. 2021. A large-scale study of machine translation in Turkic languages. In Proceedings of the 2021 Conference on Empiri- cal Methods in Natural Language Processing, pages 9876-5890, Online and Punta Cana, Dominican Re- public. Association for Computational Linguistics.
Sneha Kudugunta, Isaac Caswell, Biao Zhang, Xavier Garcia, Christopher A Choquette- Katherine Lec, Derrick Xin. Aditya Kusupati. Romi Stella, Ankur Bapna, al. 2023. Madlad-400: A multilin- gual and document-level large audited dataset. arXiv preprint ar 2r39 046602
Anoop Kunchukuttan, Pratik Mehta, and Pushpak Bhat- tacharyya. 2018. The IIT Bombay English-Hindi parallel corpus. In Proceedings of the Fleventh In- ternational Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Association (ELRA).
Alexandru Niculescu-Mizil and Rich Caruana. 2005. Predicting good probabilities with supervised learn- ing. In Proceedings of the 22nd international confer- ence on Machine learning, pages 625-632.
Joakim Nivre, ter, Jan Hajič, Christopher D. Manning, Sampo Pyysalo, Sebastian Schuster, Francis Tyers, and Daniel Zeman. 2020. Universal Dependencies v2: An evergrowing multilingual treebank collection. In Proceedings of the Twelfth Language Resources and Evaluation Conference. pages 4034-4043, Marseille, France. European Language Resources Association.
de Marneffe, Filip Gin-
Richard Lastrucci, Jenalea Rajab, Matimba Shingange, Daniel Njini, and Vukosi Marivate. 2023. Prepar- ing the vuk' uzenzele and ZA muugl--uulill South African multilingual corpora. In Proceedings of the Fourth workshop on Resources for African In- digenous Languages (RAII. 2023), pages 18-25, Dubrovnik, Croatia. Association for Computational Linguistics.
NLLB Team, Marta R. Costa-jusså, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Hef- fernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Bar- rault, Gabriel Mejia Gonzalez, Prangthip Hansanti, John Hoffman, Semarley Jarreu, Kaushik Ram Sadagopan, Dirk Rowe, Shannon Spruit, Chau Tran, Pierre Andrews, Necip Fazil Ayan. Shruti Bhosale. Sergey Edunov, Angela Fan, Cynthia Gao, Vedanuj Goswami, Francisco Guzmán, Philipp Koehn, Alexandre Mourachko, Christophe Rop- ers, Safiyyah Saleem, Holger Schwenk, and Jeff Wang. 2022. No language left behind: Scaling human-centered machine translation. arXiv preprint arXiv:2207.04677
Marco Lui and Timothy Baldwin. 2012. langid.py: An off shelf language identification tool. In Proceed- ings of the ACL 2012 System Demonstrations, pages 25-30, Jeju Island, Korca, Association for Computa- tional Linguistics.
Kang Kwong Luke and May LY Wong. 2015. The hong kong cantonese corpus: design and uses. Journal of Chinese Linguistics Monograph Series, 1(25):312- 333.
Thomas Mayer and Michael Cysouw. 2014. Creating a massively parallel Bible corpus. In Proceedings of the Ninth International Conference on Language

---

Odunayo Ogundepo. Tajuddeen Gwadabe, Clara Rivera, Jonathan H. Clark, Sebastian Ruder, David Ifeoluwa Adelani, Bonaventure F. P. Dos- sou, Abdou Aziz DIOP, Claytone Sikasote, Gilles Hacheme, Happy Buzaaba, Ignatius Ezeani, Roowei- ther Mabuya, Salomey Osei, Chris Emezue, Al- hert Njoroge Kahira, Shamsuddeen H. Muham- mad. Akintunde Oladipo, Abraham Toluwase Owodunni, Atnafu Lambebo Tonja, Iyanuoluwa Shode. Akari Asai, Tunde Oluwaseyi Ajayi. Clemen- cia Siro, Steven Arthur, Mofetoluwa Adeyemi, Orevaoghene Ahia, Aremu Anuoluwapo, Oyinkan- sola Awosan, Chiamaka Chukwuneke, Bernard Opoku, Awokoya Ayodele, Verrah Otiende, Chris- tine Mwase, Boyd Sinkala, Andre Niyongabo Rubungo, Daniel A. Ajisafe, Emeka Felix Onwueg- buzia, Habib Mbow, Emile Niyomutabazi. Eunice Mukonde, Falalu Ibrahim Lawan, Ibrahim Said Ah- mad, Jesujoba O. Alabi, Martin Namukombo, Mhonu Chinedu, Mofya Phiri, Neo Putini, Ndumiso Mn- goma, Priscilla A. Amuok, Ruqayya Nasir Iro, and Sonia Adhiambo. 2023. Afriqa: Cross-lingual open- retrieval question answering for african languages.
Roberts Rozis and Raivis Skadinš. 2017. Tilde MODEL multilingual open data for EU languages. In Pro- ceedings of the 2/ Nordic Conference on Computa- rional Linguistics, pages 263-265, Gothenburg, Swe- den. Association for Computational Linguistics.
Alex Salcianu, Andy Golding, Anton Bakalov, Chris Alberti, Daniel Andor, David Weiss, Emily Pitler, Greg Coppola, Jason Riesa, Kuzman Ganchev, et al. 2018. Compact language detector v3.
Nakatani Shuyo. 2010. Language detection library for java.
Kathleen Siminyu, Godson Kalipe, Davor Orlic, Jade Abbott, Vukosi Marivate, Sackey Freshia, Prateek Sibal, Bhanu Neupane, David Adelani, Amelia Tay- lor, et al. 2021. Ai4d-african language program. arXiv preprint arXiv:210400251
Martin Thoma 2018. The wili benchmark dataset for written language identification. arXiv preprint arXiv:1801.07779.
Jörg Tiedemann. 2012. Parallel data, tools and inter- faces in OPUS. In Proceedings of the Fighth In- ternational Conference on Language Resources and Evaluation (2) pages 2214-2218, Istanbul, Turkey. European Language Resources Association (ELRA).
Atul Kr Ojha. 2019. Einglih-bhoojurii smt system: Insights from the karaka model. arXiv preprint WwXiv:9055002239.
Pedro Javier Ortiz Suárez, Benoît Sagot, and Laurent Romary. 2019. Asynchronous pipelines for process- ing huge corpora medium to low resource infras- tructures. Proceedings of the Workshop on Chal- lenges in the Management of Large Corpora (CMLC- 7) 2019. Cardiff, 22nd July 2019, pages 9 16, Mannheim. Leibniz-Institu Institut für Deutsche Sprache.
Daan van Esch, Tamar Lucassen, Sebastian Ruder, Isaac Caswell, and Clara Rivera. 2022. Writing system and speaker metadata for 2,800+ language varieties. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 5035-5046, Mar- seille, France. European Language Resources Asso- ciation.
Mohammad Taher Pilevar, Heshaam Faili, and Ab- dol Hamid Pilevar. 2011. Tep: Tehran english- persian parallel corpus. In International Conference on Intelligent Text Processing and Computational Linguistics, pages 68-79. Springer.
Guillaume Wenzek, Marie-Anne Lachaux, Alexis Con- neau, Vishrav Chaudhary, Francisco Guzmán, Ar- mand Joulin, and Edouard Grave. 2020. CCNet: Extracting high quality monolingual datasets from web crawl data. In Proceedings of the Tweifihh Lan- guage Resources and Evaluation Conference, pages 4003-4012, Marseille, France. European Language Resources Association.
Matt Post, Chris Callison- Burch, and Miles Osborne. 2012. Constructing parallel corpora for six Indian languages via crowdsourcing. In Proceedings of the Seventh Workshop on Statistical Machine Translation, pages 401-40 401-409, Montréal, Canada. Association for Computational Linguistics.
Titus Wormer. 2014. Franc library.
Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021. mT5: A massively multilingual pre-trained text-to-text transformer. In Proceedings of the 2021 Conference of the North American Chap- of the Association for Computational Linguistics: Human Language Technologies, pages 483-498, On- line. Association for Computational Linguistics.
Ye Qi, Devendra Sachan, Matthieu Felix, Sarguna Pad- manabhan, and Graham Neubig. 2018. When and why are pre-trained word embeddings useful for neu- ral machine translation? In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 529-535, New Orleans, Louisiana. Associa- tion for Computational Linguistics.
Lisa Yankovskaya, Maali Tars, Andre Tättar, and Mark Fishel. 2023. Machine translation for low-resource Finno-Ugrie languages. In Proceedings of the 24th Nordic Conference Computational Linguistics (NoDuLiDa). pages 762-771, Tórshavn, Faroe Is- lands. University of Tartu Library.
Xingzhang Ren, Baosong Yang, Dayiheng Liu, Haibo Zhang, Xiaoyu Lv, Liang Yao, and Jun Xie. 2022. Effective approaches to neural query language identi- fication. Computational Linguistics, 48(4):887-906

---

Jihad Zahir. 2022. ladd: An integrated arabic dialect identification dataset. Data in Brief, 40:107777.
Omar F. Zaidan and Chris Dallison-Burch. 2011. The Arabic online commentary dataset: annotated dataset of informal Arabic with high dialectal con- tent. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Nu- man Language Technologies, pages 37-41, Portland, Oregon, USA. Association for Computational Lin- guistics.
Biao Zhang, Philip Williams, Ivan Titov, and Rico Sen- nrich. 2020. Improving massively multilingual neu- ral machine translation and zero-shot translation. In Proceedings of the 58th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 1628- 1639, Online. Association for Computational Linguis- tics.
Michat Ziemski, Marcin Dowzzys, and Bruno Pouliquen. 2016. The United Nations parallel cor- pus v1.0. In Proceedings of the Tenth International Conference Language Resources and Evaluation (LREC'16), pages 3530-3534, Portorož, Slovenia. European Language Resources Association (ELRA).

---

A List of data sources
smugri (Yankovskaya al., 2023), finno-
ugric (Yankovskaya al., 2023)
Wikipedia articles: Wikipedia dumps," 12
Trasnlation: GlotStoryBook which is
WiLI-2018 (Thoma, 2018), Leipzig corpora- wikipedia split (Goldhahn et al., 2012) News: BBC News (Hasan et al., 2021), Global Voices (Tiedemann, 2012), Leipzig corpora-news split (Goldhahn al., 2012),
a collection of children storybooks
174 languages from Global Storybooks", AfriQA Ogundepo et al. (2023), smugri-
flores (Yankovskaya et al., 2023)
Multiple domain: Universal Dependencies v2.12 (Nivre al., 2020), Abkhaz National Corpus
SETIMES (Tiedemann, 2012)
Translation: NLLB Seed (NLLB Team et al.,
2022)
Lyries: lyricstranslate"
Religious: PBC (Mayer and Cysouw, 2014),
Government domain: Vuk uzenzele (Las- trucci et al., 2023)
Jehovah's Witnesses," 13 1000Langs
Crowdsourcing: Tatoeha
Multiple domain: MT-560 (Gowda et al., 2021; Tiedemann, 2012; Burchell al., 2023;
Specifically, GlotSparse and GlotStory Book are two corpora that compiled a side of this project to include more languages and domains for LID.
Post al., 2012: Ziemski et al., 2016; Rozis and Skadinš, 2017; Kunchukuttan al., 2018;
Qi et al., 2018; Zhang al., 2020; Bojar et al., 2013, 2014, 2015, 2016, 2017, 2018; Barrault et al., 2019, 2020), LTI (Brown, 2012), Arabic (Zahir, 2022; Alsarsour et al., 2018; Abu Kwaik et al., 2018; Medhaffar et al., 2017; Meftouh et al., 2015; Zaidan and Callison-Buree 2011; I; El-Haj al., 2018; Bouamor et al., 2019), Persian (Pilevar al.. 2011; Kashefi, 2018). Turkic (Mirzakhalov et al., 2021), Bhojpuri (Ojha, 2019), Can- tonese (Luke and Wong, 2015), Guaraní (Gón- gora et al.. 2022), Manipuri (Huidrom et al.. 2021)
B GlotLID-M hyperparameters
We provide the hyperparameters used to train the GlotLID-M in Table 4.
argument
description
value
-mineoont
minimal number word occurrences
1000
-mioCountabel
number label occurrences
Ngramms
length of word ngram
-bucket
number of buckets
le6
-min
min longth of char ngram
2
maxn
max length of char ngram functioo word number of epochs
5
Joss
softmax
dim
256
epoch
2
Government domain: Autshumato (Groe- newald and du Plooy, 2010)
-Ir
learning rate
.8
Table 4: GlotLID-M training hyperparameters
We also introduce additional data sources suited for LID, but they are not included in the training of the version of GD-MLID-M discussed in the paper:
Evaluation data issues
Crowdsourcing: Common Voice vl1 (Ardila et al., 2020)
C.1 FLORES-200
There are some mistakes in the FLORES-200 dataset which have been raised by the community. For example, in GitHub issue, it is pointed out that yue_ Hant and zho_Hantt should actually be very easy to distinguish from each other, and the Cantonese (Yue Chinese, yue_Hand data FLORES-200 is completely wrong.
Web: Wanca 2016 (Jauhiainen et al.. 2019a)
News: GlotSparse 16 which is collec- tion of news websites in low-resource lan- guages, MasakhaNEWS (Adelani et al., 2023), Goud.ma (Issam and Mrini, 2022), AI4D Siminyu et al. (2021), Radio Ramogi,""
Phttps://dumps. http://umpp...iiimm..e.e.m www.wiiii..i...mmmm
Phttps://www. jw..g///.or////
h https/thuw..cmm/.aa//// G GlotstcryBook
#https://// thub. http:///////gthhab.aoceeeeeeeeee htt..c/////ssaaarr.....
1000Langs
httttps://gtthub..comgg.... http:ub..cc/////aaaa
"nttps::/clarino. uib a
"nttps://githabb.cm
"nttps://g/ thub. http://gttthbb.cmom//aaaaa/
Rattesss//gtthbb.cc///a/aaoooore Flores/issues//
Luo-News-Dataset

---

In another issuc??, it is mentioned that the Cen- tral Atlas Tamazight (tzm) actually in Standard Moroccan Tamazight (zgh), as confirmed by a na- tive speaker of Central Atlas Tamazight.
639-3 codes to. For example, Table 30 indicates that we map ISO 639-3 code fas pes/prs in FLO- RES. In other words, to evaluate our performance for the language fas FLORES, we (only concep- tually) create a new test set in which all sentences labeled as pes prs in FLORES, are relabeled as fas.
C.2 UDHR
There are some mistakes with UDHR. For example, both ckb and kmr files the same. ckb is known for the Arabic script, although it can also be written in Latin. There are also some files that the writing system not popular use (based on Kargaran et al. (2023) metadata):
F
Calibration
As stated in $2, an LID model should provide a cal- ibrated confidence measure in addition its predic- tion. Reliability diagrams illustrate model calibra- tion (DeGroot and Fienberg, 1983; Niculescu- Mizil and Caruana, 2005). These diagrams use expected sample accuracy as a function of confidence. If the model is perfectly calibrated, then the diagram plots the identity function.
ckb Latn (Arabic script is in use.)
azb Latn (Arabic script is in use.)
khk Mong (Cyrillic script in use.)
vie Hami (Latin script is in use.)
We provide the reliability diagram for GloLLID-M on GloLLID-C test in Figure 2. For GlotLID-C test, the plot nearly close to the identity function. However, for some the low confidence scores, it's not calibrated. This mostly happens because we included so many languages in our models, and some of these languages are very similar to each other have small training sizes.
D Performance of GlotLID-M per
language
The list of languages used to train GlotLID-M. along with the corresponding amount of available data and detailed results for each language, can be found in Tables 5-29
E Language metadata matching
The per-language comparison between GlotLID-M and the baselines (CLD3, FT176, OpenLII, and NLLB) for each benchmark in scenario is as fol- lows:
Roliabilty Diagram
1.
0.
FLORES-200. (i) GlotLID-M vs CLD3: Tables 30-31 (ii) GlotLID-M vs FT176: Tables 32- 33 (iii) GlotLID-M vs OpenLID: Tables 34-35 (iv)
GlotLID-M vs NLLB: Tables 36-37
Porfect Colltation
UDHR. (i) GlotLID-M vs CLD3: Tables 38-41
Model Cal bratin
Average Adox racy
(ii GlotLID-M vs FT176: Tables 42-45 (iii)
6.2
0.6
1.0
Conticence
GlotLID-M OpenLID: Tables 46-47 (iv)
GlotLID-M vs NLLB: Tables 48-49
Figure Reliability diagram for GlotLID-M on
GlotLID-C test
The underlined results each table show the best result for each model, and the bold result indi- cates the overall best result.
G Analysis of Baseline Results
The tables also contain the metadata matching rules define. Column "isocode639-3" contains the ISO 639-3 code of each language. This corre- sponds the class used by GlotLID-M (since all our classes are ISO 639-3 codes). The following columns contain the codes that we mapped the ISO
In this section, we analyze baseline results sum- marized in Table 2. For detailed breakdown of results for each language in scenario SET?, see $E. CLD3: The largest performance gap between SET? and SET! is for CLD3. This could be attributed to the different architecture CLD3, which performs better when the set is known. Also, since CLD3 supports fewer languages compared
23 Phttps: //g! thub. com/ acceoook research/ flores/ /issussssss/3

---

other models, has an advantage in terms of supporting higher percentage of high-resource languages within its base set.
CLD3 achieves a lower FI score than the other FastText-based models the UDHR benchmark in scenario SET?. However, in scenario SET! for the UDHR, it outperforms all other FastTTxttbbsedd models on Fl. Additionally, the good performance achieved in both benchmarks (.952 and .922) in scenario SET! illustrates the robustness of this LID. FT176. When comparing FT176 and GloLLID-M FLORES- 200, GloLLID-M achieves the highest F1 scores in FLORES. This may be attributed to the fact that all languages in FT176 are supported by Wikipedia, and GlotLID-M has strong support for these languages. On the other hand, FT176 has the worst overall performance among the FastText models.
NLLB and OpenLID. In FLORES-200 Open- LID and NLLB have an advantage scenario SET?, as this scenario closely aligns with SET! Both models provide near-complete support for the languages available in FLORES-200 For the rest of the models and benchmarks, GlotLID-M displays marked difference in performance and takes the lead. Among the baseline models in sce- nario SET?, OpenLID shows the best performance in both benchmarks. We will now investigate which languages OpenLID performs better or worse in compared to GlotLID-M.
In scenario SET?, when comparing OpenLID and GlotLIID-M on FLORES-200 200,, most of the time the per language scores are very close to each other (see Tables 34-35). 35). However, there cases where OpenLID performs noticeably better, for exam- plc, with a .39 improvement for azb and .29 for awa. On the other hand, GlotLID-M performs bet- ter by for zho, which can be attributed to the poorer performance of OpenLID in zho_Hant Ad- ditionally, both models perform poor on languages such as Yue Chinese, which could be attributed to issue with FLORES-200 200 (see SC.1). How- ever, this situation is quite different for UDHR, as GlotLID-M supports more languages than Open- LID, GlotLID-M performs much better in handling languages that are outside the intersection of both models' base sets.

---

GloLLID-C
FLORES-2000
UDIR
iso639.3
Language
Seetteness
FIf
FPR.
FI1
FPR
FI
FPR
ani
Arifama- Miniaaaa
7954
0.9995
0.0
aak
Ankave
7784
1.0
sau
Abau
7888
1.0
0.0
aaz
Amarasi
9071
0.999
aba
Abé
28725
0.81648
0.00371
0.00416
abk
Abkhazian
28
abu
Abua
61385
0.94699
0.00091
abq
Abaza
48
1.0
0.0
abi
Amnulas
15129
0.999
6c-05
abx
Inabuknon
7855
0.999
3c-05
aby
Aneme Wake
7070
0,998
3e-05
0.00011 I
abz
Abui
16642
0,98171
6c-05
Achagua
4408
0.99929
0.0
acd
Gikyode
7884
1.0
0.0
ace
Achinese
31228
0.98176
0.00014
0.95503
0.00579
0,90909
0.0012
acf
Saint Lucian Creole French
21811
0.97035
0,00019
ach
Acali
42735
0.995
0,00017
0.00044
Mesopotamian Arabic
4902
0.79445
0.00091
0.01562
0.00023
acn
Achang
7919
0.9995
0.0
0.00099
acq
Ta Aizzii-deni Arabie Achi
1598
0.60947
0,00058
0.00197
6c-05
acr
27176
0,996
0.00011
0.00044
acu
Achuar-Shiwiar
11509
0,999
0.0
0.584
0.00011
ada
Adangmme
217601
0,998
8c-05
0,91045
0.00131
ade
Adele
7924
0.9985
6c-05
adh
Adhola
8975
0.99549
6e-05
0.00055
adi
Adi
30717
0.9945
0.00017
0.00022
adj
Adioukrou
7883
0,998
0.0
adl
Galo
7956
0.998
3c-05
ady
Adyghe
4885
0.99596
0,0
0.83495
0.0
adz
Adzera
1489
0.99541
0.0
aeb
Tunisian Arabic
26935
0.86433
0.00231
0.28501
0.00199
acr
Eastern Arrennte
9577
0.9995
0,0
0.00011 I
acu
Akeu
7853
1.0
0.0
0.00504
aey
Amele
9061
0.9995
afb
Gulf Arabic
136
0.2
3e-05
afh
Afrihili
79
0.75862
3e-05
afr
Afrikaans
1436086
0,98216
0.00074
1.0
0.0
0.95238
0.00066
agd
Agarabi
7917
1.0
0.0
agg
Angor
7788
1.0
agm
Angaataha
7889
0.9995
0.0
agn
Agutaynen
7844
0.9985
6c-05
agr
Aguaruna
23895
0.93626
0.0005
0,81429
0.00033
agt
Central Cagayan Agta
7554
0.9985
3c-05
agu
Aguacateco
7928
0.9985
0.0
agw
Kahua
35771
0.998
0.00011
agx
Aghul
1150
0.97898
0
aha
Ahanta
18467
0.999
0.0
ahk
Akha
134957
1.0
0.0
aia
Arosi
7804
0.9995
0.0
Assyrian Nco- Aramaic
10736
1.0
aim
Aimol
7949
0.9975
3e-05
0.00022
ain
Ainu (Japan)
324
0,91111
3e-05
0.00022
aig
Aja (Benin)
35237
0.99245
3c-05
0.00028
0.64516
0.00722
aja
Ajië
9916
0
3e-05
0.00175
aip
South Levantine Arabic
28203
0,75111
0.00341
0.10836
0.00102
ajz
Amri Karbi
7956
0.999
3e-05
0.00033
aka
Akan
1174
0.95114
0.0
0. 0.99852
0.0
akb
Batak Angkola
7940
0.98993
8c-05
ake
Akawaio
7933
1.0
0,0
akh
Angal Heneng
7756
0.9995
3e-05
akl
Aklanon
28
akp
Siwu
7919
0.9985
3e-05
ald
Alladian
7939
1.0
alj
Alangan
7877
0.999
6e-05
0.0001 1
aln
Gheg Albanian
71977
0.98587
0.00014
6c-05
alp
Alume
7829
0.999
0.00011
slq
Algonquin
8025
0.9995
0.0
als
Tosk Albunian
403221
0.97813
0.00077
0.99852
0.00011
0.85507
0.00208
Southern
89192
0.99701
0.00017
0,95495
0.00033
aly
Alyawarr
7411
1.0
0.0
alz
Alur
103655
0.99651
0.00014
Yanesha"
7697
0.9995
1.0
0.0
and
Hamer- Banna
7808
0.998
3e-05
0.0012
amh
Amharic
682875
0.99206
0.00044
0.99951
6c-05
1.0
0.0
ami
Amis
104294
0.999
3c-05
0.26087
0.0
Table 5: Performance of GlotLID-M on GlotLID-C test, FLORES-200 and UDHR benchmarks (part

---

GloLLID-C
FLORES- 200
UDIR
1s0639.3
Language
Sentences
F11
FPR]
FT
FPR
FT
FPR.
amk
Ambui
7850
0.9995
3e-05
amm
Ama (Papua New Guines)
7826
1,0
0.0
Amanuh
14975
0.9995
3e-05
0.00011
amp
Alamblak
7720
0.9985
36-05
amr
Amarakaeri
7671
0.0
amu
Guerrero Amuzgo
8056
0,999
0.0
0,00022
amx
Anmaryerre
1949
1.0
ang
Old English (ca. 450-1100)
379
0.88889
3e-05
anm
Anal
30940
0.99499
8c-05
Obolo
7948
0.9995
anv
Denya
7928
0.9995
3--05
any
Anyin
7949
0,999
0.0
aoc
Pemon
98
1.0
0.0
0,00044
aci
Animdilyakwa
3813
0.99644
3--05
0.00028
arj
Mulian
15638
0.9995
3e-05
com
Ömie
7848
0.9995
3e-05
son
Bumbita Arapesh
7832
1.0
0.0
0,00011
Uah Meto
7962
0.99501
0.00019
uph
Sa'a
7764
1.0
0.0
apc
Levantine Arabic
68045
0.79142
0.00891
0.17857
0.00915
ape
Bukiyip
7832
0.9995
3e-05
0,00033
apn
Apinayé
7351
1.0
apc
Arop-Lokep
7853
1.0
0,0
apt
Apulami
7941
1.0
0.0
0.00011
apu
Apurina
7937
1.0
apw
Westem Apache
7930
0.9995
0.0
apy
Apulaí
0.9995
apz
Safeyoka
7830
1.0
0.0
0,00022
ara
Arabic
1044573
0.9985
8c-05
arb
Standard Arabic
7100859
0.81268
0.01268
0,25705
0.21515
0.98333
0.00011
are
Western Arramta
7810
1.0
0.0
arg
Aragonese
30103
0.97236
0.00011
arh
Arhuaco
3968
0.42298
0.00121
Arabela
7914
1.0
0,99187
0.0
am
Mapadungun
154241
0.81037
0.01037
0.93913
0,00011
arp
Aragaho
1151
0.9971
0.0
arq
Algerian Arabic
3826
0.89381
0.00036
0.00114
0,00011
ars
Nujdi Arabic
23194
0.7232
0.00624
0.00894
0.00574
ary
Arabic
31432
0.83395
0.00272
0,56588
0.05164
0,00011
arz
Egyptian Arabic
183549
0.86439
0.00635
0,46309
0.09806
asg
Cishingini
7900
0.9975
0.00011
Assamese
213937
0.99749
0.0
1.0
0.0
aso
Dano
7694
1.0
0.0
Asturian
1030498
0.97023
0.00151
0.9916
0.00051
0.97521
0.00011
ata
Pele-Ata
9433
0.999
3e-05
stb
Zaiwa
7905
1.0
0.0
std
Ata Manobo
7849
0.9985
6c-05
agg
Tyhie North-Oppela-Arhee
7947
0.9995
0.0
Attié
17922
0.78761
0.00209
stt
Pamplona Atta
7954
1.0
0.0
auc
Waorani
7930
1.0
0.01504
0.0
sui
Anuki
652
1.0
0.0
6o-05
auy
Awiyaana
7710
0.9965
8c-05
0,00044
ava
Avaric
7833
0.9975
0.00011
avk
Kotava
4103
0.95715
0.00019
0.00017
0.00022
avt
7878
avu
Avokaya
7590
1.0
0.0
awa
Awadhi
13074
0.94648
0.00011
0,38951
6c-05
awh
Awa (Papua New Guineu)
7880
0.9995
3c-05
swi
Ackyom
7790
0.999
0.0
0.00011
Awara
1635
0.9958
Aymura
368899
0.99402
0.00025
ayo
Ayoreo
7898
0.9995
3c-05
ayr
Central Aymara
173203
0.99449
8c-05
0,99557
0.00045
0.98361
0,00022
azb
South Azerbaijani
532
0.97778
3e-05
0.36583
0.00011
a2e
Azerbaijani
1069419
0.9995
30-05
a2g
San Pedro Amuzgos Amuzgo
7939
1.0
0.0
arj
North Azerbuijaami
472589
0.9935
0.00019
0.99901
0.74306
0.00668
32z
Highland Pucbla Naluatl
7943
1.0
0,0
0,00011
bak
Bashkie
171555
0.98846
0.00022
1.0
0.0
0.0012
hul
Baluchi
141
0.84211
0.0
bam
Bambara
20569
0.92308
0.00061
0.52563
0.05119
0.49682
0.00635
Balinese
32978
0.99348
8c-05
0.97521
6c-05
0.97561
0.00011
bao
Waimaha
7940
0.9995
3e-05
bar
Bavarian
130954
0.98611
0.00061
0.00011
Table 6: Performance of GlouLID-M on GloLLID-C test, FIORES-200 and UDHR benchmarks (part 2)

---

GloLLID-C
FLORES-200
UDHR
39.6399.3
Language
[Sentences
FIf
FPR.
FI
FPR.
F1
FPR.
bas
Basa (Cameroon)
132504
0.996
0,00014
0.00017
0.00022
hav
Vengo
7954
1.0
0.0
0.00011
bba
Baatonum
33686
0,999
6c-05
0,92187
0.00099
hhh
Barai
10264
1.0
0.0
bbc
Batak Toba
279942
0.98618
0.00074
bbj
Ghomálá'
27499
0.79298
0.00465
0.00109
hhr
Girawa
7675
0.999
3e-05
0.00011
bcc
Southern Balochi
1753
0.99313
3c-05
Bariai
10603
0,998
3e-05
bei
Baoulé
382411
0.998
8c-05
0.00011
0.97521
0.00033
hel
Central Bikol
1193855
0.99104
0.00039
1.0
0.0
Kaluli
bew
Bana
7874
1.0
0.0
hddd
Banama
7866
0.9985
6c-05
bdb
Baka (South Sudan)
7948
0.9995
0.0
0.00011
bea
Beaver
677
1.0
0.0
Benabena
7857
0,999
3c-05
0.00044
hel
Belarusian
428690
0.998
6c-05
1.0
0.0
0,98333
0.0
bem
Bemba (Zambia)
1477399
0.98039
0,0011
0.9906
0.00045
0.98333
0.00022
Bengali
1659455
0.97466
0.00143
0,99852
6c-05
1.0
0.0
Beembe
7934
0.60249
0.01
ber
Berber langunges
628585
0,84139
0.00514
0.01335
0.00099
Modo
10586
0.0
0.00274
bfd
Bafut
7941
0.999
3e-05
bfo
Mulba Birifor
7898
0.998
0.00011 1
bdz
Mahasu Pahari
0.83333
0.0
bgr
Bawm Chin
7859
0.99599
6c-05
hgs
Tagabawa
9968
0.999
0.0
bgz
Banggai
7819
0,9995
0.0
big
Binandere
2782
0.99632
0.0
bhl
Bimin
9573
0.999
3c-05
bho
Bhojpuri
62722
0.64017
0.02426
0.94329
0.00443
0.78519
0.00033
bhp
Bima
7876
0.9995
0.0
bhw
109097
0.99045
0.00011
bib
Bissa
7940
0.9995
0.0
big
Biangai
7814
0.9995
3e-05
0.00022
Bihari languages
10000
0.18688
0.00025
bik
Bikol
30000
0.98838
3c-05
bim
Bimoba
30166
0.998
0.00011
0.00033
bin
132682
0.9975
3c-05
0.9927
0.00011
bis
Bislama
1062315
0.998
0.00011
1.0
0.0
biu
Bicte
7929
0.99499
0.00011
biv
Southern Birifor
7936
0.999
0.0
0.00022
bjn
Banjar
32196
0.95164
0.00022
0.79496
0.05329
0.00536
bip
Fanamaket
877
1.0
0.0
bjr
Binumarien
9867
1.0
0.0
bjv
Bedjond
7919
0,998
3c-05
Binukid
0,998
0.0
bky
Bakainí
7773
0.9995
0.0
bku
Buhid
7911
0,999
3c-05
Bekwarra
0.0
bla
Siksika
204
0.98361
0.0
blh
Kuwan
7902
0.999
0.0
0.0001
Balangao
7883
0.9995
0.0
0.00033
biz
Balantak
7917
0.997
6c-05
hmb
Bembe
8008
0.56335
0.01023
bunh
Kcin
7688
0,9995
3c-05
brnk
Ghayavi
650
0.95385
6e-05
bunq
Bomu
7930
0.9985
0.0
hrnr
Muinane
7926
1.0
0.0
brnu
Somnha-Siawari
9565
0.999
3e-05
0.00011
buj
Eastern Tawbuid
78S1
0.9985
3c-05
Bola
14309
0.996
0.00011
boa
Bora
7650
1.0
0.0
0.99213
0.00011
Tibelam
24952
0.98419
0.94589
0.89091
0.00011
boj
Anjam
14853
1.0
0.0
0.00022
bom
Berom
7960
0.998
3c-05
bos
Bosnian
507207
0.76033
0.00138
0.58206
0.00312
0.18026
0.01007
buy
Tuwali
7907
0.9985
0.0
Buamu
7830
1.0
6c-05
0.00011
hpr
Kormaaaal Blaun
7840
0.96566
0,00066
bps
Sarangani Blaan
7840
0.96682
0.00118
bpy
Bishnupriya
30000
0.98785
0.0
byc
Boku (Benin)
30639
0.99551
0.00017
Table 7: Performance of GlotLID-M on GlotLID-C test, FLORES-200 and UDHR benchmarks (part 3)

---

GloLLID-C
FLORES-200
UDHR
1s0639.3
Language
[Sentences]
F11
FPR]
F1
FPR.
FT
FPR.
bij
Bandial
7903
0.998
3c-05
byp
Busa
7894
0.9955
0.00014
0.00011
bre
Breton
16810
0,98688
0.00011
0.98361
0.0
bru
Tastern Bru
7888
0.999
0.0
brx
Bodo (Iadia)
bac
Bassari
7949
0,9995
3c-05
0.00022
ben
Barasana- Eddrri
7547
0.999
3e-05
bsp
Baga Sitemu
4533
0,99089
0.00017
0.00011
beg
Bassa
4073
0.96104
3e-05
0.00028
0.00055
bas
Akoose
7905
0.999
0.0
bid
Batak Dairi
9423
0.99246
6e-05
0.00011
big
Gagnoa Bété
16784
0.77055
0.00396
0.00055
bth
Biatah Bidayuh
7889
0,999
0.0
bis
Batak Simalungun
8277
0,99097
0.00017
bit
Bete-Bendi
7866
0.999
0.0
0.00011
bix
Batak Karo
191461
0,97756
0,00069
bua
Buriat
10906
0,97807
6c-05
Ntcham
7916
0,999
3e-05
0.00022
bug
Buginese
15386
0,98379
8c-05
0,99802
6c-05
0,95312
0.00055
buk
Bugawac
7776
0.9995
0.0
bul
Bulgarian
1762751
0,98762
0.00061
0,99951
0.96
0.00055
bum
Bulu (Cameroon)
141188
0.99102
0.0003
6c-05
0,54762
0.00022
bus
Bokobaru
7882
0.99399
0.00014
0.00011
bvr
Burarra
7882
1.0
0.0
bvy
Bay bayanon
76
bvz
Bauzi
7501
1.0
0.0
0.00011
bwd
Bwaidoka
1522
0.99797
0.0
bwi
Baniwa
262
0.96104
0.0
bwq
Southern Bobo Mudaré
7921
0,99699
3c-05
0.00011
bwu
Buli (Gianaa)
7824
0.998
3e-05
Bubutu
4196
0.99915
3e-05
0.00011
bxr
Russia Buriat
8599
0.997
8c-05
byr
Baruya
8233
1.0
0.0
byv
Medumba
2171
0,97605
0.00394
byx
Qaget
7783
0.9995
0.0
bad
Bribri
8660
0.999
0.0
0.00077
bzh
Mapos Buang
7937
0.9995
3e-05
0.00011
bzi
Bisu
7830
0,999
0.0
bxj
Belize Kriol English
124087
0,94398
0.00179
bzt
Brithenig
357
0.92308
3c-05
0.00011
cuu
Chorti
7940
1.0
0.0
cab
Garifona
228814
0.98498
0.00039
0.992
0.00011
cac
Chuj
38234
0.999
35-05
0.00055
caf
Southern Carrier
7943
0.99198
0.00017
cag
Nivaclé
9167
0,9995
0.0
0.00033
cak
Kagehikel
164900
0,99601
0.00019
1.0
0.0
caa
Chicoho
0.999
cap
Chipaya
15847
0.999
3c-05
0.00011
cag
Car Nicobarese
32067
1.0
0.0
Galibi Carih
9359
0.9995
3e:05
0.00055
cas
Tsimand
7870
0.998
3c-05
0.00011
cul
Catalan
1137480
0.97554
0.00129
0,93023
0.00099
cav
Cavinelta
7741
0.9995
0.0
0.00011
cux
Chiquitano
15872
1.0
0.0
0.00011
cay
Cayuga
cbc
Caripuna
7791
0.9975
3e-05
ebi
Chachi
7863
1.0
0.0
0.9771
0.00022
cbk
Chuvacano
1115
0.98029
0.00025
0.00033
chr
Cashibo-Cacttaba
7813
0.9995
0.0
0.7033
0.0
ebs
Cashinahua
7502
1.0
0.0
6c-05
0.67308
0.00033
cbi
Chayahuita
7804
0.9995
0.0
0,97479
0.00011 1
ebu
Candoshi- Shapra
7588
1.0
0.0
0,33803
0.0
chv
Cacua
7694
1.0
0.0
ccc
Chopi
120353
0.99299
0.00014
6c-05
0.00011
cco
Comaltepec Chinuntec
7890
1.0
0.00131
eeb
Cebuano
2111383
0.97987
0.00107
0.99503
0.0
0,96721
0.00044
ong
Chumacozz
7912
0.9995
0.0
cek
Eastern Khumi Chin
7873
1.0
0.0
ces
Czech
1679384
0.99156
0.00044
0.99951
6e-05
0,98387
0.0
cfm
I'alm Chin
38517
0.98848
0.00028
0.85714
0.0
ege
Kagayanen
7823
0,9995
3c-05
ogg
Chiga
40958
0.98943
0.00011
0.00022
cha
Chamorro
16006
0.996
0.00014
6c-05 6c-05
0.8381
0.00011
chd
Highland Oaxaca Chontal
8393
0.998
0.0
0.00011 1
ebe
Chechen
60837
0.995
0.00017
Table 8: Performance of GloLLID-M on GloLLID-C test, FIOORE--200 and UDHR benchmarks (part 4)

---

GloLLID--C
F0OOES
UDHR
1s0639.3
Language
Sentences
Flf
FPR
FI
FPR
Fl1
FPR.
chf
Tabasco Chontal
8213
0.998
6c-05
0.00022
chg
Chagatai
chj
Ojitlán Chinantec
6824
0.72469
0.00091
0.12727
0.00471
chk
Chuukese
377198
0.99701
0,00017
0.97521
0.00011
chn
Chinook jargon
53
0.8
0.0
cho
Choctaw
124
0.78788
0.0
0.00263
chq
Quiolepec Chinantes
7891
1.0
0.0
0.00044
chr
Cherokee
7982
0,99699
3c-05
0.09677
0.0
chu
Church Slavic
7836
0.4985
0.0
chv
Chuvash
39476
0,99301
0.00025
0.86154
0.0
chw
Chuwabu
119383
0.997
0,00011
6c-05
chz
Orumacin Chinuntes
7919
1.0
0.0
<jk
Chokwe
193952
0,9955
0.00011
0.84493
6c-05
0.92641
0.00033
cjo
Ashéninka Pajonal
7774
0,99354
0.00033
<jp
Cahécar
7921
0.9995
0.0
cjs
Shor
1553
0,98929
0.0
0.65217
0.0
<jv
Chuave
7852
0.9995
0.0
<jy
Jinyu Chinese
18
ckb
Central Kurdish
138398
0,99007
0,00047
0,99901
0.00011
cko
Anufo
7882
1.0
0.0
0.00131
ckt
Chukot
1675
0.98619
6c-05
0.00033
cle
Lealao Chinuntes
7934
1.0
0.0
clu
Caluyanun
7850
0.99451
0.00019
cly
Eastern Highland Chatino
7928
0.77037
0,00217
0.00066
eme
Cerma
7890
0.9995
0.0
0.00011 I
cmi
Emberá- Chamí
16270
0,6085
0.01438
Mandurin Chinese
1073282
0.83808
0.01009
0.05727
0,51064
0.07 148
cmo
Central Mnong
16195
0,99699
0.0
0.00011
cuh
Hakha Chin
435869
0,99057
0,00047
0,93023
0.00099
eni
Asháninka
11125
0,80793
0,00228
0,90226
0.00142
cnl
Lalana Chinantee
7911
1.0
0.0
ent
Tepetotutla Chinantec
7924
1.0
0.0
cnw
Ngawn Chin
7929
0,99548
3c-05
0.00055
coc
Koreguaje
7741
0.9995
35-05
0.00164
cof
Colorado
7555
1.0
0.0
0.74747
0.0
cok
Santa Teresa Cora
18162
0,99449
0.00011
Culán
7876
0.9985
3c-05
0.00011
cop
Coptic
23773
1.0
0.0
cor
Comiss
41272
0,99097
0.00017
cos
Corsican
11141
0,97444
8c-05
0.00017
0,95082
0.00044
col
Caquinte
7879
0.9975
3e-05
0.96774
0.00022
spa
Pulanila Chinantec
7946
0.9995
0.0
cpb
Ucayali-Yurda Yurda Ashéninka
7947
0,98448
0.00039
0.00022
opc
Ajyíninka Apunccayali
7939
0.998
0.0
opi
Chinese Pidgin English
spu
Pichis Ashéninka
7945
0,98754
0.00044
0.89908
0.0
<py
South Ucayali Asháninka
6127
0.9937
6c-05
eth
Crimcan Tatar
28216
0.97352
0.00022
0.98902
6c-05
0.97561
0.00033
Säotomense
2594
0,94133
8e-n5
6e-05
0.84404
0.00033
erk
Plains Cree
293
0,86486
0,00011
em
Moose Croe
8076
0.9995
0.0
0.00547
EI Nayar Corn
23287
0.99451
0.00019
ciq
Iyo' wujwa Chorote
7731
0,36727
0.00954
0.00055
crs
Seselwa Creale French
506502
0,99104
0.000399
0.0
eit
Ryojwa'ja ja Chorote
7826
0.554889
0.0192
0.00066
erx
Carrier
7935
0,99151
0.00028
osb
Kashubian
10947
0,98221
35-05
0.00011
esk
Jola-Kasa
7918
0.999
0.0
6c-05
eso
Sochiapam Chinantec
7936
0.0
Swampy Cree
476
0,93671
3c-05
0.0
0.00186
esy
Siyin Chin
30034
0.98449
0.00041
cta
Tataltepec Chatino
7908
0.999
0.0
0.00066
ctd
Todim Chin
35149
0.88063
0.00564
0.78431
0.0
cto
EmberáiCatio
22638
0.86318
0.00358
6c-05
0.00011
ctp
Western Highland Chatino
7910
0.9995
3e-05
ctu
Chol
174449
0.9879
0.00011
0.00044
cub
Cubeo
7943
0.9995
3c-05
0.00033
euc
Usila Chinantee
7901
1.0
0.0
cui
Cuiba
7882
0.9995
3e-05
cuk
San Blas Kuna
26510
0.998
3c-05
6c-05
0.00022
cul
Culina
7821
1.0
0.0
cut
Teutila Cuicatee
7835
0.999
0.0
cux
Tepeuxila Cuicatee:
7954
1.0
0.0
owd
Woods Cree
278
0.91304
0.00022
Kwere
7950
0.97405
0,00077
Table 9: Performance of GlotLID-M on GlotLID-C test, FLORES-200 and UDHR benchmarks (part 5)

---

GloULID-C
FLORES-200
UDIR
iso639-3
Language
[Settences
FIf
FPR
FI1
FPR
FI1
FPR
ewt
Kuwaataay
7828
1.0
0.0
eya
Nopula Chatino
7946
0,82102
0.00891
0.00013
eym
Welsh
273301
0.99501
0,00019
0.99951
6c-05
1.0
0.0
cdl
7otungg Chin
9478
0.999
3e-05
0.00011 I
daa
Dangalćar
7839
0.999
3c-05
0.00011
dud
Murik
7726
0.9995
3e-05
daf
Dun
12719
0.9995
0.0
dah
Gwahatike
7704
0.999
3c-05
duk
Dakata
2366
0.99856
0.0
dan
Danish
3921737
0.96039
0.00173
0.9931
0.00051
0.85135
0.00241
dur
Dangwa
2195
0.99543
3e-05
0.00055
ddg
Fataluku
1603
0.98934
6c-05
0,00077
ddo
Dido
20
ded
Dedua
10276
0,999
0.0
des
Desano
7740
1.0
0.0
deu
German
1800293
0.96193
0.00212
0.99901
0,98745
0.00011
dga
Souther Dogaare
12612
0.998
3c-05
0,71166
0,00482
dge
Casiguran Dumugat Agta
7795
0.9995
0.0
dgi
Norther Dugara
7943
0.9975
3e-05
dgr
Dogrib
9376
0.9995
0.0
dgz
Daga
7802
0.9975
0.00011
0.00011
dig
Dhangu- Bhangu-Djangu
630
1.0
0.0
dhm
Zemba
7908
0.99649
6c-05
0.00023
0.00022
dhv
Dehu
403863
0.95252
0.00223
0,00514
dig
Digo
7865
0.99449
0.00011
dik
Southwester Dinka
33790
0.99551
0.00019
0.99653
6c-05
0.00799
din
Dinka
3290
0.38141
0.0
dip
Northeustern Dinka
7933
0.98742
0.00017
diq
Dimli (individual language)
30006
0.997
6c-05
0.00011
0.00109
dis
Dimasa
7955
0.999
3--05
6c-05
diu
Diriku
198
0.8 0.88889
0.0
div
Dhivehi
30040
1.0
0.0
0.96774
0.0
dje
Zarma
7899
0.99699
3e-05
6c-05
0.00109
djk
Easter Maroon Creole
98689
0.99548
djr
Djambarrpuyngu
7771
0.9985
8c-05
0.0004
dks
Southeastern Dinka
10409
0.98705
0.00047
0.00011
dng
Dungan
944
0.98962
0.0
0.00033
dnj
Dan
15219
1.0
0.0
dob
7854
0.9985
0.0
dop
Lukpa
7934
1.0
0.0
dow
Doyayo
7919
0.9995
3e-05
0.00022
drg
Rungus
244
0.82857
0.0
drt
Drents
40
dru
Rukai
30027
0.996
0.00011
dsb
Lower Sorbian
11095
0,97823
0.00025
dip
Kadazan Dusun
19197
0.95972
0.00162
6e-05
0.00011
dts
Toro So Dogon
7822
0.9995
0.0
dua
Duala
$2748
0.9975
3e-05
0.00011 1
due
Umiray Dumaget Agta
7864
0.9985
0.0
dug
Duruma
7836
0.99499
0.00011
duo
Dupanimam Agia
7795
0.998
0.0
dur
Dii
7870
0.9985
3c-05
0.000L1
dwr
Dawro
7801
0.99044
8e-05
dws
Dutton World Speedwords
57
0.46154
0.0
dww
Dawawa
7876
0.9985
3e-05
dyi
Djimini Senoufo
7927
1.0
0.0
0.00033
dyo
Jola-Fonyi
9559
0.9985
3c-05
0.97391
0.0
dyu
Dyula
218015
0.95472
0.00234
0.12435
0.01449
0.23188
0.0069
dzo
Dzongkha
6899
0.9843
8c-05
0.9496
0.00585
0.90769
0.0012
ebk
Tastem Bontok
7913
0.9975
3e-05
eñi
Efik
1078995
0.99451
0.00022
0.00755
cgl
Emilian
144
0.38889
3c-05
6c-05
eka
Ekajuk
7942
0.9985
3e-05
ekk
Standard Estonian
300000
0.98759
0.00055
0.90226
0.00142
eka
Koti
3176
0.99494
0.0
ell
Modem Greek (1453-)
4450890
0.98862
0.00061
1.0
0.0
0.97908
0.0
emi
Mussau-Emira
4352
0.99753
0.0
eml
Emiliano-Romannod Romagnolo
30000
0.98601
0.00041
0.00045
0.00952
emp
Northern Emberá
8463
0.997
6c-05
Erromintxela
12
cub
Markweeta
7871
0.9995
3c-05
eng
English
10703345
0.63088
0.03212
0.98732
0.00148
0.85294
0.00197
enl
Enibet
7903
1.0
0.0
cnm
Middle English (1100-15000
39815
0.9823
0.00017
0.00022
enx
Enxet
7890
1.0
0.0
Table 10: Performance of GlotLID-M on GlotLID-C test, FLORES and UDHR benchmarks (part 6)

---

GloLLID-C
FLORES-200
UDHR
3so639..3
Language
Sentences
F11
FPR.
F11
FPR.
FI
FPR.
epo
Esperanto
1161088
0.98377
0.00091
0,99852
0.900117
0,96825
0,00044
eti
Ogen
7908
0.9995
3c-05
ese
Esc Eija
7842
0.9995
0.0
0:75
0.00011 1
esi
North Alaskan Inupiatun
7915
0.992
0,00022
0.00011 1
esk
Northwest Alaska Inupiation
7921
0,99249
0.00017
0,00011 1
est
Estomian
3664155
0.99399
0.00011
1.0
0.0
esu
Central Yupik
7942
0.9995
3e-05
eto
Eton (Cameroon)
208
0,95082
3e-05 05
0.00011
Edolo
3182
0,99898
0.0
etu
Ejagham
7900
1,0
0.0
eus
Basque
938424
0,97363
0,0014
0.99951
0.0
0,00131 0,91045
eve
evn
Even
Evenki
1149
116
0,98382
0.73684
3e-05
6e-05
0,30755
0,27869
0.00372
1000300
ewe
Ewe
1698872
0.99206
0.00044
1.0
0.0
0.98361
0,00022
Ewando
7942
0.9985
0.0
0,00097
ext
Extremaduran
10066
0,96701
65
0.00017
0,00011
cza
Ecaa
30880
0.99353
0,0003
faa
Fasu
7854
0.9995
3c-05
fad
Wagi
1147
1.0
0.0
fai
Faiwol
7941
0.999
3e-05
fal
South Fali
7916
0,999
3e-015
fan
Fang (Equatorial Guinea)
22472
0,99144
6c: 05
0,00142
fao
Faroese
90577
0.99399
0.00011
0,99951
0,98300
0.0
fas
Persian
1000000
0.88456
0,00718
fat
Fanti
54615
0.99448
3e-05
0,97521
0,00022
fm
Massina Fulfulde
7872
0,97054
0.00855
0,00044
Fijian
1232612
0.99106
0,000044
0.99951
0.0
1.0
0.0
Filipino
38283
0.81967
0.00022
fin
Finnish
3909988
0.94857
0,001286
0.99901
0.00011
0.36
0,02452
fkv
Kven Finnish
539
0.67187
35-05
0,28571
0.0
fmp
Fe'fe'
104
0.92308
0,0
0
fon
Fou
259192
0.9945
0.00017
0,99752
0.0
0,94118
0.00044
for
Fore
7892
1.0
0.0
0,00263
fra
French
2570543
0,74664
0.01862
0.99951
6e-05
0,95238
0.08066
frm
Middle French (ca. 1400- 1600)
fro
Old French (842-ca 1400)
216
0.08333
0.0
frr
Northern Frisian
12856
0.98633
3e:05
0.00017
0,00022
fry
Western Frisian
131535
0.99651
0.00017
0.99174
0.0001 1
fub
Adamawa Falfulde
34642
0.98303
0,00052
0.00017
0,00175
fuc
Pulsar
fod
East Futuna
11205
0.99246
0.00028
0,00011 1
fue
Borgu Fulfulde
4664
0.9815
0.00022
6e-05
fuf
Pular
12566
0.98898
0.000255
0,04762
0.0X044
fuh
Western Niger Fulfuulde
7899
0,97049
0.0008
0.00022
foq
Central-Fastem Niger Fulfuulee
7908
0.96761
0.00099
fur
Friulian
$4651
0.94985
0,00072
0.99951
6e-05
0.80272
0.00306
fuv
Nigerian Fulfulde
22406
0.97815
0.9008
0.96844
6c-05
0,76984
0.00241
gaa
Ga
1217812
0,99501
0,00019
0,93846
0.0008S
gag
Gugauz
17347
0.99548
35-05
0,94017
0.0
gah
Alekano
7879
0.999
3e 05
0.0001
gai
Borei
7866
0.9995
3e:05
gam
Kandawa
7799
0.999
3e-05
0.0001 1
gaw
Nohonob
7838
1.0
0.0
gaz
West Central Oromo
335746
0,99301
0.00022
0,99411
0.00068
0.85221
0.00274
gha
Gbaya (Central African Republic)
1010
0.99617
0.0
ghi
Galela
7722
0.9985
6e-05
ghm
Garhwali
gho
Northern Grebo
7939
1.0
0.0
0.0001 1
ghr
Giaagi
7660
0.998
0.0
0,0001
gef
Guadeloupean Creole French
82017
0.98254
0.00055
0.00023
0.00471
ger
Guianese Creole French
32425
0.98852
0.00036
0.00022
gde
Gude
7901
1.0
0.0
gdg
Ga dang
7827
0.9995
0.0
0.0X022
gdn
Umunakainu
7737
1.0
0.0
0.00022
gdr
Wipi
7880
1.0
0.0
geb
Kire
7822
0.9995
3e-05
gej
Gen
7953
0.9985
6c-05
gfk
Parpatar
7657
0.9995
0.0
ghe
Southern Ghale
7953
0.9995
0.0
ghs
Guhu Samane
7461
0.998
3e-05
gid
Gidar
7919
0.9995
0.0
gil
Gilbertese
428828
0.998
8c-05
giz
South Gizigu
35118
0.9965
0.00011
gin
Gonja
30289
0.997
6e-05
0.90476
0.00088
gkn
Gokana
96606
0.98947
0.00022
Table II: Performance of GlotL JDD- on GlotLII-- test, FLOORSS-200 and UDHR benchmarks (part 7)

---

GloLLID-C
FLORES-200
UDHR
1s0639.3
Language
[Sentences]
F11
FPR]
F1
FPR
F11
FPR.
gkp
Guinca Kpelli
45287
0,54231
0.01537
0,92063
0,00109
gla
Scottish Garlic
95320
0.99298
0.00011
0.99951
6e-05
0.94574
0.00022
gle
Irish
274216
0.9935
0.00017
0.95
0.00088
glg
Galician
343533
0.98495
0.00033
0,99703
0.00011
0,98305
0.00011
glk
Gilaki
10879
0,93362
0.00011
0,00818
glv
Manx
45473
0.997
0.00011
1.0
0.0
gmv
Gamo
15674
0.99701
0.00014
0.00011
gab
Gangte
7955
0,99296
3c-05
0.00022
gnd
Zulgo-Gemzek
7873
0.998
6c-05
gng
Ngangam
7833
0,999
0.0
gnn
13043
0.9985
0.0
0,00022
gaw
Westem Bolivian Guaraní
17302
0.996
0.00011
goa
Guro
1971
0.99286
3e-05
0.00142
gof
Gofa
7908
0,98541
0.00022
gog
Gogo
8642
0.99348
8c-05
0.00062
0.00011 1
gom
Goan Konkani
89931
0,96933
0.00069
0.00017
0.00044
gor
Gorontalo
7895
0.999
3c-05
gos
Gronings
5670
0.94176
6c-05
got
Gothic
3857
0,99652
0.0
gqr
Gor
7931
0.9975
8c-05
gre
Ancient Greek 1453)
87146
0,98889
3e-05
0.00055
grn
Guarani
62443
0,98069
8c-05
1.0
0.0
grt
Garo
8033
0.9985
0.0
6c-05
g50
Southwest Gbaya
7902
0,9995
0.0
gsw
Swiss German
108513
0,95317
0.00201
0.00011
0,98333
0.00011 I
gub
Gunjajára
30075
0,51632
0.01438
gue
Wayuu
248890
0.999
6c-05
0.96063
0.00055
gud
Yocoboué Dida
7933
0.9985
6e-05
gug
Paraguayan Guaraní
384689
0.87191
0.00762
0.81967
0.0
guh
Guahibo
9114
0.95575
8c-05
0,0012
gui
Eastern Bolivian Guaraní
15213
0.99499
8c-05
0.00153
guj
Gujarati
1018488
1.0
0.0
1.0
0.0
1.0
0.0
guk
Gumuz
7906
0.9985
0.0
gul
Island Creole English
0.9985
6c-05
Guambiano
8925
0.95248
0.00039
0.00011
gun
Mbyá Guarani
7929
0.9985
0.0
guo
Gunyabero
7809
1.0
0.0
gng
Aché
7925
1.0
0.0
gur
Farefare
71300
0
0.00014
6c-05
0.00011
Gun
765163
0.9975
0.00011
0.00011 1
gux
Gourmanchéma
7927
0.9975
3e-05
0.00022
guz
Gusüi
7935
0,99599
6c-05
gve
Guanaano
7775
0.9995
0.0
gvf
Golin
7896
0.9985
0.0
gvl
Gulay
7956
0,9985
3e-05
0.00022
gva
Yalanji
8219
0.9985
6c-05
gwi
Gwichin
7845
1.0
0.0
gxx
We Southern
24463
0.63256
0.01293
0.00208
gya
Northwest Gbaya
35345
0.99499
0.00011
gym
Nglibere
272246
0.99551
0.00017
0.00022
gyr
Guarayu
15925
0.9985
8c-05
0.62745
0.0
Fastern Oromo
7951
0.99649
hag
Hanga
7659
0.9985
6c-05
0.00044
hak
Hukka Chinese
38023
0.99598
0.0
hat
Haitian
380465
0,98213
0.00069
0.99852
0,00017
0,90706
0.00274
hau
Hausa
401986
0.98854
0.00041
0.95427
0.00551
0,94488
0.0023
hav
Havu
8780
0.98434
0.00014
6c-05
haw
Hawaiian
7859
0.99497
0.0
6c-05
1.0
0.0
hay
Huya
16371
0.92727
8e-05
6e-05
hbo
Ancient Hebrew
102471
0.9985
8c-05
0.00011
hiss
Serbo Serbo-Croatiun Croation
300000
0.99401
0.00019
hch
Huichol
13394
0.9985
0.0
hdn
Northern Haida
3
heb
Hebrew
2021869
0.98328
0.00094
0.99606
0.00045
0,99145
0.00011
heg
Helong
9068
0.999
3e-n5
hch
Hche
8605
0,9955
0.00011
0,00051
0.00022
her
Herero
154169
0.99549
6e-05
6e-05
hif
Hindi
17894
0,98742
0.00017
0.00034
0.00077
hig Kamwe
7900
0.999
0.0
Hiligaynonn
1762387
0.98665
0.00069
0.00057
0.0
hin Hindi
2227417
0.80097
0.01342
0.67444
0.05551
0.62
0.00832
hix
Hixkaryáma
7756
1.0
0.0
hla Halia
7706
0.999
3c-05
blt Matu Chin
7916
1.0
0.0
0,92982
0.0
hmn Hmong
163469
1.0
0.0
Table 12: Performance of GDoMLID-M on GloILID-C test, FL.ORES-200 and UDHR henchmarks (part 8)

---

GlotLID-C
FLORES-200
UDHR
iso639933
Language
Sentences
F11
FPR.
FIf
FPR,
FI
FPR.
hmo
Hiri Motu
1029005
0,97791
0.00113
hmr
Hmarr
50021
0.99449
8e-05
hne
Chhatissarhi
60771
0.95589
0.00017
0.90296
0.00256
0.00011
hnj
Hmong Njua
61134
0.99552
0,00022
hnn
Hanunoo
7931
0,9995
0.0
hns
Caribbean Hindustani
9791
0,99599
6c-05
0.88889
0.00033
hoc
Ho
1448
0,97035
8c-05
hop
Hopi
7910
1.0
0.0
hot
Hote
7646
hra
Hrangkhol
30908
0.9935
0.00017
hrv
Crontian
1901496
0.80685
0.01081
0.75157
0.03272
0.58824
0.00919
hrx
Hunsrik
46411
0,97319
0.00041
hab
Upper Sorbian
31517
0,98205
0,00058
1.0
0.0
han
Xiang Chinese
hto
Minica Huitoto
7926
0.9995
3c-05
huh
Huambisa
15262
0,9985
6e-05
0.00033
hui
Huli
7838
0,9995
0.0
0.00011
hun
Hungarian
4361329
0.9726
0,00138
1.0
0.0
0.82192
0.00285
hus
Huustec
42824
0.98231
0.00019
0,97814
0.00044
hun
Murui Huitoto
7704
0.0
0,98305
0.0
huv
San Mateo Del Mar Huave
10600
0.998
0.0
0.00011
hvn
Sabu
7902
0.9995
0.0
Hawai'i Creole English
17681
0.85699
0.00283
0.00011
hye
Armenian
1425068
0.9985
6c-05
1.0
0.0
1.0
0.0
hyw
Western Armenian
679292
0.9995
3e-05
ian
Latmul
7674
0.999
3c-05
iba
Iban
210100
0.9995
0.0
ibg
Ibanag
117204
0.997
8c-05
0.00033
ibo
Igbo
537005
0,99402
0.00025
0.99951
6c-05
0.98718
0.00022
ier
Islander Creole English
7849
0,99699
0.0
ido
Ido
40359
0.97836
0.00041
6c-05
0.95
0.00011
idu
Idoma
74684
0,97688
0.0005
0.0
0.00011
ifa
Amganad Ifugao
30672
0,98516
0.00072
ifb
Batad Ifugao
7913
0,98637
0.00011
6c-05
ife
Inc
7870
0.9995
0.0
6c-05
0.00011 1
ifk
Tuwali Ifugao
7699
0.99397
3e-05
ifu
Mayoyao Irugao
7819
0.999
6c-05
0.00022
ify
Keley-I Kallahan
29643
0,99552
0.00022
ige
Igede
76711
0,9995
0.0
0.00022
ign
Ignaciano
15364
0.9995
3c-05
igs
Interglossa
32
Sichuan
60
ij
Izon
3785
0.60347
0.00069
0.00022
ike
Eastern Canadian Inuktitut
30996
0.999
3e-05
0.96842
0.00033
ikk
Ika
7953
0.9995
0.0
ikw
Ikwere
7934
0.999
ilb
lla
6805
0.99094
6c-05
Interlingue
17750
0.96945
0.00033
6e-05
0.0001
ilo
Toko
2917828
0.9828
0.00096
0.99951
6e-05
0.90625
0.00131
imo
Imbongu
7720
0.998
6c-05
ina
Interlingua
58339
0,97512
0.00083
6e-05
0.84892
0.00219
inb
Inga
8000
0.9995
0.0
0.00153
ind
Indlonesian
2835053
0.8254
0.01117
0.91929
0.00983
0.72
0.00405
ino
Inoke-Yate
7474
0,9995
0.0
0.00044
iou
Tuma-Irumu
12564
0.9985
0.0
ipi
Ipili
6181
0.99944
0.0
iqw
Ikwo
7943
0.98849
0.0003
in
Rigwe
7948
1.0
0.0
ick
Iraqw
7936
0.9985
0.0
iry
Traya
7904
0.999
3e-05
isd
Isnag
7892
0.999
0.0
ish
Esan
111255
0.97726
0.00033
0.00011
isl
Icelandic
150748
0.9955
0.00011
0.99901
0.0001 1
0.9916
0.00011
iso
Isoko
656673
0.995
0.00017
0.00022
Itaiian
2485451
0.91225
0.00523
0.99803
0.00017
0.78947
0.00339
its
Isckiri
11791
0.70713
0.00267
itv
Itawit
7928
0.9975
3c-05
ium
Iu Mien
$4967
0.00099
ivb
Ibatan
7894
0.9975
0.00011
ivy
Ivalun
7895
0.9975
3e-05
jwss
Scpik twam
7956
1.0
ixl
Ixil
24719
0.9985
3e-05
izh
Ingrian
izr
Izere
7945
0.998
8c-05
6c-05
izz
Izii
7947
0.98445
0.00033
0.00011 1
Table 13: Performance of GlotLID-M on GlotLID-C test, FLORES-200 and UDHR benchmarks (part 9)

---

GlotLID-C
FLORES-200
UDHR
3906399.3
Language
Sentences
F11
FPR.
F11
FPR.
F11
FPR.
jac
Popli'
8027
0.998
0.0
0.00011
Yahem
7624
1.0
0.0
Jasuicca Creole English
23393
0,99299
0.00014
6e-05
jav
Javanese
233182
0,98066
0,00077
0,98346
0,00187
0,97581
0.000444
jbo
Lojban
16508
0,99347
6c-05
0,000999
jha
Jukun Takum
7940
0.99599
Die 05
0.000344
Jadeo-Tal
gic
Tal
7907
0.9995
0.0
Shuar
17084
0,80832
0.00165
0.48387
0.01379
ine
Machame
7957
0.9975
8c-05
0.00017
inx
Westem Juxtishuaca Mixtee
234
0.13636
36-05
0.00066
ipa
Jewich Palestinion Aramacc
jpn
Japanese
2236956
0.98375
0.00088
1.0
0.0
0.71498
0,01292
Jarai
7944
0.9995
0.0
ivn
Caribhean Javanese
7750
0.99448
3e-05
kan
Kam- Kalpak
53472
0.99298
0.00011
0,96667
0.00022
kab
Kabyle
704680
0.83958
0.00572
0.85967
0.01221
0.00613
kac
Kachin
213841
0.99651
0,00014
1.0
0.0
0.00011
Kalaallisut
248414
0.996
0.00011
0,98305
0.0
kam
Kamba (Kenya)
359666
0.995
0.00017
0.92406
6c-05
kan
Kannada
733181
0,98377
0.00091
0.0
0.0
kao
Xaasongaxango
7909
0.9985
66-05
0.00033
kap
Bezhta
912
0.99259
0.0
kay
Capasshua
7924
1.0
0.0
0.00996
kas
Kashmin
12946
0.98941
6e-05
0.97674
0.0
Georgian
734914
0.9975
0.00011
kaz
Kazakh
386828
0.99452
0.00028
0.99951
0.0
0.96721
0.00033
Abc
Kadiwéu
7861
0.9995
36-05
khd
Kahardian
54247
0.99602
0.00222
0,87692
0.00175
kbh
Cansá
7884
1.0
0.0
kbm
Iwal
7796
0.9995
3c-05
khp
Kahiyé
234402
0.9945
0,00017
0.99901
65-05
0.85714
0.00219
khq
Kamano
10526
0.9995
0.0
0.00011
kbr
Kafa
74,99
0.9995
36-05
0.99174
0.00011
kok
Kalanga
23763
0,93843
0.0
0,00033
kdc
Kutu
7929
0,97477
0.00044
0.00011 1
kiko
Makoade
18825
0.99649
6c-05
0.00034
0.59813
0.0
kdh
Tem
1071
0,99408
0.0
0,77551
kdi
Kumam
7938
0.9965
0.00011
0.00022
kdj
Karamejong
0,99699
35
0.00011
kull
Tsikimba
7901
0.996
8c-05
ken
Kahuverdiamn
147918
0.9449
0.00226
0,95238
0.0
0.72727
0.00109
kei
Kei
297
0,95556
0.0
6c 05
0.00657
kek
Kekchí
203758
0.99701
0.00014
0.97521
0.00033
ken
Kenyang
7933
0.999
0.0
ket
Ket
20
kew
West Kewa
0.99198
0.00017
kex
Kukna
873
0.97143
0.0
kez
Kukele
7935
1.0
0.0
kff
Koya
7934
0.999
0.0
kell
Kube
10398
1.0
0.0
kgk
Kaiwá
7725
0.9985
8c-05
0.00011
kgp
Kaingang
9799
0.9995
0.0
kha
Khusi
36201
0,98589
0.00017
0.00011
0.97521
0.00022
khi
Halh Mongolian
176464
0.9965
8c-05
1.0
0.0
0.98462
0.00011 1
khm
Khmer
86506
0.999
6e-n5
0,99951
0.0
0.0
khs
Kasua
7848
0.9985
Se
0.00033
khy
Kele (Democratic Republic of Congo)
791
0.9975
36-05
6c-05
0.00011
khz
Keaparn
7929
0.997
8c-05
kia
Kim
30712
0,99499
85-05
kik
Kikuyu
$19523
0.98863
0.00063
0.96562
0.00403
0.00022
kin
Kinyarwanda
1575481
0.96376
0.0016
0.91471
0.00034
0,76336
0.002.
Kirghix
740492
0,98714
0.00066
1.0
0.0
0.94488
0.00077
kiu
Kimnsnjki (individual language)
6
kix
Khiamniungan Naga
7935
1.0
0.0
kjb
Q'anjob'a
30505
0.996
0.00011
kge
Kisar
7827
0.998
35-05
kjlt
Khukas
43856
0.88244
0.00322
0.83099
0.00252
kjs
Fast Kewa
7917
0.99301
0.00225
0.00011
kicc
Odnodee
5179
0.99933
0.0
kki
Kagulu
7957
0.99102
0.0003
0.00077
kij
Kako
7920
0.999
0.0
0L00044
Kosarek Yale
0.99907
0.0
kij
Khalaj
20
Kalenjin
4550
0,99782
0.0
Table 14: Performance of GlotLID-M on GlotLID-C test, FLORES-200 and UDHR benchmarks (part 10)

---

GloLLID-C
FLORES-200
UDIIR
1506339-3
Language
Sentences
FPR]
FI FPR
FI?
FPR.
kit
Nukma
752
0.99567
3e-05
Mackelynes
7892
0.999
3e-05
kma
Konni
7936
0.999
6c-05
kmb
Kimhundh
385084
0.98706
0,0005
0.96321
0,00415
0.99194
0.00011
knng
Kãte
6156
1.0
0.0
kmh
Kalam
15160
0.999
3c-05
Limos Kalinga
0.999
kram
Kom (India)
30098
0,99253
0.0003
kmo
Kwoma
7848
0.9995
3e-05
kmn
Northem Kurdish
262682
0.999
3c-05
0.99901
0.0
0.666667
0.00666
kms
Kamasau
7838
1.0
0.0
0.00011
kmu
Kanite
1.0
0.00044
kne
Contral Kanuri
16253
0.99498
3e-08
0.8634
0.00153
0,9635
0.00044
kne
Kankanaey
7675
0.997
8e-05
Mankanya
0.0
kng
Koongo
7934
0.99497
0.0
0.0
0.00033
Western Kanjobal
D.99999
3c-05
0,00022
knk
Kuranko
7863
0.998
3e
6c-05
kno
Kono (Sierra Leooe)
7827
0.9995
0.0
knv
Tabo
15492
0,9905
3e-08
0.00131
knx
Kendayan
426
0.95238
6e 05
kny
Kanyok
4209
0.99619
3e-05
kog
Cogui
7874
0.9995
0.0
0.00131
koi
Komi-Permyak
10043
D.98942
8c-05
0.95082
0.00055
Komi
10000
0.98939
0.0
kun
Kongo
885632
0.999
0.0
0,99802
0.0
koo
Konza
208525
0.99354
0.00033
0.00017
0.79389
0.00011
kor
Korcan
1736401
0.99701
0.00017
1.0
0.944888
0.00077
kos
Kosrzean
87905
0.99501
0.00019
Apf
Kounba
0.998
3c-05
0,00022
kpg
Kapingamarangi
30071
0,9055
0.00011
Karujá
7642
0.9995
0.0
Age
Korale-Yegle Yegha
7634
0.9995
3c-05
Apv
Komi Zyrian
9672
0.99699
0.0
0.00011
kpw
Kobon
0.0
kpx
Mountain Koiali
7794
0,9995
0.0
Apz
Kupsabiny
7945
0.9975
8c-05
0.00044
kge
Doroma-
0.99873
0.0
kue
Kalagan
7872
0.99298
0.00011
kif
Kakahai
654
0.99487
0.0
kul
Kyenele
667
0.99
0.0
kgn
Kaonde
500556
0.99201
0.00014
0.00011
1.0
0.0
kgo
Krahn
7891
1.0
0.0
kup
Kinur
7916
1.0
0.0
0.00022
kgs
Northern Kissi
7926
0.9985
3e-05
0.20896
0.0
kyw
Kandas
3093
0.99782
6e
kay
Koorete
7827
0.9995
0.0
Karachay-Balkee
19397
0.98844
0.00017
0,00022
kri
Krio
200310
0.96654
0.00138 38
0.96875
0.0
Kinaray A
7942
0.983766
05
Karelian
194
0,57895
3c-05
0,0875
0.0
km
Kurnkh
7898
0.9995
0.0
ksh
Shambala
0.995499
8c-05
ksc
Southern Kalinga
7833
0.999
3e 05
ksd
Kunnua
7947
0.99649
6c-05
Raña
0.99548
0.0
ksh
Kölsch
10080
0.96923
0.00014
be 05
0.00022
0.0
ksp
Kaba
4842
0.99735
6c-05
ksr
Bornng
9572
01 0.9995
3c-05
0.00033
Southem Kisi
6e
0.00241
ksw
S' gaw Karen
149668
1.0
0.0
0.00033
kth
Kambaata
7788
0.9965
8c-05
kij
Plapo Krumen
7841
0.9985
65
ktm
Kurti
2425
0.98947
0.00014
Kuot
3c-05
kra
Kinaha (Democratic Republic of Congo)
18217
D.99448
6e-05
6o-05
0.79322
0.00635
kua
Kuanyama
473208
0.93245
0.00358
0.0004
0.00022
kub
Kutep
7926
0.909
3c-05
kud
'Ashelawa
7895
0.997
8c-05
0.00011
Kumnan (Papua New Guines)
3c-05
kuj
Kuria
7933
0.9995
0.0
0.00022
kum
Kumyk
10568
0.99548
3c-05
0.00022
kup
Kunimaipa
1.0
0.0
kus
Kusaal
7926
0.998
3e-05
0.00109
Table 15: Performance of GlotLID-M -M on GlotLID-C test, FLORES- and UDHR benchmarks (part 11)

---

GloILID-C
FLORES-2000
UDITR
1so639-3
Language
Sentences
Fit
FPR.
F11
FPR.
FI1
FPR.
kvj
Psikye
7908
1.0
0.0
kvn
Border Kuna
7926
1.0
0.0
Kwaio
7876
0,9995
3c-05
kwf
Kwarn'
9038
0.9975
3c-05
kwi
Awa-CCaiquer
7920
0,999
0.0
6e-05
0.84716
0.0
kwj
Kwanga
kwn
Kwanguli
304624
0,99552
0.00022
6e-05
0.00022
kwy
San Salvador Kongo
471082
0.99352
0.00028
0.0004
0.00022
kxc
Konso
7905
0,9985
0.0
kxm
Northern Khmner
7912
1.0
0.0
kye
Kyaka
7891
0.998
8c-05
kyf
Kouya
7797
0,9995
3c-05
kyg
Keyaguna
9517
0.999
6c-05
kyq
Kenga
7902
1.0
0.0
0.00022
kyu
Wester Kayah
0.0
kyz
Kayabí
7550
1.0
0.0
kze
Kosenn
7704
0.9965
0.00011
kzf
Du'a Kaili
6792
0,99649
6c-05
kzj
Coastal Kadazan
6159
0.95996
0.00047
0.00033
Kokola
2619
0.82961
3e-05
0.00011
laa
Southern Subanen
10
lac
Lacandon
7923
0.999
3e-05
lad
Ladino
11721
0,96319
0.0
0.90598
0.00044
lai
Lambya
7869
0.99548
3c-05
laj
Lango (Uganda)
7941
0.99549
6c-05
0.00011
lam
Lamba
38968
0,98941
6c-05
0.00023
lao
Lao
26289
0.9995
0.0
1.0
0.0
1.0
0.0
(Togo)
7903
0.0
lat
Latin
217775
0.97705
0.00069
0.975
0.00011
lav
Latvian
345532
0.99551
0.00019
lbb
Label
667
1.0
0.0
lbe
Lak
218
0,98462
0.0
0.00044
Ihj
Ladakhi
0.0
0.00017
lbk
Central Bontok
9400
0.996
0.00014
Ich
Luchazi
236
0.64286
0.0
0.00114
0.00011
lem
Tungag
9544
0.99599
65
Idi
Laari
61299
0,9935
0.00017
6e-05
Idn
Ländan
144
0,97872
0.0
lea
Lega-Shabunda
3823
0.9907
6c-05
0.00011
0.00011
led
Lenda
4500
0.99485
85-05
0.00022
Ice
Lydlé
7873
0.999
36-05
6c-05
lof
Lelemi
7901
0.998
0.0
lch
Lenje
0.99249
0.00011
lem
Nomaande
7917
0.9995
3e-05
leu
Kara (Papua New Guinen)
7869
0.9995
0.0
lew
Ledo Kaili
7859
0.997
8c-05
lex
Luang
9630
0.9995
0.0
Lenghian
421
Ifn
Lingua Franca Nova
22272
0,97358
0.00028
0.00011
Igm
Lega-M MMenga
7945
0.99551
0.00017
lhi
Lahu Shi
7872
1.0
0.0
0.00022
Thm
L.ommi
7812
0.9995
0.0
hu
Lahu
73972
1.0
0.0
lia
West-Central Limba
7904
1.0
0.0
0.94643
0.0
lid
Nyindrou
9513
1.0
0.0
0.00011
Limba
15688
0.0
lij
Ligurian
36632
0.97681
0.00041
0.99901
6e-05
0.49785
0.0127
lim
Limburgan
141486
0.96436
0.00127
0.99253
0.00011
lin
Lingala
1856585
0.98225
0.00088
0.99901
0.00011
0.99145
0.00022
lip
Sekpele
7899
0.9995
0.0
Liberian English
24782
0.53173
0.01958
lit
Lithuanian
2813062
0.97971
0.00085
0.99951
0,0
0.9375
0.00088
liv
lip
Lampung Api
7900
0.99448
3e-05
[kt
Lakota
22
Lolo
30215
0.98327
0.00091
0.00023
0.00011
lld
Ladin
1049
0,9589
0.0
0.53465
0.00153
Iln
Lele (Chad)
2151
0.90074
6c-05
6c-05
0.0012
Imk
Lamkang
7953
0.998
3e-05
0.00011
Imo
Lombard
62982
0.97
0.00083
0.99554
0.00011
0.00919
Imp
Limbum
7927
0.999
3e-05
lob
Lobi
7937
0.999
0.0
0.92857
0.00022
loe
Saluan
137
0.86667
0.0
log
Logo
7844
1.0
0.0
Table 16: Performance of GlotLID-M on GlotLID-C test, FLORES-200 and UDHR benchmarks (part 12)

---

GloLLID-C
FLORES-200
UDHR
150639-3
Language
Sentences
F1"
FPR
FI
FPR.
FRI
EPR.
lol
Mongo
7934
0.997
0,00011
lom
Loma (Liberis)
8918
0,96124
0,00014
loq
Lobula
3745
0.99573
3e-05
lou
Louisiana Creole
22
loz
Lozi
961751
0.99303
0,00033
0.00011
0.84892
0.0
lsi
Lashi
7897
0.9995
0.0
Ism
Snamia
7940
0,9985
3e-05
6c-05
0.00077
tg
Latgalian
14697
0.97178
6c-05
0.99653
0.0
Luxembourgish
133586
0.98798
0,00028
0.99951
0,98305
0.0
lua
Lulua
1138848
0.99353
0.00003
0,99653
05-05
0.71
0.00175
lub
Labu- Katanaa
651814
0.99552
0,00025
0.00285
luc
Luvale
598110
0.998
8c-05
0.00074
0.99145
0.0
lug
Ganda
297207
0.98406
0.00055
0.99653
0.0
0.9589
0.00066
lun
Lunda
394558
0.9985
3e-05
0.00023
0.81752
0.00033
luo
Luo (Kenya and Tanzania)
562230
0.99155
0.00041
1.0
0.0
0.00088
lus
Lushai
568212
0.99301
0.00025
0.99653
6c-05
0.93548
0.00088
lut
Lushootseed
59
1.0
0.0
lvs
Standard Latvian
3176411
0.96455
0.00182
0.99655
0.00034
0,94118
0.00164
two
Luwo
7810
0.999
3e-05
lww
Lewo
7830
0.999
6e-05
lzh
Literary Chinese
17606
0.95639
0.00014
lzz
Laz
75
0.125
3e-05
San Jerónimo Tecoatl Mazateo
23769
0,9995
0.0
mad
Madurese
8060
0.9985
6c-05
0.00028
0.92174
0.0
Mafa
7943
0.998
3e-05
mag
Mugahi
6208
0.962004
3e-05
0.95459
0.00136
0.75385
0.00044
mah
Marshallese
532466
0.99651
0,00019
0,96063
0.00055
mai
Maithili
32796
0.95805
0.00017
0,97366
6c-05
0.83099
0.0
maj
Talapu De Díaz Maratec
7883
0.999
3e-05
mak
Makasar
7860
0.9985
0.0
mal
Malayalam
737267
1.0
0.0
1.0
0.0
1.0
0.0
mam
Mam
244791
0.99601
0,00017
0,93913
0.00022
maq
Chiquihuitän Mazatee
7930
0,989
3e-05
mar
Marathi
1382828
0.9896
0.00055
0.0
0,99174
0.00011
mas
Masai
31306
0.999
0.0
0.00055
mau
Huautla Mazatec
197845
0.9985
6e-05
0.00372
Sateré Maww
15290
0.85929
0,00171
0,0023
maw
Mampruli
7890
0.999
6c-05
0.00296
North Moluccan Malay
427
0.9011
0.0
maz
Central Marahua
9655
0.93082
0.00055
0,83582
0.00186
Westem Bukidnon Manobo
7852
0.9985
3e-05
mhe
Macushi
9275
0.92423
0.00022
6e-05
0.00022
mbd
Dibabawon Manobo
7818
0.9945
0.00014
mbf
Baba Malay
7930
0.99449
8c-05
6c-05
mbh
Mangseng
7897
0.9995
3e-05
mbi
Tianen Manobo
7894
0.9985
6c-05
mbj
Nadëb
7842
1.0
0.0
mbl
Maxakali
7908
mbs
Sarangani Manobo
9330
1.0
0.0
mbt
Matigsalug Manobo
78S8
0.9995
3c-05
0.00044
Maca
7939
1.0
0.0
mch
Muchiguenga
7743
0.999
3e-05
0.00033
med
Sharanahua
7472
0.9995
3E-05
0,97521
0.00022
mef
Marsés
7847
1.0
0.0
1.0
0.0
Mbundd
157207
0.99451
0.00022
0.00119
0.00788
men
Masana
30987
0. 1.99551
0.00014
6e-05
mco
Coatlán Mixe
230569
0.98645
0.00028
0.00449
mep
Makaa
13175
0.999
3c-05
6c-05
0.00033
moq
Ese
7924
1.0
0.0
Cameroon Mambila
7894
0.9995
0.0
nda
Mada (Nigeria)
7931
0.999
0.0
mdf
Moksha
86
0.5
3c-05
mdy
Male (Ithiopia)
37003
1.0
0.0
Melpa
7510
Mengen
7874
0.9995
3c-05
mch
Southwestern Tlaxiaco Mixtee
1543
0.10196
0.00014
0.00022
mej
Meyah
7842
0.999
0.0
mek
Mekeo
7799
0.999
3e-05
Mende (Sierra Leonc)
114881
0.99649
3c-05
0.9771
0.00033
moq
Merey
7903
0.998
8c-05
mer
Meru
3946
0.95789
0.00011
meu
Motu
19511
0. 0.98073
0.00014
mev
Mano
390
0.95726
3e-05
0.00011
mfa
Pattani Malay
319
0.925
0.0
0.00011
mfe
Morisyen
445479
0.99006
0.00044
0.00022
Table 17: Performance of GloLLID-M on GlotLID-C test, FLORES-200 and UDHR benchmarks (part 13)

---

GlotLID-C
FLORES-20
1s0639-3
Language
Seentences
FPR
FPR
F17
FPR.
Matal
1.0
mfi
Wandala
7930
1.0
0.0
0.00011
mfk
North Mofu
7937
0.99346
3c-05
mlq
Moha
9584
0,9965
8c-05
0,00045
0.88722
0.00164
mly
Mayo
10521
0.99548
3e-05
0,00011
mfz
Mabaan
7874
1.0
Marokado
1827
0.99911
0.0
mgh
Makhuwa- Meetto
78022
0,997
0.0004
mgm
Mambac
209
0.68421
6c-05
0,00011
ngo
Meta'
111
0.92
0.00022
ngr
Mambwe-Tange
164338
0.99007
0.00047
0,00182
0.00022
igv
Matengo
70
0.6
0.0
mhi
Ma'di
0,9985
3e-05
mhl
Mauwake
7578
0.999
3e-05
Eastern
22422
0,98593
0.00025
0,00022
mhw
Mbukushu
609
0.97727
0.0
0,000288
0.00011
mhx
Maru
7918
0.9995
Ma' anyan
7889
0.99448
3c-05
mib
Atatláhuca Mixtee
7887
1.0
0.0
Mi'k Mi'kmag
0,9995
0.15625
mie
Ocotepec Mixte:
7899
1.0
0.0
mif
Mofa- Gudur
7875
0.9985
3e-05
mig
San Miguel EI Grande Mixtee
7940
0,9995
0.0
0,00011
mih
Chayuco Mixtes:
7897
0.998
0.00011
mik
Mikasuki
104
0.72727
mil
Peñoles Mixtee
7894
0,9995
3e-05
0,00022
Hkaang
0.99248
0.00014
0.87591
0.00175
mio
Pinotepa Nacional Mixtec
7914
0,9985
6e-05
miq
Mískito
86121
0.97908
0,00069
0.832
0.0
mir
Isthmus Mixe
7505
1.0
0.0
mit
Southern Puebla Mixtec
7779
0.999
3e-05
miy
Ayutla Mixtec
7915
1.0
miz
Coatzospan Mixtec
7953
1.0
0.0
0.00011
mjc
Juan Colorado Mixtec
7915
0.9995
mjw
Karbi
7953
0,9995
0.0
mkd
Macedonian
809994
0.99253
0.00033
1.0
0.0
0.99174
0.0
mkl
Mokole
7866
0,9995
mkn
Kupang Malay
9069
0.998
3c-05
Silacayeapan Mixtec
7949
0.9985
3c-05
mkz
Makasae
1720
0.98266
0.0
6c-05
0.00033
mlg
Malagasy
30062
1.0
mih
Mape
7925
0,9985
6c-05
mlp
Bargum
7729
1.0
0.0
0.00017
0.00011
mlt
Maltese
2281035
0.97815
0.0008
0.97401
0.00307
0.77419
0.00383
mlu
To' abaita
1036
0.99375
3c-05
6c-05
0.00011
Mamanwa
0.999
3c-05
mmo
Mangga Buang
7937
0,9985
3e-05
mmx
Madak
10379
0.999
Mbula
13167
0.998
3c-05
mmb
Muna
7924
1.0
0.0
0.00022
Manchu
mnf
Mundani
7866
0,9995
0.0
Manipuri
48249
0,9899
0.99901
mnk
Mandinka
7913
0.9985
3c-05
0.00011
mnr
Mono (USA)
Mon
mnx
Manikion
7376
0.9995
3e-05
mny
Manyawa
50297
0.97712
0.00017
mou
Mwan
7939
0.9995
0.0
16176
0.9995
Mongondow
7903
1.0
0.0
meb
Mohawk
953
0.99678
Mongalian
102788
0.98521
0.0008
mop
Mopán Maya
8965
0.999
0.0
Moro
1.0
0.9916
0.0
mos
Mossi
626622
0,9925
0.00019
0.98138
0.0
0.97015
0.00044
mox
Molima
7830
0.998
6c-05
mpg
Marba
7909
0.9985
3e-05
mph
Maung
598
1.0
mpm
Yosondáa Mixtec
7900
0.999
6c-05
0.00011
mopp
Migabac
3959
1.0
0.0
mps
Dadibi
30278
0.9995
mpt
Mian
7692
0.999
3e.05
0.00241 0.00022
mpx
Misima-Panacati
8788
0.998
3c-05
mqb
Mbuko
7873
0.9995
0.0
Table 18: Performance of GlotLID-M on GlotLID-C test, FLORES-200-00 and UDHR benchmarks (part 14)

---

GloLLID-C
FLORES-200
UDHR
is0639.3
Language
|eentencss
F11
FPR
FI1
FPR.
F11
FPR
mqq
Mamasa
8069
0.992
0.00022
0.00011
mggy
Manggarai
7953
3e-05
mig
Mising
6373
0.9945
0.0
0.00022
mri
Maori
79437
0,98948
0.00025
0.99901
6c-05
0.8227
0.00022
mrj
Western Mari
10083
0 99041
0.0
0.00011
mzq
North Marquesan
1007
0.96774
0.0
6c-05
0.00011
Mangareva
187
0.8
0.0
mrw
Maranao
7932
0.9985
3e-05
Malay (macrolanguage)
53819
0.998
3o-05
6c-05
0.00011
msh
Mushatenya
7900
0,99549
8e-05
msc
Sunkaran Muninka
1783
0.98641
0.00077
mse
Musey
7917
0.9975
3e-05
msk
Mansaka
7775
0.99101
0.00028
msm
Agusan Manobo
7776
0.995
0.00014
mey
Aruamu
7694
0.9995
3--05
mta
Cotabuto Manoho
7857
1.0
0.0
nig
Una
7834
0.999
0.0
mii
Maiwa (Papua Guinea)
7899
0.99699
3e-05
mtj
Moskona
7846
0,9985
6c-05
mto
Totontepec Mixe
7906
1.0
0.00088
mip
Wichi Lharntés Nocten
7934
1.0
0.0
0.00011
mua
Mundang
7908
0,9995
0.0
mug
Musgu
30824
0.99451
0.00019
mub
Mündu
7469
1.0
0.0
Murle
7816
0.999
0.00011
mus
Creek
598
0,9172
8e-05
mux
Bo-Ung
7692
0,999
3e-05
Muyang
7876
0,9995
30-05
0,00011
mva
Manam
7947
0.999
0.0
Minaveha
7827
0 0,99699
0.0
mvp
Duri
7814
0.996
0.0001
Tagal Murut
nwe
Are
1150
0.99342
3c-05
mwf
Murrinh-Patha
316
1.0
mw]
Mirandese
33797
0.99247
8c-05
0.00033
mwm
Sar
30621
0.999
6e-05
Nyamwanga
54396
0,99347
6e-05
0.0004
0.00011
nwp
Kala Lagaw Ya
4301
0.9984
3--05
nwg
Mün Chin
7927
1.0
0.0
0.00022
nwv
Mentawai
7901
0.999
3--05
mww
Hmong
0.99346
mxb
Tezoatáán Mixtee
7938
1.0
0.0
0.00022
mxp
Tlahuitoltppee Mixe
7924
0,9995
0.0
0.00011
mxq
Juquila Mixe
7933
1.0
0.0
0.00011
mxt
Tamiltepec Mixtec
7916
0.9975
3--05
Metlatónoc Mixtec
141566
0.99151
0.00028
0,149925
0.0
Burmese
498977
0.998
0.00011
1.0
0.0
0,66292
0.00646
myb
Mbay
7908
0.00011
myk
Mamara Senoufo
7920
0.999
3e-05
myu
Mundarukú
7683
0,9995
3e-05
myv
Erzya
18314
0.992
0.00022
myw
Muyuw
6727
1.0
0.0
myx
Masaaba
7954
0.999
myy
Macuna
7837
0.9995
0.0
Santa Maria Zacatepec
7940
mzh
Wichi I humité Giisnay
29905
0.996
0.00014
0.00011
mzk
Nigeria Mambila.
7917
0,9995
0.0
mzl
Mazatián Mixe
7907
1.0
0.0
0.00011
mzm
Mumuye
7945
1.0
0.0
Mazanderani
30000
0 0,96612
0.00019
0.00295
mzw
Deg
7889
0.999
6c-05
0.00077
mzz
Matadomu
652
1.0
nab
Southern Nambikuára
7609
1.0
0.0
naf
Nabak
7692
0,9995
3e-05
0.00011
nah
Nahuatl languages
212
0.68293
3e-05
0.00011
muk
Nakamui
7892
0,9995
0.0
Min Nan Chinese
37894
1.0
0.0
nan
Min Nan Chinese
37894
1.0
0.0
0.0
0.00011
nap
Ncapolitan
10002
0.97646
mag
Khoekhoe
106437
0,9995
0.0
mis
Naasioi
7985
1.0
nau
Nauru
11
naY
Navajo
91313
0.999
6c-05
0.9916
0.00011
Nawuri
7948
1.0
0.0
nba
Nyemba
162410
0.98861
0.00058
0.00227
Table 19: Performance of GlotLID-M M on GlotLID-C CDDD-Ceee test, FLORES-200 and UDHR benchmarks (part 15)

---

GloLLID-C
FLORES-200
UDHR
1906.99.3
Language
Sentences
FPR.
FI1
FPR.
FI
FPR.
ube
Chang Nagu
7899
0.998
6c-05
nbe
Konyak Naga
7951
09995
ubl
South Ndehele
238555
0.9679
0.0008
0.D
0.00055
nbq
Nggem
3714
1,0
0.0
0,00033
nbu
Rongmei Naga
30767
0.99198
0.00017
nca
Tyu
7866
0.999
3e
Central Huasteca Natuad
191991
0.97793
0.00116
0.00668
ncj
Northern Puebla Nahuall
192101
0.99303
0.0003
0,00033
Michoasin Nahuati
7934
0.998
6c-05
Naga
7936
ncu
Chumhurang
7654
1.0
0.0
Central Puebla Nahoail
161081
0,97718
0,00085
0,00011
197046
06
nde
North Ndebele
234610
0,96552
0.00196
0.00273
0.00624
ndh
Ndali
743
0,96257
35-05
0.00068
0.00011
Samba Leko
7924
ndj
Ndamma
7955
0,9965
0.00011
6c-015
0.00022
ndo
Náoaga
481932
0.92504
0.0008
0.00136
0.86765
0.00197
ndp
Nãoo
7946
1.0
0.0
nds
Low German
128898
0.98218
0.00077
I.D
0.D
Ndogu
neb
Toura (Cãte d'Ivoire)
7864
1.0
0.0
Nepali (macrolanguage)
137937
0.99649
new
0.00011
nta
Dhac
7649
1.0
0.0
Nafamnaa
7930
0,9855
ngh
Nghandi
nge
Ngombe (Democratic Republic of Congo)
7941
0,99601
0.00017
Lomwe
177993
0,97725
0.00094
05
0,00033
ng
Ngulu
7956
0.97885
8.00039
0.00011
Kriong
19
Nabunll
118905
09935
0.00117
0.00011
nhd
Chiripá Pastern Huasteca Nahmoti
15822
0.83343
0.00033
nhe
7941
0.94
0.001665
ahg
Tetekcingo
Zacutlán Ahuacatlán Tepetzintla Nahuat]
7911
0.99749
0.0
nkk
Lahanus- Lohnnss-Cssleen Cosolewaque Nahuall
4466
0.8986
0.00014
0,00055
aho
3e
nhr
Naro
7906
1.0
0.0
nhu
Noose
7919
0.999
3e-05
Wester Hunstecn Nahnatl
7942
0.00146
nhx
Lahmus Mecayapan Naluatl
9954
0,999
6e 05
ahy
Northern Oaxaca Nahuarl
7931
0.996
0.0001
mi
Nius
205435
0.9985
6e-05
0.00022
nif
Nek
3713
1.0
0.0
nj
Nguju
7879
0.99399
0.00011
nim
Nilamba
7948
0.998
3c-05
6c-05
Ninzo
nig
Nandi
35203
0.9975
Re-05
0.00427
Niueun
565376
0,9975
65
1.0
0.0
Gilyak
myy
Ngiti
7950
1.0
0.0
Nocte Naga
7858
0,9955
3e-05
njm
Angami Naga
30987
039669
nja
Liongmai Naga
30075
0,99502
0.00025
njo
Ao Naga
30939
0.997
6e-08
0,95312
0.0
njz
Nyishi
7935
0.9995
0.0
0.00142
nka
Nkoya
73
0.36164
0.0
Thangal Naga
0.00011
0.00055
nko
Nkonya
7825
0.9995
3e 05
Ngombalc
312
0.92105
3c-05
0.00022
Nalea
old
Dunch
4208335
0.76911
0.01112
0.99803
0.00023
0,70238
0.00547
Orizaba Nabuall
14
nma
Naga
7934
unf
Tangkhul Naga (India)
30939
0.996
Sc-05
6e-05
umh
Monsang Naga
7951
0.9985
0.0
mmo
Moyon Naga
7950
0.9985
6c-05
0.00044
nmw
Ninuu
1220
1.0
0.0
Naudm
7W74
0.00033
mub
Nunde
80966
0.98394
0.00033
0.00336
0.00109
nng
Maring Naga
7955
1.0
0.0
anh
Ngiemboon
nnl
Northern Rengma Naga
30056
0.9925
0.00022
Table 20: Performance of GlotLID-M on GlotLID-C test, FLORES-200 and UDHR benchmarks (part 16)

---

GloLLID-C
FLORES-200
UDHR
1506399-3
Language
|Sentences
FI1
FPR.
F17
EPR
FIT
FPR.
mno
Norwegian Nynorsk
463877
0,98099
0.00036
0.95077
0.00045
0.95808
0.00033
amp
Wancho Naga
7948
1.0
0.0
0,00011
nng
Ngindo
7940
0,99649
nnw
Souther Nuni
7874
1.0
0.0
Woun Meu
15760
0.9985
0.0
0,00011
nob
Norwagiam Bokmil
2890508
0,96898
0,00129
0.98185
0.00148
0,98462
0,00022
nog.
Nogai
9743
0.99649
36-05
non
Old Norse
nop
Numanggang
12584
0.9985
6e 05
Norwegiam
1000000
0.997
6c-05
Nomatsiguenga
7819
0,97391
nou
Ewage-Notu
7758
0.9995
0.0
0.00053
Novial
430
0.71579
0.0001
nph
Phom Naga
7947
0,999
mpi
Nepali (individual language)
99103
0,98559
0.00058
0.99104
0.00011
0.98214
0.0
npl
Southeaster Pucbla Nahuatl
0,99601
0.00017
upo
Pochuri Naga
7954
0,999
26-05
Napu
7750
0.9985
36-05
Rengma Naga
off
Jerriais
1066
0,98339
0.0
0,00022
Chokri Naga
7954
0,99498
65:05
nsa
Sangtam Naga
7931
0.0
me
Naenga
106988
0.99047
0.00011
0,9004
0.00011
Sumi Naga
30168
0.9965
0.00011
msn
Nehan
7887
0.999
3e 05
nso
Pedi
2010451
0.99253
0.00033
0.99704
0.00028
0.86957
0.00197
nst
Tase Naga
30918
0.9975
3e-05
Sierra Negra Nahuail
7903
0.90599
36-05
ntp
Tepehnan
2753
Delo
7896
0.9995
0.0
0.00088
Nuer
16408
0.9985
0.0
0.99951
0.0
0.00011
muy
Nunggubayu
8201
0.998
36-05
nvm
Namine
7255
1.0
0.0
Nyabwa
7725
0.9985
nwi
Southwest Tanna
7454
0.9995
0.0
Middle Newar
11272
0,99649
6c-05
nxd
Ngando (Democratic Republic Congo)
7948
0.9975
aya
Nyanja
2582911
0,93803
0.0033
0.99753
0,00023
0,966114
0.00099
nyf
Giryama
8144
0.98947
0.00622
6c:05
nyk
Nyuncka
297246
0.98166
0.00074
0,0016
0,00066
nyn
Nyunkole
236252
0.98166
0.00074
65:05
0.85938
0.0
ayo
Nyoro
7946
0.98286
0.00025
nyu
Nyungwe
176938
0.99052
0.00833
nyy
Vyakyusa-Ngonde
110073
0.90102
0.0003
0.00034
0.00055
Njebi
70
0.00011
nzi
Nzima
384511
0.9985
8c-05
1.0
0.0
Zeme Naga
30978
0.9975
35:05
oar
Old Aramaic (up BCE)
22
obo
Obo Manoho
7651
0.9975
36-05
Occitan (post 1500)
135269
0.98549
0.00039
0.99951
6c-05
0.41101
Old Frisian
13
Khana
61553
0,999
6c-05
ujb
Notthwester Ojibwa
26949
0.70851
0.01862
0.81481
oji
Ojibwa
7989
0.999
0.0
Severn Ojibwa
7937
0,00066
Okpe (Southwestern Edd)
8c-05
0.00011
okv
Orokaiva
7780
1.0
0.0
0.00011
Mochi
7955
0,98808
0.00052
0.00023
0.000154
omw
South Tairora
9414
0.9995
0.0
ong
Olo
788
0.9995
3e-08
ons
15076
0.00011
nod
Tahono O" D'odham
7563
0.99699
0.0
6e 05
opm
Oksupmin
7657
0,999
0.0
Oriya (macrolanguage)
100397
orm
Oromo
489160
0.9995
3e-05
6c-05
Old Russian
1307
0.94602
36-05
uty
Odia
122527
0.9995
26-05
1.0
0.0
Old Spanish
23
Ossctian
630164
0.99502
0.00028
0.00679
ata
Omomnn Turkish 1920)-122)
2287
0,81188
Rc-05
0.00267
0.00011
Mezquital Otomi
$8835
0.92369
0.00237
0.9
0.00572
otm
Highland
7870
otn
Tenanoo Otomi
7870
1.0
0.0
0.00011
oig
Querélaro Osomi
7943
0.0
Table 21: Performance of GloULID-M on Glot test, FL.RRE--000 and UDHR benchmarks (part 17)

---

GlotLID--
FLORES-200
UDIER
3506339-3
Language
[Sentences]
F11
FPR
F1
FTR.
FT1
FPR
ots
Estado de Mexico Otomi
13242
0,95737
0.00041
0.0046
otw
Ottawa
108
0.00044
oym
Wayampi
7693
0.9995
3c-05
ozm
Koonzime
7891
pab
Parects
7936
0.9995
3c-05
pad
Paumarí
7771
pay
Pangasinan
1255155
0,99452
0.000125
0.99852
0.0
0,00033
nah
Tenharim
7699
0.9995
pam
Pampanga
19422
0,98532
6c-05
1.0
0.0
pan
Panjabi
722260
1.0
1.0
pao
Northem Paiute
7028
0.999
6c-05
pap
Papčamento
1601687
0,97782
0.00102
0.99069
0.00102
0,79195
0.00339
pau
Palauan
181209
0,99751
0.00011
0.97436
0.0
phb
12673
0,81626
0.00333
0.7125
0.00482
phc
Patamona
7939
0.9995
0.0
phi
Parkwa
7849
0.00011
phl
Mak (Nigeria)
7924
0,38695
0.00242
0.00011 1
pht
Southern Pashto
63256
0,96663
0.00061
0.81486
0.00051
0.00109
ped
Picard
1348
0,89552
0.0
pok
Paite Chin
30968
0,97284
0.0011
0.00033
Nigerion Pidgin
8364
0.97864
0.00011
0.71739
0.0
pdc
Peunsylvania German
11954
0,94379
0.00066
pdt
Plautdietsch
152305
0.998
8c-05
pom
Phende
9968
0,99649
3c-05
0,00034
0.00044
pes
Iranion Persian
2814370
0.80032
0.01356
0.57435
0.08493
0.65922
0.00668
pfe
Pere
10404
0.9995
3c-05
0.00044
pf
Pfaelzisch
10003
0,96516
0.00028
0.00011 I
phm
Phimbi
45602
0,98943
0.00011
0.00017
0.00011
phn
Phocnician
15
pib
Yine
7937
0.999
0.0
pid
Piaroa
7255
0.47033
0.00085
pio
Plapoco
7655
1.0
0.0
Piratapuyo
7740
0.9995
3c-05
pis
Pijin
703022
0,998
0.00011
0.9916
0.0
Pijjanjatjara
10949
1.0
0.0
0.00544
pkh
Pokoma
7824
0.9985
6c-05
0.00022
plg
Pilagá
8729
0.0
0,00011 1
pli
Pali
2
pls
Sun Marcos Tlacoyalco Popoloza
23439
0,86827
0.00286
0.00011 I
plt
Platean Malagasy
202954
0.99552
0.00022
0.99852
0.0
0.98182
0.0
plu
Palikár
8749
0.999
0.0
0.00011
0.0012
plw
Brooke's 's Point Palawano
7940
0.9995
3c-05
Paama
15067
0.999
65-05
pmf
Pamona
7956
0.9985
0.0
Piemontese
30824
0,98993
8c-05
0,00033
pmx
Poumei Naga
30182
0.99549
6e-05
pnh
Western Panjahi
300035
0.9762
0.0003
0.00102
0.65969
0.00668
pne
Western Penun
7928
0.999
0.0
poc
Sun Juan Atzingo Popoloca
19275
0.9995
0.0
pob
Pogomchi'
41239
0.998
0.0
0.0001 I
poi
Highland Popoluca
17342
0.7267
0.00613
0,00011
pol
Polish
4592867
0.9621
0.00187
0.9907
0.00108
0.7362
0.00471
pon
Pohupciaa
431877
0.997
0.00011
1.0
0.0
por
Portuguese
5403043
0,82411
0.01166
0.99655
0.0004
0.84806
0.00471
pot
Pocawatemni
2078
pov
Upper Guinea Crioulo
28501
0,99147
0.00014
0.00233
0.96552
0.0
poy
Pogolo
7955
0:9975
ppk
Uma
7760
0.999
3e-05
ppl
Pipil
0.42105
0.0
ppo
Folopa
7679
1.0
0.0
pps
San Luis Temalacayuca Popoloca
7950
0.9995
3c-05
0.00055
prl
Paranan
7865
0.9985
0.0
prg
Prussian
1088
0.96067
0.0
pri
Paici
6986
1.0
0.0
prk
Parauk
8909
0.63062
0.00144
prs
Dari
93926
0,80441
0.00083
0.14688
0.0192
D.0
0.00547
pse
Central Malay
7905
0.9985
pip
Patep
7918
0.999
0.0
ntu
Bambam
7808
0,99649
0.00077
pua
Western Highland Purepechu
7950
0.52827
0.00077
pus
Pushto
30046
0.88156
3e-05
pwg
Gapapaiwa
7914
0.9955
0.00019
pww
Pwo Northern Karen
30880
1,0
0.0
qub
Huallaga Huánuco Qucchua
64459
0.81356
0.00715
6c-05
0.02704
que
K' K'iche'
173500
0,99402
0.00025
0.95575
0.0001
Table 22: Performance of GlotLID-M on GlotLID-C test, FLORES-200 and UDHR benchmarks (part 18)

---

GlodLID-C
FLORES-200
UDHR
is0639-3
Language
Sentences|
FI1
FPR.
FI1
FPR
F11
FPR
que
Quechua
694846
0.97263
0.0014
0.9916
qui
Lambayoque Quechua
8072
0.98582
3e-05
6c-05
0.00394
qug
Chimborazo Highland Quichus
160594
0.84767
0,00787
0.0375
0.0
quh
South Rolivian Quechua
46618
0.8818
0.00861
0.00028
0,92174
0,00444
qui
North Bolivian Quechua
15852
0.77368
0,00223
6c-05
0,00011
qup
Soubhen Pastaza Quechua
7700
0.99701
0.00014
0.0093
que
Samiago del Estero Quichua
6855
0.99649
0.0
0.00208
que
Tema Lowhad Quichux.
33203
0.67937
0,01062
0.00055
Ayacucho Quechua
457040
0.97855
0,00066
0.75904
0.0
0.67403
0.00646
quz
Casco Quechua
347402
0.944
0.00281
0.02176
0,54822
0.00898
qua
Amho-Pasco Quechua
7824
0.98297
0,00041
0.00011
gve
Cajamanca Quechua
7799
0.998
6c 05
6e 05
0.92035
0.00033
que
Eastern Apurimac Quechun
7924
0.98167
0.0
quh
Huamalles- Dos de Mayo Huámico Quechua
7304
0.98799
0.0
0.00022
qvi
Intubura Highland Quichua
227687
0.97923
0.00088
0.00US5
qum
Margos- Yarewilca- Lauricocha Quechma
7299
0.9824
0.00033
0.15385
0.0
qun
North Junin Quechuua
7948
0.9945
0.00017
0.31884
guo
Napo lolland Quechua
7853
0.99198
0.00117
0.00U11
qus
San Martin Quechua
7872
0.99649
05
0,00252
quw
Huaylla Wanca Quechua
0.77036
0,00239
0.00055
qyz
Northemn Pastaza Quichua
8730
0,94558
6c-05
0.00011
qwh
Hury las Ancush Quechmu
7834
0.997
6e-05
0,48276
0.00016
quh
Panan Hudinmoo Quechmn
7727
0.99247
0,00011
gsl
Salssaca Highland Quichun
7876
0.9955
30-05
gan
Northern Conchucus Ancush Quechua
7937
0,98087
0.00033
0.03333
0.00011
quo
Souther Conchucns Aneash Quechuna
7421
0.97364
0,00088
Qushgaii
14
gat
Caior Highland Quichua
9639
0
0.00025
qya
Quenya
0.28571
05
rad
Rade
30874
0.9955
6c-05
Rameanina
7828
0,9985
0.0
rap
Rapunui
0.57556
0.00776
0.00011
ma
Rarotongan
920894
0.98958
0.000S
0.99174
0.00011
Réumion Creole French
13290
0,98891
8c-05
0.00045
0.00022
rbg
Rohingya
38.00
0.96712
05
0.00028
0,00142
ria
Riang (India)
7947
0.9985
0.0
nl
Tariiit
227
0.43137
60-05
Nyaturu
7954
0.9995
rkb
Rikbakısa
7766
0.999
3e OS
emc
Carpathian Romani
8938
0.99649
66-08
rme
Angloromani
0.76923
0.0
ral
Baltic Romani
4828
0.73838
0.00028
6e-
0.00077
mnn
Balkan Romani
338459
0.997
86-05
Ge-D5
0,86636
0.00011
Romani
0.90749
0.0
0.00011
rmq
Cald
2273
0.99541
0.0
rny
Vlax Romani
96254
0.91895
0,00184
6e-D5
0.00394
Ruund
30-05
0.00011
mg
Ronga
77778
0.9853
35
0.00044
ml
Ranglong
10406
0.994
0,00017
Romansh
0.00017
0.99268
rom
Romany
876
D.44602
0,00828
mon
Romanian
1542662
0.98327
0.00091
0.99951
0.0
0,80992
0.00493
Rotokaa
7890
0.0
rop
Kriol
29167
0.998
60-05
Waima
7946
0.999
Se 05
0.00011
Rotuman
09985
0.0
0.00011
rub
Gungu
7912
0.999
0.0
Rusyn
10117
0.95084
8e 05
Lugara
7951
0.90649
60405
6c-D5
run
Rundi
1361196
0.96652
0,00094
0.92541
0.00881
0.87591
0.00186
2219
0.99573
35
0.125
0.0
Russian
9074266
0.65157
0.02904
0.99901
6e-05
0.43321
0.01718
rwo
Rawa
15237
0.9995
36-05
Central Okinawan
42
sab
Buglere
7507
0.999
0.0
sng
Sango
1017526
0.99552
0,00025
0.99901
0.0
0.81553
0.0
sth
Yakut
115824
0.98601
0.0004]
0,53881
0.01106
Sanskrit
147128
0.97656
0.00072
0.99104
6e-05
0.66667
0.0
438
Saaak
7890
0.9985
$6-05
531
Saniali
21927
1.0
0.0
1.0
0.0
shu
Ngumbay
30900
0.9995
0.0
Southern Samo
7909
0.9975
36.05
0.00011
she
Saliba
6215
0.998
6c-05
Botolan Samhal
7848
0.0
Subiya
0.83871
0.0
Table 23: Performance of GloMLID-M on GlolLID-C test, FLORES-200 and UDHR benchmarks (part 19)

---

GlotLID-C
FLORES-200
UDHR
3.0639..3
Language
Sentences
FI7
FPR.
FI1
FPR.
FIf
FPR
shy
Soli
208
0,95833
0.0
0.00022
sck
Sadri
29
sen
Sieilian
62549
0,96888
0.00074
0.99802
6i-05
0.00263
sco
Scots
101211
0,97563
0.00025
0.00028
0.92683
0,00044
scu
Toraja-Sa SSt dam
8864
0.99198
0.00017
0.00011
sdh
Southern Kurdish
1048
0,93968
0.0
sdo
Saduug Bidayuh
787
0.98851
3c-05
seh
Senu
301781
0,98462
0.00034
ses
Koyraboro Scnni Songhai
12790
0,9955
0.00011
sey
Secoya
7912
1.0
0.0
0.02985
0.00044
sfw
Schwi
10131
0.99548
0.0
0.00033
sgb
Mag-anssi Ayta
7857
0.998
8c-05
sgh
Shughni
717
1.0
0.0
0,00077
sgs
Samogitian
10047
0,98889
3e-05
6e-05
ngw
Sebat Bet Gurage
7909
0.99749
0.0
sgz
Sursurunga
3c-05
shi
Tachelhit
10669
0,98226
0.00011
0.00011
stik
Shilluk
7907
1.0
0.0
0.88889
0.0
shn
Shan
24569
1.0
0.0
L.0
0.99145
0.0
shp
Shipibo-Conibo
9806
0,9995
3c-05
0.27397
0.0
shr
14876
0.98656
0.0005
O 0.00011
shs
Shuswap
95
0,8125
0.0
shu
Chadian Arabic
7923
0.998
6c-05
0.00835
0.0012
shy
Tachuwit
247
0,27907
0.0
sid
Sidamo
130089
0.999
6c-05
0.90625
0.00131
sig
Puasaal
0.998
3e-05
0.00011
Tumulung Sisaala
7868
0,99649
0.0
sim
Mende (Papua New Guinea)
7790
0.9995
3e-05
0.00033
sin
Sinhala
554048
0.0
0.0
sja
Epena
7870
0.9995
0.0
sjn
Sindarin
0.33333
3e-05
skg
Sakalava Malagasy
56311
0,9955
0.00011
0.00011
0.00022
skr
Saraiki
134
0,51429
3c-05
sld
Sissala
7906
0.999
6c-05
0.0035
slk
Slovak
3544374
0,97476
0.00099
0.99852
6c-05
0.86957
0.00197
Salt- Yui
7869
0.999
3e-05
slv
Slovenian
4072739
0,96881
0.0016
0.99459
0.00062
0.88889
0.00164
sma
Southern Sami
59
0.83333
0.0
smne
Northern Sami
18205
0.99448
6c-05
6c-05
0.96667
0.00022
smk
Bolinao
7859
0.9985
3e-05
sml
Central Sama
7891
0.9995
0.0
6e-05
Samoan
1640628
0,99352
0.00025
0.99603
0.0
stn
Simte
7953
0.99247
85-05
s0a
Shona
2150482
0.98521
0.0008
0.99901
0.00011
0.93846
0.00088
suc
Sinaugoro
7926
1.0
0.0
snd
Sindhi
132171
0.9985
0.0
0.99362
0.00074
suf
Noon
7907
0.99599
3o-05
0.0012
snn
Sioma
7892
1.0
0.0
0.60773
0.0
sap
Siane
15669
0.999
0.0
0.00022
Selce
7890
1.0
0.0
suy
Saniyo-Hiyewe
7848
1.0
0.0
soe
Songameno
0.96988
0.00011
som
Somali
227769
0,99649
6e-05
0.96657
0.00398
0.75817
0.00405
sop
Songe
208326
0.9955
0.00011
0.00011
0.00055
soq
Kanasi
10512
0.9985
3e-05
0.00022
sot
Southern Sotho
2131930
0.99305
0.00039
1.0
0.0
0.98333
0.00011
sny
Miyohe
0.0
0.00099
spa
Spanish
2583672
0,50519
0.05363
0.99508
0.00057
0.71681
0.00679
spl
Selepet
7031
0.9995
3e-05
spm
Akukem
1622
0.99755
0.0
spp
Supyire Senoufo
7847
0.9985
6c-05
sps
Saposa
0.959
0.0
spy
Sabaot
15366
1.0
0.0
sqi
Albunian
326340
0.999
6e-05
srd
Sardinian
53845
0.93574
6c-05
0.99951
sri
Siriano
7808
0.9985
0.0
scm
Saramaccan
75703
0,9975
6c-05
0.00055
Sranan Tongo
1166639
0.99107
0.00047
srp
Serbian
1390294
0.98142
0.00039
0.99901
0.00011
0.5124
0.00657
srq
Sirionó
7814
0.0
SIT
Sever
3524
0.99191
0.0
0.89231
0.00131
ssd
Siroi
10880
0,9965
0.00011
55g
Seimat
7891
0.998
3c-05
SSW
Swati
426339
0.97691
0.00052
0.99654
0.00023
0.94891
0.00066
68X
Samberigi
1.0
0.0
0.00011
Table 24: Performance of GlotLID-M M on GlotLID-C test, FLORES-200 and UDHR benchmarks (part 20)

---

GloLLID-C
FLORES- 200
UDIT
1506399.3
Language
|Sentences]
F11
FPR]
FIt
FPR
FI
FPR
stn
Owa
7884
0.9985
6c-05
stp
Southesstern Tepehuan
7918
1.0
0.0
0,00022
siq
Saterfriesisch
10507
0.99147
0.00014
sua
Sulka
7558
0,999
36-05
suc
Suhanon
7896
sue
Suena
7882
0.998
Se-05
suk
Sukuma
8083
0,99599
3c-05
6c-05
0.68712
0.00514
sun
Sundanese
109886
0.98094
0.00044
0.99012
0.00057
0,9697
0.00033
sur
Mwaghavul
7952
0.999
3e-05
sus
Susu
11639
0.99449
8c-05
6c-05
0.92683
0.00088
sux
Sumerian
183
0.92754
0.0
suz
Sunwar
30868
0.9985
Swahili (macrolangange)
100000
0,998
8c-05
6c-05
swb
Maore Comorian
1412
0.97387
0.00011
0.79137
0.00274
Congo Swahili
452844
0,9025
0.00454
0.00409
0,00285
swe
Swedish
3997074
0.96724
0.00154
0,99754
0.00028
0.86301
0.00219
swy
Swahian
9915
0.97342
0.00011
0,00011
swh
Swahili (individual language)
370928
0,9401
0.00267
0,94869
0.00187
0.84956
0,00044
swk
Malawi Sena
7727
0.98992
65-05
swp
Suau
11467
0.997
0.00011
sxb
Suba
7906
0,998
0.0
0,00011
sxn
Sangir
51443
0.00014
syb
Central Suhanen
7644
0,999
3e-05
syc
Classical Syriac
7926
1.0
0.0
0,00635
syl
Sylheti
15
szh
Ngalum
7940
1.0
0.0
Silesian
57496
0.99247
0.00011
0,99104
6c-05
tab
Tabussaran
7851
0.999
3e-05
tac
Lowland Tarahumara
11398
0.9985
tah
Tahitian
1185188
0.99255
0.00039
0.91892
0,00011
taj
Eastern Tamang
7884
0.9995
0.0
0.66667
0.0
Tamil
1581134
1.0
1.0
0.0
0.0
tap
Thabwa
216
0.875
0.0
0.00011
taq
Tamashey
24410
0.90069
0.00325
0.80642
0.02022
0,00438
tar
Central Tarahumara
25433
0.98077
0.00019
0,00022
tat
Tatar
372101
0.98657
0.00052
1.0
0.0
0.65556
0,00668
tav
Tatuyo
7676
0.9975
8c-05
taw
Tai
7683
0.9985
6c-05
tbe
Takia
7836
thg
North Tairora
19510
0,999
6e-05
tbk
Calamian Tagbanwa
7653
1.0
0.0
thl
Tholi
7806
0.9985
0.0
0.00011
tho
Tawula
7895
0.99701
0.00014
0.00011
tby
Tabaru
7878
0.9995
30-05
thz
Ditammuri
30712
0.9995
3e-05
0,20896
0.0
tca
Ticuna
0.81432
0.00393
0,9916
0.0
tee
Datooga
7953
1.0
0,0
tef
Malinaltepee Me' phaa
125443
0.997
0.00011
0,00022
Torres Strait Creale
12298
0.99046
0.00014
tey
Tulu
10000
0.98219
0.0
tez
Thado Chin
44548
0.00011
tdt
Tetun Dili
450685
0.99056
0.00044
0.66292
0.00646
tdx
Tandroy- Maaiaalyyaalllyyyyyaaa Malagasy
73631
0.9975
6e-05
6c-05
ted
Tepo Krumen
7812
0.999
6c-05
tee
Huehuetla Tepehua
7938
1.0
0.0
Telugu
634652
6e-05
1.0
0.0
tem
Timne
7951
1.0
0.0
0.97345
0.0
tco
Teso
36835
0.99451
0.00019
0,00044
Tereno
8019
0.999
3e-05
tet
Tetum
9162
0.99197
0.00011
tew
Towa (USA)
4831
1.0
ufr
Terihe
7534
0.9995
0.0
ggk
Tajik
232287
0.99599
3e-05
0.67429
0.00624
Tagalog
1391946
0.85393
0.00897
0,99901
0.00011
0.9403
0,00077
ago
Sudest
8042
0.9995
0.0
gg
Tanguaa
7927
0.999
0.0
tha
Thai
883065
0.99502
0.00028
1.0
0.0
1.0
0.0
thk
Tharaka
7890
0.9995
30-05
0.00011
thy
Tahaggart Tamahag
589
0.66038
0.0
0.00182
0.00011
Tifal
7682
0.9985
0.0
Tigre
3874
0.74834
3e-05
0.00125
tih
Timugon Murut
7913
0.9995
3e-05
Tikar
7900
0.9995
tim
Timbe
7791
0.9995
0.0
Tigrinya
917442
0.89658
0.00624
0.98851
0.0
0.0
Table 25: Performance of GlotLID-M on GlotLID-C test, FLORES-200 and UDHR benchmarks (part 21)

---

GloLLID-C
FLORES-200
UDHR
150639-3
Language
Sentences
FI
FPR.
FI7
FPR.
FI
EPR
tiv
Tiv
481805
0.9965
0.00011
0.98551
0.00022
tiy
Tiruray
7896
tke
Takwane
10656
0,99498
6c-05
0,00011 I
dkl
Tokelau
629
0,91026
0.0
dkr
Tsukhur
912
0,98932
0.0
tku
Upper Necaxa Totonac
7936
0.99303
0.0003
Tabelo
7902
0.999
3c-05
0.00011
tif
Telefol
7676
0.9995
3e-05
tlh
Klingon
55689
0,99498
6c-05
6c-05
Talinga-Bwae Bwisi
7885
0.999
3e-015
tll
Totela
584747
0,99304
0.00036
0.00017
0.00011
dy
Talysh
66
0,57143
0.0
0.03175
0.0
mdd
Haruai
7725
1.0
0.0
tor
Jewish Babylonian Aramaic
218
0.0
Temuan
toa
Tacana
7705
0.999
3c-05
toe
Taninuca-Retuad3
3025
0.0
ink
Kwarnera
7870
1.0
0.0
ton
North Tanna
7865
0,9975
3c-05
mp
Whitesands
7864
0.998
8c-05
tob
Toha
39496
0.99599
6e-05
0.97561
0,00033
toe
Coyutla Totonac
7915
0.9985
6c-05
0,00011
tod
Toma
11025
0.996
8c-05
tog
Tonga (Nyasa)
231197
0,99699
3e-05
0.00011
toh
Gitonga
107233
0.995
0.00014
toi
Tonga (Zambia)
746307
0,98961
0.00058
0.00011
1.0
0.0
Tojolabal
189077
0,98953
0.00036
6c-05
0.83688
0.00022
tok
Toki Pona
52772
1.0
0.0
0.00044
ton
Tonga (Tongu Islands)
1234253
0,99502
0.00025
0.0
too
Xicotepec De Juárez Totonac
7940
0.99297
8c-05
top
Papantla Totonse
238408
0,99601
0.00017
0,97561
0.00033
tos
Highland Totonac
7906
0.998
3c-05
0,00022
pp
Taupota
656
0,94792
8c-05
tpi
Pisin
1846778
0,99155
0.00039
0.99951
0,98361
0,00011
pp
Tampulma
30663
0.997
8c-05
0,00066
pp
Pisaflores Tepehua
7926
0.9995
3e-05
0.00011
pl
Tlachichiloo Tepehua
7927
0.9995
0.0
6c-05
rp
Tupí
563
0.90566
6c-05
0.00011
lp
Tinpuix
7846
0.0
ngb
Tombé
30084
0.48998
0.01293
tre
Copala Triqui
7881
1.0
0.0
tm
Trinitaroo
7840
1.0
0.0
0.00022
tro
Tarao Naga
7952
0.9995
0.0
tpp
Kok Borok
30861
0.998
8c-05
ing
San Martin Itunyoso Triqui
7937
1.0
0.0
tsc
Tswa
297653
0,99097
0.00017
0.00011
tsg
Tausug
7892
0.9995
3c-05
tsn
Tswana
799821
0.94537
0.00303
0.99753
6e-05
0.98361
0.00022
tso
Tsonga
2723082
0,97416
0.00143
0.99803
0.00023
0.94158
0,00153
tsw
Tsishingini
7902
0,99699
3e-05
tsz
Purepecha
132907
0,74587
0.01735
0.81944
0.00274
Tektiteko
7954
1.0
0.0
0,00208
tte
Bwanabwana
7734
0,99551
0.00017
Tooro
106342
0,97983
0.00102
0.00017
0.00175
uq
Tawallammat Tamajaq
2766
0.98647
0.0
tts
Northeastern Thai
55
0.0
0.00011
tu
Mutu
15211
0.9995
3c-05
tu
Tuyuca
7812
0.999
0.0
uf
Central Tunebo
7875
0.9985
3e-05
tui
Tupuri
30989
0.998
3e-05
tuk
Turkmnen
696538
0,99601
0.00017
0.99803
0.00023
0.94821
0.00142
tum
Tumbuka
808156
0.99551
0.00019
0.99852
0.00011
tuo
Tucano
15602
1.0
0.0
Turkish
2439747
0,92159
0.00465
0.9907
0.00108
0.4918
0.01357
tuv
Turkana
237
0,89655
3c-05
0.00153
tvk
Southeast Amhrym
0.9985
0.0
tvl
Tuvalu
520271
0.99206
0.00044
twi
Twi
1954311
0,98668
0.00074
0.99951
0.0
0.95349
0.00131
Termunu
7903
0.9975
0.00111
tWX
Towe
31794
0.98376
3--05
Exq
Tii
9062
0.997
8c-05
IXUI
Kayapó
7661
1.0
0.0
tyv
Tuvinian
147493
0.9945
0.00014
0.98361
0.00022
Tzeltal
223502
0.98949
0.00228
0,98333
0.00022
trj
Tx' utujil
17101
0.99352
0.00025
0.00011
Table 26: Performance of GlotLID-M M on GlotLID-C CD test, FLORES-200 and UDHR benchmarks (part 22)

---

GlotLID-C
FLORES-200
UDHR
1so639-3
Language
Sentences
FI
FPR]
FI
FPR.
FI7
FPR.
Ixl
Talossnn
343
0.775
3e-05
tzm
Contral Atlas Tamazight
8142
0.88564
0,00292
0.94421
0.0054
0.01754
0.00569
Iz0
Tyntzil
5213363
0.9985
8c-05
0.97499
0.00011
ubr
Ubir
9424
0,9975
8c-05
ubu
Unmbu-Ungu
15261
0,9985
3c-05
udm
Udmmutt
95670
0,99053
0,00036
0.00022
udu
Uduk
7952
0.9995
0.0
0.98361
0.00022
uig
Uighur
118987
0.9995
0.0
0.99901
0.00011
0,89916
0.0012
ukr
Ukrainian
2374463
0,95992
0,00212
0.99951
6c-05
0,98361
0.00022
umb
Umbundu
646290
0.99301
0.00022
0.88585
0.00045
0.87931
0.00109
upv
Uripiv-Wals Wala-Rano-Atchin
7916
0.998
6c-05
ura
Ururina
7752
0.9985
3e-05
0.82963
0.0
urb
Urubú-Kasn Kaapor
7742
0,9995
0.0
urd
Urdu
775141
0,97065
0,00143
0.98346
0,00187
0,96522
0.00077
urh
Urhobo
181379
0,99302
0.00028
0.00011
uri
Urim
2487
1.0
0.0
urk
Urak Lawoi"
7911
0. 0.9995
0.0
urt
Urat
7816
0.9985
0.0
Usarufia
8134
1.0
0.0
usp
Uspanteco
7898
0.999
0.0
uvh
7690
0.9995
0.0
uvi
Lote
7908
0.999
0.0
uzb
Uzbek
181303
0.999
6c-05
Northern Uzbek
1516837
0,99054
0.00039
0.96885
0.00364
0,48819
0.00788
vag
Vagla
7938
0.9995
0.0
0.00011 1
vap
Vaiphei
30918
0,98504
0.0005
0.00011
var
Huarijio
7954
0.9975
3c-05
vec
Venetian
124915
0,96296
0.00099
0.99703
6c-05
0.86957
0.00197
ven
Venda
806164
0.998
0.00011
1.0
0.0
vep
Veps
13
vgt
Vlaamse Gebarentual
9618
0.78372
3c-05
vid
Vidunda
7943
0,99154
0,00036
0.00017
0.00022
vie
Vietnamese
2010052
0,99303
0.00033
0.99951
6c-05
0.66304
0.00011
viv
Iduna
7521
0.9985
3e-05
yls
Vlaams
30000
0,97166
0.00044
0,00023
0.00011 1
vmk
Makhuwa Shirima
1970
0.88306
0.0
0.00011
Makhuwa
306018
0,97205
0.00132
0,00011
0.95798
0.00022
vmy
Ayautla Mazatec
7941
0.999
0.0
0.00263
val
Volapiik
105178
0,99751
0.00014
0.00044
vro
Vóro
10015
0,98739
0.00011
0.00328
vun
Vunja
7951
0,99145
8c-05
vut
Vite
7912
1.0
0.0
0.00328
waj
Waffa
7854
0.9985
0.0
wal
Wolaytta
309841
0,98961
0.00058
0.00033
wap
Wapishana
14453
0.9985
0,0
0.00099
war
Waray (Philippines)
606273
0,99305
0.00039
0.99951
0.0
0.9916
0.0
Kaninuwa
2596
0.99868
3c-05
Wayana
7923
0.9995
0.0
wba
Warao
388
0.97521
3c-05
0.00044
whin
Wa
30852
0.76979
0.01416
0.00197
wbp
Warlpiri
10006
1.0
0.0
wea
Yanomárni
7788
0,80093
0,00803
wed
Wodau
1309
0,96658
0,00019
wer
Weri
7854
1.0
0.0
0.00011
Cameraon Pidgin
91652
0.94163
0.00267
0.00252
wew
Wejewa
884
0,98893
0.0
whg
North Wahgr
977
0.9699
3e-05
0.00011
whk
Wahau Kenyah
7945
0,99548
3c-05
wih
Southern Toussian
6982
0.9995
3e-05
wim
Wik-Mungkan
7697
1.0
0.0
wiu
Winu
7809
0.9995
0.0
0.00099
Walloon
30053
0,99198
0.00014
0.62105
0.00777
wls
Wallisian
880578
0,99255
0,00039
wlv
Wichi' Lhamtés Vejaz
1809
0.9899
3--05
0.00033
wmt
Walmajarri
1.0
0.0
wmw
Mwani
7940
0,99549
8c-05
0.00022
wne
Wantoat
10243
0.999
0.0
Usan
7720
1.0
0.0
wob
WE Northern
7924
0.9985
3c-05
Wolof
79608
0,97988
0.00039
0.99852
0.79747
0.0035
Hunga Hundi
7811
0.999
0.0
0.00011 1
wrk
Garrwa
4838
0,99862
3c-05
Waris
8985
0.999
0.0
wsk
Waskia
13069
0,99701
0.00014
Chinese
104765
0.8768
0.00033
0.00011
0.35811
0.01905
Table 27: Performance of GlotLID-M M on GlotLID-C test, FLORES-200 and UDHR benchmarks (part 23)

---

GloLLID-C
FLOORES-20
UDHR
1506699-3
Language
|Sentences
F11
FPR]
F17
FPR.
F11
FPR
wuy
Wuvulu-Aun
7938
0.998
wwa
Waama
7901
0.9985
0.0
0.88636
0.0
Kalmyk
8727
0.99147
0.00014
0,00022
xav
Kavánte
24284
0.78285
0.00803
0,00022
xhi
Komhio
4526
0,99924
xhr
Kambera
7547
0.999
0.0
xed
Hdi
7945
0.9985
3e-05
0,00011
xho
Xhosa
1011828
0,98372
0.00183
0.99118
0.08097
0.93846
0,00088
xla
Kamula
9641
0.9995
3c-05
xmf
Mingrelian
124528
0.99303
0.0003
xmv
Antankurana Malagasy
76882
0.99699
3e-05
6c-05
0.0001 1
xnn
Northern Kankanay
7735
0.99649
3e-05
xog
Soga
25897
0.99249
0,00017
xon
Konkomba
15023
0.99651
0.00019
xpe
Liberia Kpelle
31074
0.55157
0.00919
xya
Karakhanid
xeb
Eastern Karaboro
7887
0.9995
3e-05
xsb
Sombal
7915
0,997
0.00011
xsi
Sio
10302
1.0
0.0
0.00044
Kasem
7874
0.9985
05
xsr
Sherpa
7935
1.0
0.0
xsu
Samuna
7788
0.76697
0.00371
0.00011 1
xtd
Diuxi- Tilantongo Mixtee
8182
0.9985
35-05
0,00044
xtm
Magdalena Peñasco Mixtec
7929
0.9985
0.0
0.00427
xtn
Northern Tlaxiaco Mixtee
14968
0.9985
8c-05
xuo
Kuo
7884
0.999
3c-05
yaa
Yaminahan
yad
Yagua
7454
1.0
0.0
0.88889
0.0 0.00055
yal
0.998
yam
Yamba
7927
0.9995
0.0
yan
Mayangna
30722
0,99449
8c-05
yau
Yao
204519
0,99701
0.00014
6c-05
0,97345
0,00022
yap
Yapose
352203
0.99651
0.00017
0.96721
0.00022
yaq
Yaqui
7934
1.0
0.0
ylab
Yemba
8028
0,91398
0.00828
0:00197
ytyy
Yaweyuha
7883
0,999
0.0
yen
Yacuna
7857
0.9995
0.0
ydd
Eastem Yiddish
911
0,99661
0.0
0.99603
0.0
0.99187
0.0
yid
Yiddish
44101
0.99497
0.0
yim
Yimchungru Naga
30883
0.99548
3e-05
ykn
Yakun
8455
0.9975
3c-05
5kg
Northern Yukaghir
yle
Yele
7585
0.9995
0.0
6c-05
Angguruk
7905
0.999
3e-05
0.00088
yml
famalele
7317
0.9985
0.0
yom
Yombe
73077
0,98856
0.00047
6c-05
0.00646
yon
Yongkom
11798
0.9995
0.0
0,00022
yor
Yoruha
1812160
0.99206
0,00041
0.99406
0.08023
0,85106
0.0023
yth
Yareha
7117
0.9975
Re-05
yre
Yacuré
7908
0.0
yrk
Nenets
1908
0,99112
35
0.8381
0,00011
Nhengatu
7439
0.4322
0.00022
yss
Yossan-Mayo
15808
0.999
6c-05
yun
Yocateco
616290
0.99701
0.00014
1.0
0.0
yue
Yue Chinese
64647
0.9549
0.00022
0.00394
0.0
0.46897
0.00449
ytj
Karkar- Yuri
7845
0.9995
0.0
yup
Yukpa
.299
0.96078
05
0.90077
yut
Yapoa
10245
1.0
0.0
yuw
Yau (Morobe Province)
0.999
6c-05
yuz
Yuracare
7826
1.0
0.0
yva
Yawa
7651
0.9995
3e-05
0.00011
zan
Sierra de Juárez Zapatec
7891
1.0
0.0
zab
Western Tlacolula Valley Zapotec
9862
0.99549
6c-05
zac
Ocotlán Zapotec
7941
0.9995
0.0
rad
Cajonos Zapotez
7938
1.0
0.00033
zae
Yareni Zapolee
7916
0.999
65-05
0,00011 1
Isthons Zapotec
276740
0,98903
0.00039
D.00055
zam
Mishuatión Zapotee
7943
0.9995
3e-05
0.0
0.0001 I
zao
Ozolotepee Zapotee
7940
0.998
3e-05
0.00109
7ar
Rinzón Zapatec
0.9975
705
Santo Domingo Albamadas Zapotec
7928
0.9995
3c-05
D.00131
Tabaa Zapotec
ZaV
Yatzachi Zapatec
11233
0,87575
0.00999
D.0008S
ZOW
Mida Zapolee
7951
0.9995
2Ca
Coatecas Altas Zapotec
7938
1.0
0.0
0.00022
Table 28: Performance of GlotLID-M on GlotLID-C test, FLORES- and UDHR benchmarks (part 24)

---

GlotLID-C
FLORES-200
UDHR
1so639.3
Language
[Sentences]
F1
FPR]
FIt
FPR
F11
FPR.
Ngazidja Comorian
2097
0.97484
6e-05
0.00011
0,78125
0.00208
zea
Zecuws
10012
0.91037
0.00025
0.00045
zgh
Stundurd Moroccan Tamazight
3285
0,93333
0.00039
0,00068
0,17647
0.0
zho
Chinese
42 42222
0.998
0.0
1.0
0.0
Zia
7893
0.99749
2ow
Zigula
7957
0,97857
0,00069
Malay (individual langunge)
ome
Zande (individual language)
315310
0,99053
0.00036
0,00219
20s
Francisen León Zoque
7953
0.999
0.0
2pa
Lachiguiri Zapote:
67840
0.75362
0.01389
ppe
Choapan Zapotec
7914
0.999
0.0
2pd
Southeastern Ixilán Zapotec
106
0.82427
3e-05
0.0023
zpt'
San Pedro Quiatoni Zapotec
94
1.0
0.0
0,00208
2pg
Gueven De Humbolds Zapotec
1020
0,98462
zpi
Santa María Quiegolani Zapotes:
7856
0.999
65-05
0.00011 1
2nj
Quiwizuzzs 7apotes:
124
0,89362
0.0
0.00077
Lachixio 7apotec
7953
0.999
3e-05
2pm
Mixtepes: Zapolec
7695
0.9995
0.0
0.00033
2po
Amatlán Zapolec
7913
0.9955
3e-05
0.00033
Zoogocho Zapotec
7889
0.99649
3e-05
2pt
San Vicente Coatlán Zapotec
up
Yalálag Zapotec
7935
0.999
0.0
2gV
Chichicapan Zapotec
pp
Texmelucun Zapotec
7936
0.9995
0.0
2sm
Standurd Malay
0.00307
75T
Southern Rincun Zapotec
7918
0.998
0.0
Ziq
Quioquitani- Quierí Zapotec
7898
1.0
0.00011
Yaloe Zapotee
7908
0.999
6e-05
zul
Zulu
990448
0.99295
0.0
0.96893
0.0
0.95305
0.0
zyb
Yongbei Zhuang
10631
0.9985
0.0
0,912
0.00066
Zyp
Zyphe Chin
7934
0.999
0.0
0.00011
Zaza
1713
0,936
Table 29: Performance of GlotLID-M on GlotLID-C test, FLORES-200 and UDHR benchmarks (part 25)

---

with ondfilens thnshold
GleLLID-M
CLDS
GieLIDD- 6
GMLLID-M
CLDJ
0-.5
CLDS th.7
2053933
FLORES Colcs)
CLD3 Codeso
FIT
EPR
FET
FPR
FII
FPR.
F11
FPR]
FIT
FPR
FIT
FPR
300
200
0.0100
0.07793
0.56663
0.0
0.88422
0.00275
0.9199]
0.0055
anth
a0th
0,9991
0
04661
0.01066
0.99551
0.0011
0.59555
0.0001
0.6579
0.0041
0.56579
0M3281
0.0001
0.8545
010137
019325
0.0512
0.9233
0,04535
0.99253
0.01776
0.90924
D.0055
19:11
18
0/0
100
0.0
1,0
0.0
312
0.0000
000006
059057
0.00310
and
2n5
389511
0.00012
000011
0/5193
077188
01000
045576
0000011
0/10019
OL0TER
DOUSN
tek.
buk
10
0.0
100
6.0
1.0
0.0
tom
hom
0.1122
8,58864
han
0000011
0:96827
0.00034
0.98972
0.00021
DUOOOEE
IL91156
the
MIO
0,00613
0.00012
1
0.0%
0.00099
063852
000011
698846
059851
0.001
0,5005
0,02053
DOOD
bal
bad
0.00012
055589
0.001
bsa
hor
bug
bos
000000
000062
0100518
0
00
0
099951
0,93006
0.00141
036413
0.00073
0,59051
0,00034
13
0011
00
Lo
110
ceb
ccb
0,0
035456
0.06C81
80
00
061137
0.00995
0.56434
0,00908
coc
oos
000001
0000
DAOBEE
099388
000005
gj
qk
0.00011
0.000
00
0.00077
0.22922
0.0
0,98
00
000
dan
Jan
059951
09515
0.00044
0,00012
003422
0.84123
0.003%
(99951
003011
0A0022
039554
1.0
0.00010
0.0
055022
0.9947
0.0005
0
0
0.5071
DLNIESS
0000046
drn
den
0.0
00
00000
0.94633
0,00112
aya
890%
0/1221
0991
BL0002
ell
el Lanue
18
0,0
019811
00000110000000
0.00257
1,0
6.0
1.0
0.0
0.94352
0,0012
036691
D000066
epo
430
m
60
09901
0,977
0,00025
OUOGME
0,00007
000011
10001
0.0
009911
8.99856
1.
0,091
0.
098150
085121
0000
BE1000
00000
19000
10002
1.0
00
0,0
0L99881
0
3:59951
0.0
fas
poopes
0.12542
0,62221
0,02444
080002
0.13173
8.33343
0,06448
0.64500
0,22366
0.99001
0.019ED
0.1711
0.1419
098825
0.00000
0L90783
000
00
00
039951
0,00012
0.12909
0.00EE
0.99511
0
0,99832
0.0001
035541
0,00445
0
try
R00225
0.0066
fir
09955
0.00112
000013
1.
00
0.00012
094693
0.93112
0000013
0442742
010466
0.0
1.0
00
0.00113
alc
1.0
0,
0:92995
0.00162
10
0.0
1.0
0.0
0.5524
0,00001
0.576
di
AL99THE
076147
gm
gmn
3.11
100
00
10
0:97003
0.0
059700
ha
0.95924
0
0.46655
0
0.9844
000076
0.00150
haw
icb
0,
000835
4:45
10
0.0
1,0
B0OSSS
O G.W00
3:46
hin
bin
N--aamm
0.11
0.107%
0.0008
BAWRR
0 22572
hmn
000002
0L00014
0.90296
0.00555
C0048T
0.855
0,035
br
1825187
00349
0.26
lun
1u
1.0
0,
0.1104
0,08222
L0
60
359951
039599
0,00235
DUO02TS
eye
by
1.1
00
100
100
BO
io
10
000001
00
0000001
000011
000013
00000
0A2016
00001
ipe
gn
La
WV
019661
0188122
10
008883
00007
00
035638
069627
00156
10
00
10
063136
00000
000300
001244
000045
kah
tab
000000
las
0,0
1.0
0,0
00
10
00
00
0.9915
lus
kas
0.97643
0,0
0.0
0.0
bat
1.11
QVO
B0
10
00
1,0
099354
0
0L99354
BO
taz
14
0.0
0.71835
000031
00
0.99511
0.72531
0.00788
0 TLTO
sbg
1op
0.00012
000013
0.0001
05413
065366
thm
khm
0199334
0.0
(195511
0.0
0.99511
0.0
0.9504
lik.
0.06776
(196509
0.07734
096456
BL00572
kin
kin
0.91771
0.00074
0L9177L
0.00065
0.1471]
BL00055
0.
0A3745
0101114
10
1.0
0.01181
inb
kub
0.03901
0.96923
0.0664
8.37713
0.0038
trc
(19649)
0,85222
(19639)
6.50966
kon
0.99151
0,00111
0.9910
0.0088
0,99233
0.00019
kee
ker
In
1
0,0
099657
Ro48
1.
0.0
1,0
0.0
0198805 0597266
4p-15 0,00025
2-006
iyo
1ao
16
0,0
8.0
0.95001
0.0
0.99803
0.0
0.97726
0.00342
Table 30: Comparison of GIotLID vs CLD3 on FLORES-200 benchmark (part 1)

---

with confidence threshuld
GhaLID-M
CL.33
Glol.ID-M IH_3
Clel.ID-MMrs.
Am.5
CI.B
0-.5
CLID3 Wm07
3 FLORES Codeest
CLB3 Cudels)
F11
FPR.
FPR
F1]
PPR
P11
FPR
F17
FPR.
F1"
FIR
Issfig
1v
0,99081
0.00012
0.80002
0.00348
8,99775
0.0
0.99051
0.0
0.91429
0.00225
0.2211
0.00122
18
[L9990]
0.00012
0,99852
1
0.99755
0.0001
100011000
lam
lin
IL59253
8.0
0,92553
00
0.9915)
0.0
lan
lin
[159901
0.000225
0398522
000002
0.0001
11
181
[15995]
0.0
0,75767
0,8951
0.
0
0.0
0.84719
091057
0.00199
18e
I159554
0.00025
0,9335
0.0002
0.0001
1tx
122
1b
[L5995]
8.0
0.00133
0
0.0
0.96011
0.00066
017821
0L00643
lua.
lua
1159455
0.000012
086553
0.0
00
lug
11g
0.0
00
luo
100
1.0
0.0
1.0
00
00
lus
mag
mag
345
IL55459
0.00012
0,95987
0.89703
000023-
0U
039005
0,0012
0.0
mal
mai
mal
m13
ml
IL97366
1.0
0000012
0
00
1.0
0000011
00
039710
1:0
0d
0.9651
0.00
0.0
mar
100
1.
0.0
10
2.0
0.39555
mli
mkd
mlg
mid
ml)
pli
TH
mp
nk
087731
1,0
0000122
0.0
0
000000
00000000000000000000000000000000
0
10
0 10
2.0
0
8 00001
000000000000
062252
0001 16
177682
00001
000755
2e 10
mm
mni
0000122
000000
000000
00000
mon
ktk
0.0
000001
Ze 10
mn
mus
mis
mos
09900
0.97122
000012
000277
0.0
064861
00000
0.975
0511000
00
00
897478
000000
00000110000
064477
067476
00022
0.7575
nbi
mya
aso
nop
nor
mya
too
upt
my
ml 099603
0,9729
L1997044
089857
1.
0.0009
0.00002
100011
0.0
0.96373
0 803308
0.72455
01000
00000
030033
022200
098888
10220000000
10
000055
000002
000088
000072
05000
0.000
0
000000
10001
00
00
067005
0
01977
0.0007
0,0
000ERS
003687
nsa
muc
15a
ny
019753
1
0,00000
0.0
0 000552-000
0:00800
0-09551
00
0.000
1000
00
0415066
0.0293 102955
0
0.0 0.01996 0
oci
0.99911
0.00012
00
652211
00
ory
ary
1.0
0.0
00
060314
00
pag
IL99852
0.0
00
00
pan
p4n
1,0
0.0
0.
1999553
0.0
1.0
0.0
000053
0.0
pai
poc
pap
24p
D00
1199655
0.99UT
0.00086
0
0.0222
8.77611
000000
0.01110
18
000158
0001ER
0
0.91641
061774
0.01289
0
pus
241
0,88805
0.0019
0.66115
1166115000
00000010000000
0
0000
0.0088
0.0088
0770388
000069
080064
0 0.00494
quy
4U7
1776766
0.0
0.025
00
63716
coo
100
0.9951
0.0 8.82399
0.00442
0.37289
0.00297
091857
0.00172 1
cun
110
0192441
Q.01913
032627
res
1U5
nU Latoro
0.99901
0000012
0194235
000034
00000
00000100000000
096659
0.00072
0
0 0.00012
542
1
0.0
032200
00
sun
3001
0.99104
0000012
00000
sut
1.0
0.0
540
0.54602
0,99960
000
sha
3221
1
0.0
2.0
2-0
00
sin
1,0
0.0
0000000
1.0
00
000
0000003
U.0
0000100
0000000
0100000
0,00055
0.000
00000
0
000077
0000
14
097167
084554
sua
and
sum
10m
503
50
000067
0000012
801009
00220
0.0007
0090000
080011
1.0
000000
0100000
0.992
5
00224
001754
001435
usl
00001
10
0352383
00185
spa
21
063533
0
0:0111
054508
000000000000 00012
167251
up
000011
0.9961
002523
0
0.0000
090322
000117
0.99981
0n
up
000025
099753
0.00051
090902
1e-05
099653
1o-05
0
0,09455
0.99455
0,00299
sun
0.9611
000676
000099
036800
000000
0006644
0000s
0 ww0.0....
0.9635
000220
0:29567
0.6952
0.00044
0.0000
055461
1
001544
0.0165
0 0000490000
0,00445
097991
0.00020
0.99104
0.00112
00000
0.99
10
0.000
1.6
0.0
0.9908
0.99303
0,0
099803
00
bj
004454
0.81861
0.5440
0.0875
1.0
0.0
00
1.0
0.7381
0,0
0.97881
0.0
tgk
16
L.
0.0
0.00012
10
0.
0.99754
50-05
0 199901
36-05
(L99951
1.6
000012
0.06740
00
0,00900
0.0001
0
0.000
10001
0.98749
0,0
0.57449
0.0
(L98851
0.0
0,98811
41
1199753
(199951
0
0000012
0.0
1
0999533
1
1199753
099753
000001
000014
tuk
1uk
0.99951
000011
099951110
000011
1
(199652
0.99901
1
05
0.99119
000052
0
0991119
054114
0.0075
0.36000
0194421
001175
0.94221
004421100
0.0051
ukr
u8g
0.99551
1.0
0000012
0.0001
0,99951
1.0
1
7.0
00001
059409
9e 05
unb
0.88555
03781
0,0009
und
und
096346
000107
064759
000133
0,98166
000665
199121
046866
0.01005
0
0.00234
uzb
055011
0010557
0.01167
000115700000
0.96396
0.9411
0,00117
0.70282
0 1
0U0060
0.99700
0100012
0.9920
0.00011
D39655
0,
(199551
0.00012
0.7781
100
000
1.0
0.30667
0.04728
0,3033
0.0075
(.99981
0.0
0,99951
00
D.99051
0.0
wol
vnl
(L99852
0.0
0,99800
0.59822
0.0
xho
Xho
0198115
0.58763
0,0121
0,98913
0,00177
0.99113
0,00117
066438
008821
000591
yid
yol
1.0
0.0
(196791
1e-15
1.0
0.0
1.0
0.0
0.99044
1e-05
(19251
16:06
0.99855
0
0.00877
(11844)
0.03556
0.99355
0.00013
0.59053
0.0001
(11978
000662
021661
0.00533
thu
yauV/nn
Xh- anwhh
L
0.0
0,89912
0.00753
1.0
0.0
1.0
0.0
0.93855
0.04355
0.3610
0.00000
zul
096155
0.00185
0.41074
0,02900
9.96383
0,00122
0,90285
0,00085
0.4457
0102500
0.01863
Table 31: Comparison of GlotLID vs CLD3 on FLORES-200 benchmark (part 2)

---

with culldea Beeobdd
30639-
FLORES Codeto FT176 Codedo
F11
FPR
FAT
FPR.
EPR.
F21
FPR.
F1
FPR.
SPR.
800
0,0253
B0131
081
155676
0 010614
R-55601
08
0 0.59951
0 0100014
100054
109551
0000112
0
0.53345
502225
BUSSS
0028331
0J6TES
0.86122
1.0
00
30
CINSSS
0 0.50200
000113
047151
DU0003
0000KC
0100112
0,30555
2e-05
01 133851
0 010014
0.0
0
0B
0.1116
0 146056
0 0
DU03541
069627
0.38066
DO22
bal.
bok
b0
0 0.0005
S5
12
01.0
1.0
0.0
0.56754
00
0,0751
15-05
012179
059521
000014
057467
bar
bur
A-41
0.00051
ton
0,550
000113
000054
059156
0LCOOO
bax
ton
ILWAKCE
CHIESS
CONEET
st
bed
bod
to
0L54600
O 00004
0 86067
D04559
G00004
0000112
C
trc
Sc-05
10001
DUV333
tal
0 MM09T
DORRST
IDD
OD
3e415
TH-OS
or
0 067
15
RDD
10
00
0 7561
DUOONES
sh
0 SSS0
EP9454
RL5
0.0
0 2A8A
COODEE
on
ces
1.99911
0 00014
0 MUSSS
CUPISS
000004
IUSS01
000012
0.0071
DO00OES
So.05
D.SLUO
000014
075834
0000012
do
LR
CH
1.
ILD
L.D
0.
MAASS
ILS
an
000041
0.000SS
oh
0,5
0,565
0.0
qpn
055951
0 10004
0 74229
SUMISS
000014
LO
0
0LOOKS
0
IL99001
dis
4A
0 00000
ILD
0.0
de
0 LEGGS
50002
000023
0.000US
dya
DIESTA
DOTESS
dm
do
DEL
BS401
0591667
50
020115
0L000K2
60
0 DEISO
0 30007
B00204
0 6-5550
000412
0
0
5e-05
ca
On
058351
40-43
OVC
12
1.
159961
009551
055651
To
6 19655
0. 114966
0 035221
600799
0.10716
0.85491
0.00227
fj
G
0L50011
0LD
0L5500
0.
IL99752
115
0B
fi
6
0.9995
0 00004
0.40978
5/0222
109951
800054
099852
0,00012
081844
0.00075
0
0000122
in
000041
0.000SS
for
D 000014
1,
OB
no
I 14004
096945
20001
OB
0.910
2-46
0.71818
25
2%
0 DOK0S
13
D00221
0.5
D DUSSTOS
LO
000012
0.0
0.88890
0.023
0.0015
GO009E
14-05
ghr
pm
LA
0 277754
1.
ILD
1.0
52991
no
0,0ECS
20
13
L.
1.0
0.59522
0 DOON2
0 200022
GPASS
008841
2
00001
2e-05
hoo
001211
B00452
0LOOTS
ILS
OB
30
1.2
bif
0 00006
2:05
GOOSSS
213
16
0
0,2581
503373
10666
25913
003577
0,502%6
000535
14h
hon
38777
D0974
12
ILD
BU90011
OB
0.556T
0 00014
L0
0.0
D003
74-05
IL906C1
1001d
000014
D155511
000002
DOISLE
AC4S
ol
tal
0,50001
0 LEKGG
030613
0000014
0
039961
01041S
jw
0.94422
0 00033
002217
085211
00016
0.6871
010077
0,34535
35-05
goo
18
DJ57855
DU02284
EL96666
0,121
1.c
LO
0.
1.5
0.0
1.0
0.0
000014
001193
0.33366
0.00022
500077
L0
0.09551
le40
11
tu
Ioc
Cel
on
tx
1.0
10
1
1/0
0.99511
009951
099651
Sc445
lhp
0 000014
L09931
000554
039960
0.00012
lea
lea
6.0
00513
0.
002556
0.0
010655
kn
0 00066
0.000G
is
1.3
0.00552
0.99933
0LS
0
D0081S
0
trc
AL 9.9991
0 BN00S
5/00122
RDD
0D
0
0.00015
0.90193
2e-05
DO00SS
0000122
0L99259
20012
DU02109
EL99783
ber
lor
Ko
LO
C0
59001
2:055
12
IDD
L.0
00
100
300
1.3
06
lon
50004
009931
0.00117
2-005
bn
boogg
Iv
ILD
QB
CU00554
101
0L59901
(99952
000004
0199353
0.00012
lin
019935
231
2:45
(99153
0.5
0.9155
0.0
0.20008
0.0
0.00875
0,
In
In
(159901
0 0000S
(098552
000022
(U5501
0.00012
0 100ET5
0 2.7242
5/01131
0.91551
D105337
(U5556
0100022
1121
by
Iv
1045
Table 32: Comparison GlolLID vs FT on FL.RES- benchmark (part 1)

---

with coufddance thrshald
GloLLDD-M
FTI76
GlALID 6-33
GioLLDD.M 5
FT176
6-3
FT176 1
FLORES Cideoo
FT17 Cadeso
FPR.
P1
FPR
F4y
FPR
F1-
FPR.
FPW.
PPW.
12.
0.99151
00
DL00011
0.99901
0.0
0.9590)
0.0
0.94996
0.0
0.50035
D.
113
100
0.99655
0.00014
0.99555
0.0
0159024
0.0
344
34a
0.99083
00
0.996SB
0.0
0,99454
0.0
1mn
0.0
0.90911
30
0.9K65
0
8,99703
0.99633
0.97366
0.00044
100014
0.3334
0.97366
0.00044
0.97102
0,0015
0.32554
le 15
0.1099
0.0
ml
10
0.0
0.0
10000
1.0
1.0
0.0
0.0
1.0
1.0
0.0
0.0
0.99511
0,500
110002
le485
0.9655
10
To45
mod
mid
mk
1B
00
1.8
0.0
1.0
0.0
1LMIS
fe-05
alg
0J0004
023112
0.59951
000014
0,99551
0
R51311
2:005
0.20655
0.0
m.
09777K
0.
001 In
0.99716
0.98913
0
30-t5
075569
aui
0.99411
000011
0.99901
000004
0.59901
0J00012
099961
0.09511
0:8118
00
0,97415
0.0
0.9621
ani
0.99001
0.00014
8.99551
0.0
0.99903
0.0
inj
Te
una.
0,91122
0.0290
0.52247
0,05449
899915
0.01221
0,97476
0.01333
B66591
0.01246
0L57549
0.00236
mil
mel
0.66689
1.0
0.0
0.66755
0.01006
0.6702
0,00H81
000197
040177
10000
100056
le 16
ads
76-05
aki
nep
03000
000000
0/005
110000
8.5901
000027
001012
1
0.00012
000
13 1399999
010016
0200E7
0.00072
Re-Os
asb
nno
mn
bob
000006
110004
035355
$39666
080353
0,89231
0.00
010000
0.00500
000000
0,00435
0000000
0.0
0,99551
0.0
0.998151
0U99803
040041
0.9983
0.00012
oci
000014
00007
LB
0.0
009991
000
D 0.7379
0000
043818
05
un
00
10
0.D
1,0
0,0
1.0
0.0
1.D
D.D
om
0.00799
pas
PAA
0.0
0.98852
0
0
pam
pm
gan
pan
114
0.0
0.0
0,90059
0
0002311
0000
2:45
gre
0,00007
pab
CO02A5
0000356
0.00266
0.00145
pol
20
12
6-50187
099167
0,001
4.1 18
por
por
000002
0.97044
D.1247
030409
0,00665
gas
314
QU5
0
0,3825
DO
nch
0.000
1o65
can
ton
196.
10
022551
000
0
075523
0.92584
0099511
0,0093
0.0
022022
089951
0,
0
0
344
roc
T
0
00
09:501
0
0.0
991166
san
F10000
0000044
0.91003
0.9599
05
0991399
0,99600
0.00014
0000000
0.00012
3
0.0
0,12419
DO
00005
100004
C
0.0000S
1e:05
shn
18
000
10
0.
1,0
sio
T0
000
1:45
10
0.D
LO
0,0
1.
0.0
1.0
D.0
16
D
0.99011
Se-th
0
C.000RS
0
0
0 0,00081
0,00013
000004
0,99551
0,0003
ond
000623
1U00021
0.9060
0
0.98364
0
COWNS
30-0
som
50m
056777
100065
0.98975
D 104677
28 16
D.0
sst.
514
00
1.
0.0
1O
0
rpa
100
0.99516
002812
0.0044
20
0,
0
020055
000039
and
300
30
0,9901
0.
0.
15
0.995SS
051105
8.9116
0000411
099904
000011
0.00244
041875
DO
2002
1900
699611
1
(95003
0 02380
105311
0397542
059951
0.9933
10
000545
00
0U0031
CA219
0e-05
odl
0.99106
600004
0.99004
000002
mm
13m
00
00000 1
00
1.
00
ua
0.83861
C02064
0.44499
0.90117
80
0.
18
DO
10
80
055556
10
LO
1e485
099752
DO
15
000000
0033
0.99401
000011
0.99931
00000
0
0,00531
094112
0.00119
du
12
10
00
52403
10
0.
LO
00
1
0.0
1.0
0.0
117
90
198851
00
90
mp
17
000000
0.9753
099951
0.00014
00
09970
36
0.59951
0.00014
000000
0.0
0,92394
D.0
um
0.59901
0.00114
nr
056113
0.591
(39119
0.01
067699
0L00M5
ua
140
0.94221
001291
0.4624
000002
un
00
0.
10
100
1.8
00
1.0
ulr
24
010014
036667
1000077
1000777
0,99951
0000044
0.99951
0
LA
D.D
0000933
0.98346
0.0449
0
0.00551
0
0,00991
uzb
vec
Vec
(3881)
00008
0.96396
C.0449
000014
0.99
0.9965
30
887171
033345
000011
2e D6
0,7054
066847
DO
vap
3c-05
20405
vac
sie
009951
0000044
031449
1.
0.D
1.0
0,0
BL92126
0,00153
0,00074
0.0006
vic
000055
Tedt
000054
vol.
0.99911
165633
0.9951
0.99511
0,2045
0L00220
065677
0.30029
1L00062
16
Vod.
0.99832
0.0
0.99802
0.D
0.99902
0,0
X3l
Ze
Alu
Abp
0.8918
0,00225
0.98918
0.00177
0,99113
0.00148
y1
ydd
yi
0.0
1.0
0.0)
1.8
0.0
yot
yor
30
0.99ES
0.00042
0.7053
DL000D4
0.99355
0.00441
0.99053
0.02012
0.67366
0.5491
stacimn
0.96206
0.00197
0,9635
0.0015
0.96293
0.00111
Table 33: Comparison of GlotLID vs FT on FLORES-200 benchmark (part 2)

---

with confidlncc throhuld A
GloILID-M
OpenI.D
GleLLD--M I-:3
Glut.ID-Mds.5
OpenI
OpenI.ID 6=.8
3
FLORES Codets)
OpenLiD Codels)
F17
FVR
F11
FPR]
F11
VPR.
FPRI
F31
L
600
302
a0c
0:95503
0.00579
0.90012
0.00734
0.955
0.0043
0.5550
0.00292
0.9012
0.00734
0.90099
0.0067
6631
6601
0033
001562
0.00023
0.02279
0.00051
0.07844
0.00017
0.00011
6.0223
0.00051
002213
0,00338
604
06b
dkatei
s6b
00g
060
0.08
0.00336
0000000
0000000
66-05
00
1
055511
000000
0
0.0
0.
0.200
0,00333
020099
1.0
0,0004
00000
000042
46:05
0.0
01500
0.00206
0.0
0,0
0501197
0.9951
0.19924
039552
0.00025
0,00206
0.0
0.0
000199
03322)
1199012
0.00163
0,0014
0.0
25
85
00006
6
1
0.0
60 05
1.0
amb
00000000000000000000000000000000
000965
0 1
46
00055
Se 11
0
0
031129
039651
001287
be
021749
00000
te
arb
srh
am
000554
03145
2.0
620537
6
619388
002332
005457
554415
01430
03130
00000000000
001734
00110 001T14
013319
ary
ore
372
252
044669
006900
104234
118857
003125
106151
001601
0 028895
043187
011116
006458
0.1266
cot
sem
sen
101
101
0
0000LL
0,0000
00
000
0
000000
000
09001
10001100
1.0
000000000000
600055
000
11
00000
no
0338911
000
035318
000
00000000000
000851
aye
adb
syr
salo
036658
0
12513
00000
000
039139
02505
4
003108
0:0
0:0
00000
00
be-0h
90
00
6
hak
tok
hui
1.0
on
00
00
0.0
000
120
00
10
ham
beat
bun
0.01110
0
00000
ban
best
bus
0:97521
000
bel
bel
bdl
UP
12
00
bem
ben
bem
ben
ben
0.94853
0
0 1000
bja
0
bod.
0
bug
bos
bos
0.9900
0
020056
hil
hul
bil
05455
69651
000
600
1.0
0.
La
00
1.0
0.0
0.0
oob
gk
cos
ccb
ojk
co
odh
on
10002
0
199551
-1- 10-05
00
0 00384
06:000
Se 05
42-05
0.0
0599004
0.59881
0.9558
1
00003
600619
6-48
066716
0,0031
0.0
okh
irb
<kb
cth
0000000
038802
000001
0,0000
0888
000
000
00000
0
1.0
00
10000
10
no
dan
oym
dou
eym
das
deu
2e4
Jan
0:20901
[19811
0
0.00001
La
00000
00
030000
039681
00000000000
au
109953
7o-41
0.0
0.0
0
1:0
1.0
0.0064
00
00
10
no
dik
dsu
dyu
dic
00:00
3
100100
000000
000
0,99011
00
dro
0.07500
0
0.55646
<ll
1.0
00
1.0
0.
1.0
0.0
1.0
00
00
ong
epo
eng.
epo
1
10000
00024
10000
0.99933
0,99215
46:05
1.9
0.00002
0,0
1.0
0000000000000000000000000000000
000002
00000
UO
ess.
est
000000
000000
0
1
1.0
00
00
0,0
000000
000000
00
00
for
fon
frs
tho
for
fur
fra
IU
100
far
6m
639961
1
10000
2200
80
00
000000
009800
1
0000000000000000000000000000000
000000
0.0
000
000
099143
036551
(3955)
199951
40-05
00
0.0
0.0
099853
09270
11999119
10
1.
000
0.0
0.0
0
0000000
000000
00005
039001
0.0000
1.0
1,00
000000
010000
000000
be
00
100000
00000000000
100
10
10
10
100
100
gla
fav
gaz
for
gla
257
036141
0.9091
22005
10000
105599
100900
00000
000000
00000
135200
1.0
000001700
00
0.0
0
004699
10
Te-05
0.0
0.0
0
0:99784
039660
0594088
000000
00088
00000
10000
100
gle
els
glc
1,0
0
1
on
000022
0.0
0
0
00
899708
1.0
40-05
0.0
6.9700
1.0
00
0,0081
00
gm
2m
10
000022
L.
00
10
0.0
0.99500
gui
241
10
au
1
0.0
1.0
0u
1.
0,0
1:0
00
10
0,0
hut
bat
121
118655
0
000013
Te us
0997544
00004
hau
000145
195111
00002
19201
L.
00
10
0.0
03920)
0/0011
his
luse
Inc
his
hus
hus
02013
trv
07478
han
1a
1.
1.
60
00
bye
10
tho-
0.00681
099991
ind
ind
ilo
isl
ind
100666
TO
its
0:96
00
639166
000667
1
0000
2--05
1
0000
kab
kalo
0.55067
002349
0.
0.87586
1.
000709
00
199909
0
0,0002
0.0
7
kas
kac
L.
0.
1.0
000
1.0
00
0,0
kam
4.924
0.91416
00
0
088768
0
166651
66-05
kan
kas
215
00000
0.97977
1.0
00
00
00
00
0:9447
1.0
0.0001
000
006445
10
no
kat
kaz
kaz
Kat
kel
0,0090
00
1.0
00
0.0
0.99901
1:0
000
00
10
00
kea
Kian
kik
ktp
likk
klas
kea
100
kilc
099177
1.0
0L004425
0.000114
be
00
0.
05
0,8872
1.
0.
000
09
00
0,36200
039221
0,91571
118111
0.0001
46
00
0.
00
091671
(81580
02:011
0.0002
0
0,00022
00
0.0
00
0.90256
0999511
1:0
1.0
0/000
010
010
00
00
110
1.0
90
1.0
10
0.0
1.0
kmeh
kmh
0:96111
0.9623
0.0025
097713
0.04201
000865
095100
knnr
0.99901
19385
0/0011
0.90501
0.0
0.0
0.99852
000850
kre
08634
0.00155
0.0013
0.56869
0.00221
0.86%6
40-05
0.56935
0.0013
0.99502
0.0058
0.99502
0.0
0198822
0.0
0,99507
0,0001
lor
knr
1.0
1,0
0.0
1.0
0.0
1.0
0.0
1.0
0.0
10
Table 34: Comparison of GlotLII vs OpenI ID on FLORES-200 benchmark (part 1)

---

with confidence threshold
GloLLID-M
OpenL.D
GleLLD-M 1-.
GloiLID-M
d=.5
OpenI
(-3
OpenI 6-.5
1s0629.3
FLORES Codets)
OpeaLID Codels)
FT:
FPR
FIf
FPR.
VPR.
FR7
FPR]
F11
FPR 1
Ian
I00
1.0
1a
0:99201
0.0
0.9802
0.0
1ij
0,59901
6c-05
0.95803
0.50019
0.99552
40-05
099753
46-05
0:99503
0.0019
0.9852
0,00213
lim
1a
0.95554
0.50019
0.99253
0.0
899153
0.0
0.997113
0.00019
099933
0,00213
311
0.9MM01
0
0
0.98552
3e-05
0.9901
0:9900
0.0013
099822
0100013
11
0000130000
lano
0,90554
0
B99753
0.0025
0.99554
$c.05
090504
42-05
0.99755
0.00036
0.9953
0,99633
0.0
099500
0,0
0,59552
0.0
0,0
los
lug
Ivs
mag
Ins
ma
mag
m21
1og
100
1on
115
112
191
mag
0.9955
0.99555
0,9H65
0.99957
0.95559
0.97166
4.9955
166
1.
0
0.0011
0.00133
66-00
0.0
0.0
109833
00000000000
(0000)
00000
020074
00005
6e 15
6e
0.0
0
0.9010
0.94555
0,99911
0.9M00
032445
055NIT
1.0
010097
QU0021
46-05
0.0
0.0
QU
QU
0.0
0
0.5018
091100
0.91551
0 199951
099501
099653
0.0
0.0
0.0
0,94455
0.0000
0.0985
0
0398012
0596001
0.99511
0,94158
0,9455
1
000005
000000
000088
000015
009122
be
be 05
010
00000000000
0.99911
(19951
(195555
096701
DIQ0ESS
D.00ST
6o IS
0.0
mis
mal
mur
mll
0
16665-
1
1,0
0.00007
00
000000
00001
0.56183
1/0
000
00
10
45-05
00
80
000000
170
0000000000000
00000
600015
000
00
0.
mid
000
ult
mai
mos.
sil
0000300
00
U00:32
100000
10000
104000
0:0
0000
00000
mii
000
10
uns
0.0
0.0
ald
0
10X000
nob.
nto
000
100
00000
0.0
00000000000000000000000000000000
015
0.0
000000
100000
000000
0
60
pag
pap
phe
100
pb1
ery
0
en
1000001
1998811
19660
10000
100002
000
0.0
0
1.5
00002
000
800
000
0 068522
000000
600
600
00000
0.
1.0
0000000000000000000000000000000
000305
000
09911
10
no
nO
pli
pol
pes
000000
00
LA
0
0.0
1 2
000072
000
050000
803167
05008
00
699996
1:0
000007
000
10
poc
201
11000
100020:
000025
109004
02001E
pus
SEL10D
611223
000223300000
0
cun
ron
quy
2001
mm
mm
0
0
120231
0300
1
00002
0022
0.0
du
0.0
100H
1
001100
bo
05
U.
cus
66-05
sag
san
100
0e
90
131
121
0000
90
10
000000
000
sha
0.0
10
F
00
1.0
slk
son
snd
snd
2001
0
1
00
009000
00001
00
10
099511
3000
4
10005
000
02002
600
000-0
00000000000000000000000000000000
0
039004
659001
0000000000000
800000
00001
000010
000077
005657
000000
0,000
n0
10
00
epu
and
snl
0000077
on
100000
1000211
000099
00006
0
03900
000
196900
010
000
8000
00221
4000
000000
00
0,002
no
sip
512
100001
000
00
su
swe
ssw
suh
550
500
0
0000000000
010000
000007
000002
1
4
60000
00000
000077
000025
08188
885555
1
00001
000017
10.00
010
000000
00000
00000
00007
10
up
sal
321
4655
4
tun
1.
0.
1.0
au
1:0
00
11
0,0
104
0.80000
0,5380
000827
121
121
80
09
1.0
1
0000000
1.0
1.0
00
101
181
00
10
0.0
igk
121
194
0000
00000
1
0000000
100000
1.0
00
00
171
117
0
0/0
00
0.0
00000000000000000000000000000000
100
00
100
700007
(09001
500004
0.000
039955
100001
40-05
42-55
160051
00000
659665
10
T6
010013
00
tur
1201
000002
p0055
1
184430
1200
059002
0000
uig
1
La
00
000
D0
00
1.0
no
uki
62/55
40-05
009951
0
6.005
und
unb
100
0300000
183374
QU0011
657816
2000
vec
uzn
03008
010000
1
00000
0.940
(9965
00
0000000000
vic
112
1.0
9.0
1.
00
66
000
was
099222
0,0
QU
(19951
00
1.0
00
0,
wal
020026
0
QU
019910
0,0
059753
0100026
0
xho
<30
0.99114
0.00099
0195611
040154
0,98968
0196688000
0.00067
010007:000
11991131000
0.0004
0:9675
000155
09998
0.001
yel
1
10911
09900
you
yuu
3117
102
0.00000
0.00033
0,0032
0/0011
0.90385
1.99355-000
0.00013
1100013:000
099053
4
8.9961
6.06586
00002
0.0001
0
0,06385
1.0
0.0
1.0
0.0
0.75575
0.86088
0,93915
0,00507
0.92455
0,0016
0:94511
0.00127
0L94972
0.00081
0.97589
0107666
(19665
m01
72
0.96801
0.0
0.96277
0.00135
0.96803
0.0
0.96755
0.0
0.98325
0.00155
0:001
Table 35: Comparison of GlotL.ID vs OpenI.ID on FLORES-20 benchmark (part 2)

---

with confidence thresbold 0
GRILLD-M
NLLB
GOoLLID-M (
GIoLLD
M
0
NLLB
6-,3
NLLB
(4..5
Bo639-3
FLORES Codeia
NLLB Cedets)
FI1
FPR.
F11
FPR.
F1"
FPR.
Flt
FPRI
F11
FPR.
F1"
FPR L
000
000
0,9550
0002579
0.995322
0111099
0.555
0.00426
9,9565
0:00299
0.95532
0.91309
0,919
0.01076
afr
31r
387
1.0
0.0
0.99822
000001
1.0
010
1.0
0.0
0.99901
0.0011
019651
00
aka
6k3
0. 99852
0.0
033335
0,056
0.0
0,99852
0.0
0.82354
0.008
05227
0.0854
a1c
1/4
0 199832
0.0001
0. 0.99003
0000030000000
000022
062339
00
0.44755
0.0
0.09001
D00022
89991
5o-05 1
senh
smmh
mmh
8 999E1
62-05
0
0.00001
00001
1
0.0005
4o-05
0.99951
4:-05
1.99961
000011
099551
So-85 3
arb
arb
3db
0,21515
0.11612
0.46965
0.
0.6522
0,00457
0.11811
D.46965
0.31789
0.41516
sean
20m
mmm
1
0.0
100
80
100
0.0
1.0
0.0
1.0
0.0
10
80
sed
441
14
00006600
0.00151
0. 0,00015
000026
0.00223
0.99257
0.0001
0.09065
000026
0,902
000647
apt
2003
32h
251
204
0.8951 1
0 095557
0,00045
0.0001
6:405
000045500000
1
0:90022
08396
00000
00000
0 56
0
38982
4o-05
0.0
0.
0.55313
0,0119
111112
00
00
00
0:00000000
0.33000
0.9111
0.39001
033086
0000 19
200092
30-15
0.0066
000057
0.
ser
203
8
000
0.90704
100000
0
0
0.0
0,27304
00
0:02004
100000-000000
QU0051
bak
bak
bek
1.
0.0
1.0
0.0
110
00
0:300L1
002001
So-tt
bun
ban
bedi
0,52553
1001199000
0
0
ban
ban
baa
0.97221
62-05
00000
0997571
010
00
0.00026
bom
bel
bon
bam
ban
bel
bes
bea
bel
0.9922
0.9906
1,0
0.00945
62-05
0.
022300
0. 199951
10
000277
1
000010
de:25
010
0
0.9857
692256
1.0
0.00015
8:405
00
039951
0.97677
1
1.0
D
33-15
0
809951
009223
0U0008
tho
Nio
bbo 0,93229
0,00445
033355
000000
000623
6,94%
0
0.8354
0U00120
tyn
byp
by 0.79590
0,03329
0.1222 032222
6,795
00
0
bod
bod
bod 0,91599
096212
000020000000
091389
69676
195009
000221
bos
bos
bcs 0,55206
0,00312
0,9854
0.000
00000
0
0,59625
0,00126
0,5954
0
0.006IS
bug
bug.
bug
0,99992
dc 40
032099
0.0006
0.99001
097782
0.00036
bul
bul
bul 0.99911
0.9991
10
0.94911
0,0
019952
cat
1.0
00
0
1.
00
100
68826
0.00
0000:01
0
0.00557
oth
ceh
rub 0.99223
0.0
00
0.9600
60
0
000
009651
00
ces
ces
co5 0,9911
1
0c 400
44 51-
0.3400
1
1
1.
gik
gk.
ok
08
0:30578
(1851)
088895
0 0
cib
ckh
cbb
0.0
0.0000
00
1.0
90
1.0
0/0
1.0
00
ch
:th 0,0002
0.98221
000
0998
40
0
0.
00R
00
cym
cym
<ym
039211
3
1.
00
1.0
50.05
1.0
0.0
dan
den
dan
de
de
dom
0.9900
01911
0,00011
1000110000
0.0
08007
099456
000070000000000000000000000
00000
0.0000
00
7e-05
00
0.9950
1.0
000000
000022
00
dik
dik
0.99653
6:-06
0.9225
00
00
0.0
000000
100000
00
dzn
dyu
dyu
dzo
don
0.12435
11942
000585
000000
011878
0.949
00:073
16400
0.0218
10008
0.96885
0
Seat
cll
00
1.0
000
1.0
1.0
0.0
1.0
opo
ong
cng.
opo
0.9983
0.9777
000000
600148
7
0 mo000
002035
00000
000063
46-05
0.
8.99556
1.0
6.000
0:8825
0.99003
000033
000224
002162
0.000
eit
cst
1.0
00
1,0
0.0
00
199001
1
cus
cus
0
00
00
Q0
ewe
eve
1.
1,0
00
dap
fio
10e
0 1
90
0.30910 1.505
0.0 0.9951
00 0
0.
0.
1
0.0
0.0
0.0 0.99900
00
1.
0.9
032242
0,00011
10001110000
1.0
000
1.
1.0
0.0
ton
ton
099051
6e
00
00
1
4e-00
00
0
00000
5e
0.
tu
1
6c-05
0199911
46-05
1.0
QD
0.
ttv
tuy
6e
0190099
00
0.M
087275
0.00016
gaz
gda
gat
gla
20
0,0000
030081
03000
000015
1.9
(00011
00
0.78933
10
7e-05
00
0.9855
0.9000
1
0000
Se
gla
gla
0.0003
000021
1.
00
0.50251
zlg
2/2
3,99701
0000554
12
10004
gm
gm
20
1.
00
059666
000016
1.0
00
10
0.94703
099700
21)
0
0.9900
0:0055
099651
hun
han
0.99m
000007
0,0000
0.9970
1.99744
0.000 1
hih
beb
099000
1.0
00
1.0
hin
hin
008451
0
001594
001504
hrv
hne
hnce
0.005
0.75157
053155
086056
000556
151661
has
Ian
000000
bye
bye
1.
0.0
Q
Lo
0.0
iha
iba
0.995E
6-005
S
199661
46
ind
jay
ind
103
000000
000000
100001
00000
10224
1
000066
Se
6
095244
00017
0.0016
0018
X0
00
0.0031
kab
kab
0.89667
1211200
0.55700
001911
0.01692
kac
kac
16
kam
kam
0.92 9.92666
0--08
0:38611
000001
091416
0.
0:7488
0,04155
0,000
kin
kin
10
CLU
1.0
0.0
1.0
BO
kis
kal
khp
kax
15p
kur
kal
khe
kaz
kal
8. 19951
0 1
0.97014
1.0
62-08
0.0
0.0
00000
0.7735
199960
10
60
0
00
06
0000000
000000
1
10
0.9081
20
0.0
0.0
US
0.0
U57555
0
099001
1.0
U.O
0.0
0.0
0.0
ktkc
kin
kik
kher
kea
1hm
Lnkk
kin
kea
kik
ktk
Kit
0.952
6. 99051
8 16562
0.1471
10
0,00463
0.00034
0.0
0.0
n 1001557
0:90001
000011
000004
000015
000
00
000000
0500555
000003
K0
036057
110000
5
0.0
000
103000
1.0
19
0.0
0.0
0.0
kir
ki
kir
10
60
1.0
1.0
0.0
n.0
kmrb
kmnh
kmbh
0,00415
000564
199221
6.7711
0:00144
00000
004588
kmnr
kere
kmn
8 8.9901
0.0
0.00508
000004
09990
00
0.9901
on
199655
000054
1077
0 0u0002
kne
knc
Inz
D 1.8634
0,00153
036555
0/0016
000011600000
086809
0.00211
10002110000000
4:-05
0336784
120016
086745
Sc-15
kou
kon
kou
0. 19982
0.0
0.536
0.99007
B80354
koe
kox
10
0.0
0.99005
1.0
010
1.0
0:99511
020043
18
loo
lao
lao
Lo
0.0
0.99511
6.0
a9990
0.0
0.99802
0.0
0.99301
0.0
009852
no
lij
lij
ljj
0 199901
62-05
197738
209852
4o-05
0.99753
42-05
0:97355
D
0U7423
0.00166
lan
lim
8.99253
0.0
[198701
0000011
0.92233
0.0
0399153
0.0
0.96701
0397001000
08001]
1 095701
0,0001
lin
Un
Lia
8 8.99981
0,00011
0199550
000003333
0.99852
Se 10-05
0.99901
42-05
0.999015
08003
0199544
199054000
0U00016
In
III 0.99951
0.0
0199901
0.00011
0.99981
0.0
0.99951
0.0
0.99951
0000011
(19900)
Se t0
Table 36: Comparison of GlotLID vs NLLB on FLORES-200 benchmark (part 1)

---

with confidence thresbold 0
GBaLLID-M
NLLB
GROLLD--M [
GloLID
M
0
NLLB
6-,3
NLLB (4..5
Bo639-3
FLORES Codeia NLLB Codets)
FIT
FPR.
F11
F1
FPR
F1t
FPRI
F11
FPR.
F1"
FPR J
1gg
imo
1x.
1n1
11g
301
Imo
18g
0. 1
9,99554
0. 19653
000001
0.0
000
0.55201
0.9964
0196661
0.0011
0,0
00
3.99554
019683
3e 05
010
0.
0.9091
0.99903
0,99501
42:05
0.0
0.0
0.972013
0.542113
0.99651
0.00119
0.0
0.0
099509
099651
059155
QL00015
lug
lus
3ag
lng
hus
8 19653
99633
6x-05
0.000
0.0007
0
1.80214
0
000070
0.99003
0.90533
0.0
00
0.99654
0.90644
0.0
0.0
0.99357
0.99311
000076
000085
0.9554
09N45
0.0044
0:00M
00004
luo
300
100
0.0
S-65
1.0
0.
0,99911
0.0
0.99755
Se-05
00
lus
luc
1s
0 09655
66-05
S-65
0.0
0.99653
00
0.99404
S-40
0990
00
Ivs
1ve
8 99558
0.000
0
0000
093396
0.00211
0000211000
0,80150
0.00015
0.00015
000005-00000
0.99362
0,000
0.0062
mag
mai
mag
mag
0.95459
0.00138
62-25
00111
10000
000031
0
0000977
4o-05
0,97100
0005408-000
0.00048
0.0
0.9111
0.98700
0.0023
000233
0
0U0063
mal
mar
md
mal
mar
L
La
0.0
0.0
0.09908
000004
00
0.0
0.0
10
10
0.0
0.
0.09555
1.0
000064
0.0
0.000
0O
min
zin
mas
0.6616
200007-0000
0.20545
So-05
0.66116
1--05-0
189558
S--15
0.
ultt
mmi
mnkd
midd
alt
087800
1
10
0,003007
6c
00
0.0
0.99999
119991
10
0000011
000001000000
000
1.0
0.0
0,9903
1.0
0,00015
4:
00
0
1.0
00001
0,0
0.0
100
Se 15
110
0.
mos
1004
0.0
100
40
0366000
0.0
00
aui.
mii
0,99001
62-25
Seatt
00
00
aya
1
10
10
1.0
00
000
1.0
10
ald
ald
0
be 20
no
mo
08550
00000
0800
00000000
00000000000
nob
nob
155115
0:011
3
npe
nys
nso
nus-
up1
150
Tus
1
100
1g1
085101
1
020000
0,0011
08
09900
896880
0000 30
0000
000000
7V
1898
0
3e
010
0.000
199633
0
00
0
038380
00000
1000
0
72
0
00000
oct
Oc 50
L.
1
09911S
ory.
pag.
pag.
ur
0.
00
10
00
00
00
40
10
10
000000
0/0
00
pan
pte
pap
pap
pb1
pan
000000
0000900000
LG
100100
0.0
098
00
00000
00000
0.0
10
000000
00
0
10
00
000
pes
0.04400
05380
pli
(1)
0.0
69
00
00
10
00
00
poc
po
201
1622
000108
0
09622
0
0.970
692167
6.0000
00000
60000
0
0
000022
72
1331:
prs
U1177
quy
con
00
00
039
0,00266
3
0625
00
00
0,9951
40
6
00
1.0
0
0.0016
00
cun
0,00e8
0
692627
0
0
rus
155
129
Ge-50
4 4:10
010021
sag
00
0
26 11
199700
sen
6:55
100000
4
0.99103
0
000001
0.
505
2.
0.0
60
1.0
0.0
10
1.0
0.0
1.0
U.O
scn
039802
05
QUOCEE
shn.
100
19
00
10
sin
0.0
00
00
40
0.0
1.0
0.
slk
46-25
1.
stv
0
0 0
48 12
00
431
2002
000000
00000
60
190
000014
0000000
0.9077
10
50m
0006277
0000770000
00000000
10
0
0
0100000
09
00
sul
1,0
00
0.0
ypa
0000000
000000
00000
000043
0.00
0.9777
0,9000
stp
09995
4c-05
0.0
00000
00000
00000
0
006000
000000
199355
0994%
00007
0
000017
00001
a00072
0,7459
1.000
056129
1
D
0.0077
0
0,001
0,02558
55
So-85
46:05-0
0.0
1.0
1.0
0.8861
188861
000827
0.00
0.32180
00
081780
gk-
1.0
1
00
10
1.0
1.0
de 1
00
10
10
0.0
1.
000
0.0
hha
0
Y.0
10
V.
10
0.0
1.
00
00
0.0
002000
1
ukk
cuk
0.950
000000
00000
1
10
199900
44.00
16
1000
0.0
098872
ben
um
twi
bam
C94421
00864
60
0
038550
003401
00
000000
1014
ukr
unl
vir
umb
vec
uzn
umb
uzn
Vic
uig
und
vic
und
vic
CL88SES
C C08346
0 1
000955
0990
1
0.00564
0000045
87
66-05
000045
0.0
62-05
6c-08
039682
0391159
000010
000000
Se
60
a T
0
1.0
aoSI 50
4
4e
00
0.
0
10
10
00000
00005
0
000001
120055
000016
0000 10
0000
S
101165
1.8
Sc-tt
Do
00
war
19951
0.0
1.95001
000011
09955
0.
009931
00
0.9951
10001
no
wel
vol
wel
0 1
0.000
1190504-00
000011
0.0
0.99022
0.0554
D.00011
009554
Sc-t S
xho
xbo
0 18
0,00097
0.9799
000163
3.9996
0.00067
039113
0,00044
0.00022
ydd
ydd
ydd
0 1P
10
6.
1.0
0.0
1.0
u.0
you
you
0 094066
0,00003
0000003300000
0.99556
0
0.93355
0.00013
0.99053
0.99554
0
1,000330000
099752
yuc
y02
0 000055
0.0
B.4777
0633995
0.42311
D.63395
0
046292
0.01752
tho
220
2ho
10
0.0
0,20155
0.2244
1.0
0.0
1.0
0.0
0.69122
062543
0 0.617
0.1121
zim
23m
2501
0,45500
0,00307
0,93159
0.00442
100044210000000
3.94641
000122
0,94972
0,00081
0.98924
020042
091869
QU1017
zul
2u1
201
0.90893
0.0
0
0,96955
0.00243
0.98833
0.0
0.96735
0.0
0.9379
0
897901
0.00176
Table 37: Comparison of GlotLID vs NLLB on FLORES-200 benchmark (part 2)

---

confidence
GheLID-M
GhaLID-M 66,3
GhoLID-M t
CLD3 6
bvo3393
UDEER Codeso
CLDS Codets)
FPR.
FPR
FPR
FPR
0.9661
0.03155
0.9005
0,00122
091605
0,00118
140
0,59635
0.0
0L55625
000
0155055
0.0
ida
0.91045
0,0015
0:3113
110011
DU00IN
ady
a5y
0.83485
0.0
083475
0.0
085455
0,0
alt
46
af
0.55338
0.00775
0,85105
0.00006
(L96774
0,00340
IL97561
0.00885
0.89552
0.00054
0.9900
0,00041
157
357
0.81428
000077
082006
0.00004
083014
0,0004
211
311
6.6
066516
IUK0222
all
ILIXU3T
0,00224
OLD
Ta
0.00774
0.0
02 000073
0,7
0,00272
0.20896
0,13S
00
0.053A
OLD
Lo
0.0
0.90187
00
0.01877
OLD
an.
370
O 595952
0.33333
0.0010M
099361
0,5510
0U9ITUS
0,00045
art
257
0.0
0191197
0,
0.99877
0.0
an
0392913
(L00012
000
096737
0LD
ast
AS1
0.93311
IL00012
1L97521
0,00012
0.93333
0.0
asc
200
0.01504
ass
25T
0.00012
0.00771
0.9415
22jizsh
06415
0,96361
Nés
b4a
0.92177
0.00122
0.92913
0,00399
0.3348
0.00001
bei
bx
0.97521
IL00037
0.9916
0,00012
1.0
0.0
hel
bel
1,0
0.0
1.0
0.0
0.9466
0.0
hel
bel
0.0
0,50693
0.00529
0,0
0.9315
0L0
0.53571
D 000H69
0.8888
0.00N3
hon
bom
0.000S
0.9835
0.00034
0.9335
0,00004
ben
Den
ba
1.0
0.0
1.0
U0
1.0
0,00
1.0
0L0
1.0
0.0
1.0
0.0
NG
0.78519
IL00037
0.7811
0,00387
IL00012
Nit
0.92113
1U00022
UM9213
0,00012
0.0
bed
0.39091
0000122
0,00012
0.9951
000002
bos
Des
0.181
IL01134
0,23913
0,08511
IL18605
0,00114
0.14600
0.00331
0.37286
0.00011
0.0
Sc-05
bee
bec
0.58361
0.0
[196361
0.0
096361
0.0
he::
lus
beg
0,05312
0.00622
0.90633
0,00040
0.00044
tul
bul
bphg Lata
0.96
BL00822
0,22112
0.00483
0.56
0,00001
097561
0.59
OUNTOSS
0,00269
hum
hrm
0.547K2
0.0025
0,5659
0,00017
0.6654
0.00022
cab
0.0
cak:
0
0.0067
0,37297
0L01077
0.00225
LD9315
che
dbs
047166
000017
0.66
0,00012
060059
0.0
cte
<h
0.97770
0,00122
0.97499
0,00012
0L0
cbu
cbu
0.18462
0.0
LUU6077
0,0
003335
0.0
ccp
ccb
ccb
92b
0.6721
0.0005
0.431R2
0.00635
0:0916
0
1.0
0LD
0.456
B00G11
QL487I8
0,00554
ccs
0,93387
0.0
B.6766
0.00265
EL96387
0,0
098387
0LD
0,74699
0.00977
0.79487
OL0014
zfm
0,88914
085714
0.
0.84155
cha
0.00012
08
<hi
chi
0,00378
0.8333
0.5154
1166154
0.86154
qj.
4k
0.93641
0.00037
0.92641
0,00287
0.03335
0.00012
066217
0.0
0.65217
0.0
0.63736
0.0
ckb
cth
reh
oth
120012
08995
0,00099
094488
0.0088
<di
cn
0.90226
ILU0162
0,00146
199663
0,0033
coc
CEE
out
0.74747
0.54188
055083
0.30227
QU41812
0,00756
0.50918
0.0
0.98361
IL00025
IL96361
098361
0000224
0.56414
IL0OUST
0,00204
0.8076
000004
L.
0.0
1.0
0.0
1.0
0.0
20W
000199
OLD
0,00090
zid
rd
0.7843 1.0
00
0.78431
0,0
0.74477
OLD
c5m
<3m
0.0
0.667M2
0.00643
3.0
0,0
10
OLD
0.55111
0,0016
0U63918
0,00E17
dgg
dag
rbn
dan
0:85914
0U00632
0.91473
000030
0.0394
0.001
0000044
0.99651
00001R
0.91161
9e-05
din
0.967.45
0.92549
0.0007
0,00012
0.944
0,00041
IL00488
0.73855
dbv
0.97744
0696774
0.
083333
0LD
dun
850
0,7391
0.0
ELU7391
0,0
0196991
0.D
d5u
UL22059
0,00756
ULLTEE3
doi
dan
0.0137
(L90769
04001
B.00500
DONES
ell
all
cli-Lata
857918
0LD
0.81579
0.00246
EL97905
0.0
0.97988
OLD
0.91045
0.95886
emk
mg
eng
68
0.0224
0,40955
000760
0R721R
0.00881
0,885
0.4972
0.077
0,00739
5po
opo
0.6825
0.35582
0.00955
0.00257
000861
0,00997
0.5970l
0.00009
0.40116
0.00744
B163492
6.7362
0.46332
000633
0
0.00111
[U00112
0.36747
0.00958
[U96825
0.00249
0.488
000555
0,00362
Table 38: Comparison of GlotL.ID CL.33 on UDHR benchmark (part I)

---

with confilence thrssheld
GlsLID-M
ClaLID-M 6m,3
ChoLID-M 5
CLD3 6
CLDD3
bo639.3
UDEER Codess) CLD3 Codets)
FPR
F11
FPR.
FPR
FPR
FPR
0.27596
0.0061
0.00207
0.00012
0.31151
0100266
0.2029
0,00077
003335
0.0
0.96161
0.0025
0993611
0.00304
096351
0.00044
1to
0.96366
0.0
BUM6305
00
099325
0.0
Tts
pesips
0:501552
0.56331
0,00133
0,00243
019094
0
DODDES
0L00066
6a
5e
0.97931
0.0025
097521
0.00034
0.95211
0.00004
(j
fjj
1.0
0.0
1.0
0.0
1.0
0.0
0011
000019
fin
0.03544
0.27191
001534
09252
0.00177
00211
0.3
0,0134
0.01E13
Ikr
0.28671
022571
0-1229
500
UMMMES
0:05238
0.00755
0,37855
0.00699
0.48583
0L0KERS
byy
0:09174
0U00012
0,67089
0.00299
0.0
0.74074
0.00991
0,00100
fdf
0.86762
0.0005
0.016T1
0,00000
00
00005
for
for
0.91473
0.03255
0.99115
0,00309
0.0005
for
FAY
0:77911
0.02277
0.80333
0,00122
0.00011
tvr
TVT
200
0.938666
0.001
0.00509
BAKKW?
0.00124
BAG
0,93113
0.0
0.00
0.0
sua.
946
0U5522
ILM0311
L18522
0.00171
035761
gn
0.90466
0.001
0.0039
0.5776
0.00011
skp
0:9206
0.01255
0.2244
0.0099
096667
0.00447
gla
510
0L00025
0.50169
0.006554
0M5512
0,00012
096063
0.0
0.6055
0.71351
gl
210
832
0.95
BL001
0,55882
0.00324
ILMANE
0.00061
096055
065353
000014
0.74025
0,00235
0,9K305
IU00122
0
0.00333
0.00012
0.00012
0.64314
0.7352
0,00181
LA
0.0
1.0
0.0
LO
0.0
0:98333
0.00122
0
0.0
0.9916
0.0
gic
0.80033
010052
0,00099
09K367
0.00124
010
0.50992
0.0
0,7077
0,
B66357
CLD
(1)
271
1
0.
1.8
00
1.0
0,0
1.0
0L0
1.0
0.0
1.0
0.0
guk.
24k
gir.
200
255
0.43744
0.57143
0.44444
1ut
IL00311
0.37981
001800
09313
0,00219
0.93312
0.00141
041724
001536
044203
0
0.17101
0,0081
0.30205
001134
0,01457
beb
0.98305
0.5165
5e 05
1.0
til
hil
L0
0.0
lin
lan
Lath-Latt
0.62
EL02947
0.17175
0,0222
BG23122
0,00914
062244
0.00961
0.19223
0.03229
0.23664
0.01SE
lit
0,52982
0.0
BUIM2
00
0,8999
0.0
hrrn
hmshrjihcs
hmm
0,02100 08
0.0
0.41538
0.001 19
0900
0.0
0,320
000059
0.21996
0,00045
bree
tni
Imil
tns
hns
0:89555
0.00075
0.11228
091228
la
lav
0.60812
IL00984
0.44324
0,00383
B.62176
BL0089
0.69364
0.00625
0.45055
lsb
bun
0.21779
0.1956
U34507
U189522
0.00105
0.2649
BOISIN
0.33MXX
0,010K3
0:99092
0L00037
0,98315
by
1.0
1.8
0.0
1.0
0.0
1,0
0.0
ith
14h
ibo
1b0
0.95718
0,29903
0,01447
0,00012
1.0
0.0
0,25484
0.0276
0.46526
0,C0EDE
150
350
9.95
0U00122
000
089915
0.0
idu
0000012
0.D
0,00012
ii.
ijs
155
iks
0.97872
0.00055
0.97872
0,00004
000002 000004440000
ulo
00011
103
0.0769
0.00134
0.9916
0.00012
0.53953
0,004417
0.59184
0,0036
0.60733
0,00355
0.78947
EL00386
0,30227
0.01550
(X33116
0200264
0,120ng
001121
0,00557
0.975K1
0.0005
032159
0/060
0.95311
0300300
0.36117
001199
0.00851
0-46533
0.01577
0336447
00159
001155
Pajo-Late
0.7861
IL00944
0,36516
0,02177
0,063C
003506
0.4-200
0.0654
0,50690
0,01263
0.96687
0.0005
000000
QURRTS
km1
0.0
C00
kan
ka
1.0
0.0
1,0
10
100
10
00
1.8
0.U
1.0
0,0
kit.
kar
L0
0.0
1.
10
100
10
0.0
1.0
1,0
0.0
096721
0.00377
0,30584
0.00635
0,00224
096657
0.00004
0.41522
0,0077
0.46154
0.00654
k3d
0.87692
8L00199
0:00195
0.00899
0.00174
0.0012
0.58491
5.343
0.77551
0.7384
0.0
0,53227
IL00125
0,00122
000106
lck.
Ack.
0191221
0.5916
0.00312
03916
Q00012
thi
2h1
0.97521
0.0005
0,00012
089879
thrn
kim
km
1,0
0.0
1.
00
1.0
00
110
000
1.0
0.0
1.0
0.0
kin
kin
0.76330
IUX2622
0,00306
68547
000083
kir
ktr
ky
054238
000075
0,22599
001815
0,00073
096774
0,33211
001809
0.26374
0
1/h
0.9009
000287
083089
0.005
08882
0.0235
kih
0.59194
IU00022
0.00212
099194
0100012
0L15505
0.00344
0.19125
002274
0.35764
0
0.0037
099778
0.06224
0.0
0.00077
0.0
0,00012
60
0000022
km
km
0.966677
0.0017
0.95632
0.00013
0.95172
000012
lno
100
0.79380
IU00012
IL79349
0.00012
0.73339
0L00012
bar
800
No
0.55336
0100175
1.
(196774
0,00305
0199174
0L00012
1.0
1.0
010
lap
kgm
100
0.0
1.0
0.0
1.0
CLD
lap
Aqps
0.21096
OLD
0L05224
0.0
0J83270
OLD
km
0.0
096875
0.0
099055
0L0
krl
krl
0.025K1
amSsi
0.78912
IUU0723
0.57146
BU0399
0.54354
SL00354
lad
0:92174
0L00025
(192134
0.00024
lan
1.0
0.0
100
0.0
1.0
Table 39: Comparison of GIotLID CLD3 on UDHR benchmark (part 2)

---

CLUS
GHILII-M
C1D5 ONL5
CLID
UUIER Cidito CLUS Codalo
FHI
FPR
FPL.
31R
FS1
FISE
FPI
0.533
0.00013
BOST
DINAIE
040000
0,0001
16.
0.96643
63
0D
060723
0.
B54084
0.0033
0.09115
0.0034
03
14.
0,8733
043352
UUNTT
858363
015054
033133
DURCO
341
0.90553
0.00025
D/0012
0.3
0.84882
633
0.
05188
6.3
003300
0337522
0.00-31
0/0122
0.0035
tac
10c
0.9934
163
15
(350537
B3310
6.3
la
109
015635
030033
(10006
(0006)
0.0035
00001
mut
010001
0,2999
383066
1.8
10
18
DU1
10
QD
1.0
1.3
141
CDD
0.99354
0300113
033932
001122
399111
0.0033
0332397
01313315
1L30255
DARRESS
030011
(USS2N
DUMITT
1.5%
0.0654
1.5
03
2D
63
mn
1950
03071
010037
DOTTI
DIRKOT
839225
035831
10
CD
130)
11
6.61111
055331
63
0.501S
2R.
120
010051
073550
DAAAGE
130140
DURAIT
am
120
022211
030699
DUTTI
DIRRGE
B38462
029314
1004
00001
DIRKOI
000033
010015
023955
0493
031557
591154
00005
OA DURRCT
UC
000022
03
013121
0D
0.
ma
0.00333
DINGOS
0000922
0,542
0L3033
000013
30
0D
0.
0
DA00O6
CIOISS
1.8
U.D
30
00N012
000312
ALD
di
010011
052272
DURDES
123022
104426
DANIE
8.9199
0
0
105.
030028
U2
UL3
0.500T3
0.9050
0.8
0.30411
0.0
DUMO2O
00003
540
1.8
12
DU
10
GD
0.3
100
1.0
C.D.
Po
0.00985
(4b
020006
0/
071730
GD
03
2b.
264
COM16
0L3
p6h
0
DINGEE
pel
0.0022
0401334
0.4344
0330111
30
CD
ALS
JM
065312
00038
DIWAET
B38218
00386
0200
DONESO
0.00%
05
p6c
0L00311
DANOIS
bf
030013
119174
DO0012
000033
122.
0100112
DS8111
DIRRIZ
03204)
nd
14
DOMIES
356TTS
03
03155;
016012
(18000
01004)
mp
0125
881553
03
0.8099
0D
0 075076
03
24h
030246
000100
BX6511
Table 40: Comparison of GlotLID CLD3 UDHR benchmark (part 3)

---

with confidence threchold
GhaLID-M
CLD3
ChoLID- 2m33
GlaLID-M 6
CLD3 th.5
CLD3 9m7
UIDIR Codrit)
CIEIS Codaço
[PR]
FT1
FVR.
F31
FPR
FPR
F1?
FPR
Sa
521
0.66677
0.66567
0,0
0,66657
0.0
831
800
092881
0L000
0.58
0,00012
0.94118
0.000122
523
0
0.28829
0,88300
0L86792
0.
0.99165
0,59145
0D
dap
0,27397
0.0
0.25
0,0
0L14706
0,0
<d.
0U98S25
000044
8.9399
1.8
18
00
0
0D
1.0
0
31c
0.905000
B66945
0L0025
0U8255
6.97561
0710
075667
0.0044
sne
11 1.M6667
0.00112
0.96577
0.56657
000004
sno
STV
1.0
0
0,0037
0
00
0
0
07874
DODIS
0.93546
0.001
0.23382
0,08334
0.00653
0.00061
8.98357
000000
0.30775
0.01362
000034
0.000333
00
0596227
0
0:418
0.0
0.75817
0.00481
0.1 0,12141
0,3359
0.50556
0,03441
8,89131
0.00165
0.3919
0.00008
D:02516
41
0:00112
B 8.3599
0,9916
0,0
00
84
00002
0446
D00056
qua
450
26
0:22501
000146
Da1lib
9.78
0 0-R0KS
B
55
0.87144
0.00499
0:39710
0,1296
0:6331
0,02119
0100153
0.36036
041007
0.5124
0,00744
0,55555
0,0083
01,05g
000719
0
0445355
BU00K3
0
0OOTT
0.90231
0.005
0.83445
00366
(.9651
0.00112
509
B 14000
(030
60
sul
0.68122
0.00866
0,0026
00066:
su
343
0:00337
03416
0,01077
00001
0.271
0.0004
0.394P
0.00852
0-45022
DO00RR
0.85315
0,00233
0.09722
0,0005
0.6625
000051
0.3011
0.01777
0
0
0.14115
0.00259
0 0.90292
0.0257
019615
003195
swe
0 855507
0.0033
8 89762
0,00055
0.51335
0001
18
00
0,92647
0U00046
095455
DO0GSET
143
B01892
0.00012
0.91992
0.00112
8:92727
0.0
14
0.61647
0.66667
00
10
0.6203
0.00675
0.68208
000634
0.69833
0.00599
1.24596
00
0.
8,3916
0.0
0.9916
0.0
0.9916
0,0
st
0366322
0000119
0A8517
edl
101
1.0
10
0.0
0
1.0
0.
1.0
0.0
1.0
0.0
en
0.973445
0.97345
0.94545
0.0
0,85907
0.00044
0,461
0,00007
0,90915
0/0011
6,90082
0.48739
0,00551
0,52252
DO00TS
1
0.9403
0.00057
0.95455
0.0061
6.97674
I.00244
153
1.
1A
0.0
0.0
0.
1.0
00
1.0
DO
18
10
0.0
1.0
0.0
1.0
0.0
1.0
0.0
0.0
0.97562
0.00037
0.98361
0,00024
6.99174
0.00012
0.0
0.0
(T)
0.0
19
0:000255
0.3388
000004
6.54893
000
sin
1.0
0.0
1.0
0.0
1D
0.0
107
0:000122
0.09
000011
010012
IN
010007
099361
0.00025
0.91744
000002
100
094154
0000114
54
0 0.22117
0.00299
0.56765
062507
0001188
ur
1
0 0.9383
001533
0.001
025641
0.01555
0.50356
0199488
00155:
00000
(EROR)
000332 001333
0.27907
0.01413
0.3108
0,01204
DA1
0.005
0.0000
00312
1y
898561
094553
0.00255
0.98335
00304
tea
001754
0.0648
0.01554
003834
000501
mo
0M7475
0:0011 1
00
00
ada
0.8361
0.00225
039074
0000012
0.59174
0.00012
sig
112
0.01892
(194415
00
039999
00
uk7
0.6500
0.00274
8.99175
0,0012
066639
0.00232
0.8000
DO00IS
unb
0,87931
0.00125
0.79511
000122
00
012063
0,5
041263
0.0
and
ud
0356533
000057
8 0.6277
0,00899
036122
000085
0.96112
0000833
063216
0.005T4
063214
BUSS
cab
0.70414
001255
0.1326
0.0300
0170033
0000015
8.6644
R00424
0.15904
BL02300
0.19017
DU00041
121
ses
0.38530
0:00157
0.00338
0.00033
8.96744
0.00047
sia
VI2
0.66304
0.20012
0.07687
0,06405
0.66304
0000112
0,56334
0U00012
0.06711
0
0.10832
DUORZED
smw
0.0011S
899446
00
0L99416
6.
our
9.3916
709916
0,0
0.916
0.0
wi
0.02005
0.00655
060434
000651
6.64451
BAOTSS
0.00999
0.0100
0,88636
0886366
0,0
031736
0.0
slu
sho
0.94574
0.00057
133350
0,01495
0L06325
000049
0.9%
0.29949
0.01249
0.99322
sem
3471
yal
0.97N5
0.0
0.8514
0,0
0.75
0.0
san
000013
0.987144
sup
YAS
0.96721
0.00025
0.93333
0.0
1.0
300
0.25105
0.00062
0.0
0,01333
0.53859
000013
8.9323
ILDOTIS
DO
0,00E52
0.0
DOONSE
stk
0.5381
0.00012
0L33405
OUD
0L5SSSS
GD
sto
712
1,9
9
0.
cum
0:00012
I.17647
(177547
0
0.
tho
Zatd.Cat Catn
0.71913
007015
0.81399
0,01002
0L73346
0,025
0.75177
B.03450
085431
0.0134
0,85303
D:01
rm
21A
ral.
0.6590
0.00775
0.54138
001807
0.66304
0.0049
0.67033 0.90915
B00004 0.0
0.3367
BL01499
0,44422
0U01001
z5b
0,0055
0.00012
Table 41: Comparison of GIotLID CLD3 UDHR benchmark (part 4)

---

with confidence threshold
GhiLIDMM
FTI76
GhoiLII-M 6-3
GieLLDD-M
FT17605.
PT
30039.3
UDEE Codes)
FII76 Codes)
FBT
SPR
FIT
FPR.
FET
FPR.
FI
EPR.
FST
FPRI
PAT
FPR.
9x
601
a2k
.5%
000
0.00222
001
0:91045
0.00154
0.9333
0.00114
0.93846
0.00095
0,85445
0.0
0.31805
0.0
(15985
0.0
0.95216
0,0077
0.64551
0,00522
0.96774
0.00951
(.9751
003097
092857
0,90909
0,
agr
0,31429
0,0039
0,33014
0,00225
052014
000024
4g
416
0,64516
B0085
0.65237
0,00108
RA6574
0,0077
-87
0.5589
0.0009
099364
0.00025
0.9666
00004
ass
4000
anc
2000
0D
0.55441
0.00083
0.26667
0.00276
0,25903
0,20099
00
0.125
0.0
0.0
0.99157
0.91887
0,0
0,89552
D.0338
0.64171
0000316
0.33023
0.00114
0.98361
000024
0.62288
0,66562
BJDODES
5e-05
Se-0h
4D-05
af
442
0,99157
0.0
09PLE7
0.
0.99187
0.0
amn
30D
0.93133
0,00013
00
0.0
0.97521
0,0001
0.47099
0.00003
007531
0,00013
0.0
0.63676
ant
0.00013
0.0
sohbory
G.MIS
0400850
0.23929
070177
0LSEES
0.00921
8.4188
000758
0.789 14
0.00864
0.20162
O 00077
hask
0,0000
0.00133
0.00088
0.0085
0.00096
0,00056
bam
bar
0.89682
0,007477
0,5098
0,06822
0.55319
000013
ban
ban
0.93361
0.0
098361
0.0
0U97221
0.0
tar
5c-05
tas
has
0:00116
0:92913
0.0010
0.0
0,9K335
0.0
0.62105
0,0033
055535
00
0194353
0.0
0
0,00262
0
0.9K335
0.00026
0.9ESES
0,00025
0.96383
0.00044
ben
1.0
0.0
0.97644
IL00014
1.0
0.00
1.0
0.0
1.0
0.0
1.0
0UO
tho
tho
0.78519
0,00039
0,76519
0.000E9
0U77273
BU00044
bin
bin
01 319927
0,00013
1.0
00
LO
0.0
0.0
0.0
bod
0,00313
3666667
0.00243
0
0.66667
0,0076
DUKCDS
hr
0.9K361
0.0
0.00104
0.00544
0,93361
0.
0.0
088131
0.00744
(19881
00000R
bac
buc
big
bug
0,95312
0.00064
090055
0,00551
8.9%
tal
bul.
0.96
0,00064
0.85294
0.00085
0.36
0,00633
LI756L
B00037
0
0.00555
0.0568
BJDCULS
trm
hum
0,58762
0.00006
0.51619
0,00015
CUMSS
000012
har
0.0004
9--05
cub
0/0
1.0
0.007S
0.43165
0.00445
0.00076
0.00499
0,64865
0.00239
0.81379
0.9731
0,00006
0,99255
00
0.0
ebk
0U00039
0,0005
0.00338
0.00024
0.00032
ebr
<bx
0,2133
0.0
0.
0.54321
0.9
cbe
the
0,77386
0.66
081081
0.0
<b
<tc
05747H
0,00013
057479
0,00013
0.0
cte
0.15462
0.0
0.00677
010333
og
cap
0,94367
0.00834
095357
0.0
0.60606
0.00345
0.0
0.85714
0.84615
0,0
chs
0,00013
0,3355
0.0
0.3
0.0
che
0100522
ctj
chi
0112227
0.00554
0.14246
0.00791
ILISWS
00014T
cak
dik.
0:9221
0,00013
0.93333
0.0
0L97479
0.0
ctr
ctv
dhe
0.
6.86184
0.86144
0.6154
0.0
0.88544
0.64125
0.32641
07A517
0.0
0.5217
0,0
cth
0,0
cab
0.93023
16
0J8375
0,00101
en
cm
0.00036
0.00167
0.00000
0,00152
0000344
off
oaf
0,34747
00
0.65736
0.0
0S411S
0.0
cor
IL0069S
0.00032
0.98385
0.0
0.00077
0.00099
0.93065
od
0,00005
0.00025
0.00112
<70
0.00006
0.00025
0.96361
BU00224
0,00056
0,85185
0.00025
0L90766
BU0024
1.0
0.0
0.0
L.0
0.0
cm
oo
0.00286
0,00051
cd
0,78431
0.0
0,75431
0.0
OL74747
0.0
cym
cym
1.0
0.22100
0.01999
0.0
0.0
0.6874
0.003
0
dis
dan
QU65922
BU00699
091304
1900
dre
0,96745
0U00013
0.35242
0,02099
099745
0
OL96TA5
0000122
0.3394
0.001
BJ00663
dg
0.71166
0.00566
0.3855
0.008S
0.5
dip
dig
dag
0.00120
0
0,000
000004
die
dic
0.96774
0.0
1.8
0.0
096774
0.0
0L05333
0L0
1.
0.0
LD
0UO
dmu
Oup
dyo
dyo
0.97391
0.0
0.7391
0L06401
dyu
0.57906
QU96003
L.00447
0.57906
0L97305
0,0
0
0J96755
0.0
enb
enk
enl
an)
1.0122
0,0015
0.01073
0,00588
0L0006
9c-05
eng
eng
0.35296
0,00253
0,0899
D. 19005
0.87218
0.001
0,8855
0001599
0. 0114052 0.55455
0,03713
0.31915 0,25000
I101 141
epo
0,96825
0,00051
0.18155
B.02503
IL976
0,00038
0.976
BU0037
0.0045
BJ00165
oc
0.75
0,00013
0,75789
0.00
0.73118
0.5
Table 42: Comparison of GlotLIID vs FT on UDHR benchmark (part 1)

---

with confidence threshold
GliLiDD-M
FTIN6
GhniLII-MMrr 6-3
Gisl 6
21176.0.3
FT176
3
UDER Codets)
FTI76 Codes))
FBt
FPA.
FiT
FPR
F11
FPR.
F1
FPR.
FS1
FPRI
FIT
FPR.
c4
el3.
0,60000
0.01017
0
1003866
BA3492
0.00571
0.7322
B00526
0.55
000416
0.9915
0501
0.5335
000072
098815
0.0051
0.988
003004
095032
0.00144
0,0
0000733
030215
000012
eve
0.98361
0.00025
095361
0.00225
0.98361
000024
0.3938
0.0
0.9500
00
0.9500
6.0
6ec
0.90152
0,0122
0.97999
000004
0.00004
0,00100
0.90944
003293
0.99154
96-05
0.991724
9c-05
501
0,97531
0,00035
037531
0.00025
0.95221
000024
51
1.8
0.0
10
0.0
1.0
0.0
En
0.36415
0,006
0.11397
0.04539
0.30252
0,00360
0.41311
002188
0.20228
0.02442
0,28704
0.01369
0.28771
00
026671
00
0.9
Ean
0.94118
0000311
09A015
0.99726
B00024
0,95238
0000377
SU17831
0.02607
0000112
0.29412
0011233
0.41311
0100789
0.9917L
0000013
0.944
0.00025
00
0.0
0.9616
0,9833
0,0
0.88762
0,0051
0.06335
0,00011
00
0036T
Ia
0.54
0
0,95955
0,00511
003087
for
0,70112
0.00744
0380615
0600126
000075
210
150
0.33846
0,0000
0.96615
0,00051
000005
205
2AG
0.93183
0.0
0922000
6.9
232
0001122
0
000055
24g
0.92065
0,94574
0.00129
0.00366
095512
0.00101
0200115
0.0
0.7451
0.001 18
9m-05
ald
gte
0,95
0,00100
077155
100000
0.98115
0,00363
0,96115 0L93305
0,79133
0.00041
0.M1597
als
0.00013
033832
096305
0,00013
1L00112
0.07566
0.00228
0.96643
0.0
gly
1.0
0.0
0046571
1.0
0.00
1.0
0.0
0.0
S--05
0,0
0,0
gom
pam
0.00051
0,00035
00:013
am
20
0.59649
0,00818
100007
0,67144
0,00564
0.68389
0L03779
0.31233
Sc-15
0.0
D9W
00000
0.5522
0,00064
1.0
0.0
0.90379
0.0
0,0
21k.
5uk
zu
257
0,52632
000
0.53613
0.0
0.44644
0.0
3at
hat.
1r.
0.960006
0,00322
9.45125
0L00137
0L9313
0,002E7
0L953122
00014T
0
0.00002
0.082
4G 05
330
han
0.9118
01002
0.96
0,00139
1197297
B000222
Inoe
0.0
hrshooooppeee
0,00335
0.68331
0.01666
0.00177
lubb
0.0
1.0
0,0
0.0
hif
0:000
90-05
0.00076
000003
bil
Nel
1.0
0.0
10
0.
1.0
0.0
hin
bin
0.62
0,009TS
0.33333
0
0A1312
0,00947
0008922
0.33335
0.014
0,0107
all
1d..
2292982
0.0
092992
0.0
0L89000
0L0
hms
hms
hrn
hni
0,89655
0,00005
0.91228
0.91228
0.0
359
1.0
0.0
0.77061
0000033
0.0
1.0
0.0
0.82555
0,68122
0,0
hun
0,82192
0,00355
0.58925
0.01263
0.34507
0,00278
0,8H552
0.007]]
0,57143
0,00414
0.90766
0.00222
lss
lus
0.98052
0,00239
0.99615
0.0
0.97459
0.0
han
0.9838
00
0.96552
0.0
0.0
bye
byye
by
1.0
0.0
0.9928
Se 05
1.0
0.0
1.0
0.0
1.0
00
1,0
0.0
atb
0,98718
0,00006
0.00855
0.00013
1.0
0.0
0,21339
01004336
0.24226
alu
0.00013
0
305
0,97872
0.00006
0.978722
0,00025
0.97572
000024
110001
1001
100000
Se 110
0
20
ala
alo
0,91339
0.00142
037466
10000
0.988
0.00114
0.97479
000037
00
0.04545
0.0
0.86171
000115
0.90229
0:00119
9884
0128571
8.9916
050211
0.9916
0:0011
09916
00:012
07083
0.01442
(19621
000120
0000733
518005
10530
0.32416
0,0000
001236
0.97581
0.0
000116
020055
003661
0.00337
0,48885
021244
0:50647
00144
007198
0,7861
021017
0.5
0.9225
80096
003929
0.68522
0.00607
8,9010
0
210
0.90667
000006
0,002S
0
ksl
tal
0.93386
010
0.90006
00
000
320
kan
kn
1.0
0.67366
1100259
10
00
0.9145
06
1,0
0,0
isl
00
1.8
0.0
1.0
ksc.
0,96221
0,00339
0.59007
0.00444
087531
0.00255
0,966677
0L00244
0.40645
0.00763
0.42491
0.0093
11p
0.55714
0.00257
0,90909
0.00152
US375
i1c
0.9915L
0,00013
0.0
kdc
0.98491
0.0
0.56401
0.0
0.5459
0.0
kth
kth
6.77551
0.0
0.7024M
0.9
Aca
0.72227
0,001
0.22727
0.00125
1
0,001
ack.
bck
0,00039
0.9916
0,00013
00916
B00012
0.92211
0.00005
098333
0.00115
0
Alm
Kam
0.0
1.0
kin
0.7626
00027
0381967
0:01522
0.67997
0.00059
0.69767
0.00230
0.IORL
0.00255
0.33059
000136
0.83099
0.0099
000344
kth
Mich
kub
Anbb
0.99194
0,00313
0.91944
0
0.99194
0000122
knr
km
8,0066
0.3981
D 000476
0.66657
0.00445
(166677
000721
0.5928
0.00772
0.6
0,0033
knc
kec
0.97559
0.00099
0,00225
097773
000004
3ng
kng
0,00004
0,00013
00
B00013
kim
000004
0,65002
005333
0.00025
0,911
0000012
0.65909
0000012
0.66151
0.0000
10
1.0
0.0
k2p.
0.20896
0.0
0,09534
0.0
0.60129
0.0
0.00322
00011
0.0044
In
0,96875
0.0
0.90875
0.0
0196003
0.0
Ad
0.02551
0.0
0.0
AIu
Atu
0.78812
0,00747
0,57944
0,004044
0.54369
BL00667
Table 43: Comparison of GlotLIID vs FT on UDHR benchmark (part 2)

---

ITTT
Glvit-1-32 $:.5
F11N
3.
D.YY5A
0.0
WORES
CS1S33
00055
0.6
0--55
CO
IUPS
DARRITS
MORIES
230134
CD
0.20014
46-15
0.48113
4.
In
54
0,39145
D0055
14
06
96
DAIIO
CA6218
200033
MO3D1
030321
600053
DARRVI
000211
003897
DABm33
1L0005
4245
0.3994
00313
0.0
3.0
000011
0.03355
94-05
CO
DUAEE
DJR007
0.98145
log
18g
10000
00001
000004
(32224
30
00
vob
DO0GL
14
96
30
39115
000009
0.0
run
0.55575
00
0D
1120002
01334955
00006
11991
DREWS
mun
TUE
TUNZSS
000013
000012
ESC
QM
3U
CONST4
040004
(199354
050353
CU
23
200064
0364
G00012
BOCK
CH0511
200025
0L00022
L9915
06
230
UORES
PN-05
020322
0.0035
0.00116
00001
0O3913
5--05
11131
06
ROS
DARRET
22182
000057
BATHME
CARRGY
03542
EURICON
ra
00042
9--05
DO90
000011
1,8
50
9.
106
0.90N
DOYTW
AOOIGE
00015
OD
04-05
DC
454
150
15N
15
148
41
E.0130
O 10844
01 71004
0713ES
DUOSTS
033339
0.08322
0U8231
DUOKER
E33144
01R012
030222
0.52912
LUK226
46
109
ron
0L89326
ro
12
0.0
10
00
10
0.
QU
000022
UD
0.1111
UUKIZZ
30
CHRITE
FRR
0.0002
1/3
100
10
UD
DO
CD
(UA
DO0556
030309
000303
BINOEE
pa
4.
gon
pan
071739
0.0
00014
go-
20
DM
200
D00050
O EOOKS
ROYSS
CX0013
0.0014
peb
peb
S1603
106
1
14
46
per
per
GE9477
DIAGE
055001
DOWAR
06
[1]
(14
30
0.30855
86
DONAAL
0 06356
0199787
BANI CONTTE
CXRETS
023397
0.50014
005646
4-15
100013
Table 44: Comparison of GlotLID vs FT176 on UDHR benchmark (part 3)

---

with ceufifencce fhresood
GlaliIDM
FTI7s
G
GieL.nIM
0-.5
FT176103
FT11666.
UDBE Cedelct
1T176 Codeco
F11
FPR.
FRI
FPR
FT-
FPR
F11
FPR.
FPR
FF
FPR
1901
16
0.00011
038111
000001
1180977
0/00012
190
0,02666
037647
0.00674
0
000556
0.5250
0.0065
0.50992
0002007
03193
0
B157591
090219
0
000039
0401441
0:325
0.
0.125
0.0
019534
00
71
0.02065
0221522
000004
047433
0.088
1L5184
0.1408
022001
0.01556
02W22
sog
50g
0.0007
00
00
0.5333
5:-05
0.65667
0.0
0.66567
6.0
0.58335
0.51707
0
0:00101
0.000744
0
940
0.92683
0.03053
0.
0.00342
0.5
0.50015
0
0
0.0
0.0
0.0
0.0
607
00
00000115
six
00
038389
0.0
0,86792
0.
0.9145
0.
0.99145
0.00544
0.0025
000052
0,00885
1.
00
0:99174
Sc-05
0.0
L0
00
1.0
0.0
1.0
0.0
der
AAT
000193
0-48715
0 10000
0.9375
0000011
0.0088
0596667
Se-DX
0.97270
A3
0.96667
0.02025
0.96607
0.00025
0
0,00244
1.0
8.0
00
0.93866
0000003
099003
0:00053
0
sut
an
1L56918
0.56818
0.0
0,425
0/1110
00000
0,0171
0
0.00355
0.00013
0.5916
0.0
1.75
000000
000611
075515
1U%
0
11 188S3 11
08323
63M111
040059
sod
802
0.58722
000015
008055
000239
097651
520055
00002+
0.0
0.0
0.0
0.0
AFT
B192511
0.01144
0,00077
00
0.0012
0.5597
0.0000
0228901
0.0115
6991
02001
635
040004
0.58519
0
0,35443
sus
515
B89285
000303
0000%
0,0049
KW
BA53115
00027
0.11546
0 0.4455
03582
00024
029641
0.0088
0.30221
0.00067
0
oxh
0.000A
036615
ILSARS
0/0122
3M0
0.88112
0,00219
0.86713
0.00385
0.3333
000114
00
0.967T
0.00014
008413
45-05
80
LO
0,0
0:95215
0.00257
1.
0.0
1,0
0.
1.0
0.0
1.0
0UD
B1485S
0000922
042254
0
the
00
0
0.
00
ICA
0.916
03991
0.9
0.9916
0.0
0,0
1.0
1.0
BL97345
60
0,93445
0.0
0.94545
00
155.
0L86131
0,00445
0.67429
000000
092913
0,00114
0L9EN22
0,0066]
0,68605
0.00248
0.Z19E1
0,00205
121
0009104
0.944
IL97874
0.0
DL985S1
0.0000
0.0
00
tly
TV
sob
160
(L9636)
0.00006
099361
0.00025
0,00012
303
LD
0.0
1.0
00
19
038760
0:00075
LD
0.0
0.0
002001
099134
1.91661
0,00013
099566
0.00L
000012
101
1090161
000000
0.9174
090012
034135
050077
12
0387005
00005
000123
000000
0.53659
Sc-15
0.5906
0.47436
0.09553
00158
0,99004
010230
0.5009
000381
0.013
0.1471L
BUDOSSS
twi
0.05349
0.00554
095094
0:0136
03885
063005
By
BL92611
0.00005
0.55422
0.0
099174
000015
60
0.55422
0.0
0.35615
0.0
000006
099371
6-002
09976
0/0012
uea
0.01554
0.03699
0.07544
0:00557
0
0,0023
0.98361
0.00005
0.9174
0000133
0.9914
0,00012
BL91892
00
0.0
091401
00
60
088667
0
DAN667 0.MM
BO
skr
akr
12
000005
D 000EES
09E3
000RS
IL99144
000012
0L54007
0.0000
amb
0.0022
0000266
0
am
1L52963
032263
0.0
001001
D 1
0L70414
000274
0.75033
0000355
0
0,0444
0.57354
0.0006
0
0L00033
100
11
095204
0000076
IUWTT4
D
0100014
D. C16615
BO
VOI
L0
60
1.0
0.0
10
00
sep
WET
D0127
035055
000022
0.00013
0,00122
0.6455
0
0.6667
0.00025
097436
0.0
0.97436
0.
000022
0.00014
wor
War
0
0,0
0,2983 00508S
D B0015
0.9916
DD
099966
00
0.44372
0,008
0.36364
D.000S0
wha
BA22005
0000044
0 0000299
0.24344
0008844
0
0.08577
0.00200
0.0
0.00155
woll
ool
0.00111
038987
01074
0:001
was
0,0099
039636
0000111
0.0
0.00077
sad
0,8785
0.0
0-85714
0.0
0.75
0.0
san
300
0.95214
0000
0
sap
500
B197211
0000055
BD
00
300
M
L.0
1.0
0.0077
0.66411
000055
0
000039
000011
0591992
0.
0.5785
1th.
0,8381
0.00113
0.93415
0.0
D 1169563
GO
LH
00
1,0
1
0.
2211
0.0
0,00013
0.00345
0.84466
0:00051
0
0
zho
0L71795
0,01248
0,74955
0,02263
0.73345
0.06166
0.75177
0,0665
0.58255
0.00717
0.1715
0,00552
201
zel
B.60991 BL02553
0.00077 0.00051
0.63044
0:00151
8U67833
0.00004
25b
0.95
0.00015
0.94915
Table 45: Comparison of GlotLID vs FT on UDHR benchmark (part 4)

---

with confilunce threshohd 0
GloILID-M
OpenIIII
GioLLID-M 9c3
GleLID-M da,5
OpenLID 9-3
OpenIID da,5
3
UDBER Codeta)
OpenLIID Codesa)
FPR
FI1
FPR.
FI
FPR
F11
FPR.
F11
FPR
F17
FPR
0,50509
0.00125 26
0,71856
0,00277
0.91613
0.00107
0.91505
0.00003
0.8146
0.00247
0.38389
0.00075
0.95236
000068
0.8215
0.00066
0.9774
0.00043
0.91511
0.00511
0.9000
0.00086
0.95
0.00022
silc
a1c
a/e
0.86131
0.0078
0,32153
0.01331
0.7377
0.00162
0.56963
0.00134
0. 0.3668
0.01331
0472
0,07000
sensh
amch
amh
0.0
La
0.0
1.0
0.0
1.0
0.0
1.0
0.0
1.0
0.0
abb
abb
0.9033
0000111
0,02437
0.00011
092635
0.00111
DA381
0.0001
0.94017
0,0021
004526
So-05
dst
0,9753
0.99174
0000111
000011
0.06667
0.75472
0,0030
0,00117
091671
057521
0.00011
0.0011
1000110000
0.54615
0
6.0
0.0
0.81081
0.75472
0,00209
0.03177
0.5952
031633
0.0075
0100139
aub
a2b
a2h
azj
44]
44
0.744%
0000666
0,48163
0.03544
0,15789
0.00215
0,0008
0000113
0.4856
0,00544
0.4576
0.03333
bam
bum
bam
0
0000622
0,19403
0,2275
05095
0.55319
000433
0.20553
0,2225
125
QU01543
ban
ban
ban
0.9756]
000011
0.71195
00015
098361
033453
U.7215
0:00
bel
bel
hel
0.98335
00
0,51706
OUTTS
USKEST
0396335
80
035224
DOTES
0.35553
bem
bem
hem
0.9833
000022
0,00411
0000111
0396533
000022
0,009
047581
bon
hon
bon
L.
00
0
54-05
10
DO
hho
bhn
Mbo
0.78519
0000/44
0.75119
0.001
077811
0.72253
000021
0
000005
0.78105
bod.
bod
hed
B USSM
0000111
0.7611
100011
039001
000011
038881
0000
03555
0000111
G:451
hug
hoe.
hag
hes
bag
hos
0.18105
0,65412
000051
00000
0 026100
( 19891
100898
000051
004806
0. HAN
0
000064
080022
01600
000132
0.000
bul
eeb
cat
ceb
bul
ca
cob
bel
0.93025
696721
0.66
00000
000653
000011
1000000000
0000000000000
000001000000000
033890
030000
0.8385
00085
10005 1
00085
005335
02916
0.05
01005
000011
00005
000011
000N1
100
0
0102200
000855
00005
000001
00001
0 055338
0.00332
000025
(10044
qjs
ces
ces
<lk
005 7
0.9344
0.
000033
10
22003
0
110
0.3300
00000
000
0000000
000000
200022
1279
Se 15
osb
cih
cbb
ccb
cch
cch
0.97561
0.00054
081759
020130
0.0000
0
000022
088538
020135
000096
cym
com
<3tn
1.
100
081667
1:0
110
10
100
1000
059122
059122
dan
dan
din
023025
000251
210113
0
08817
deu
dya
dyu
dru
den
deu
023008
0007 19
00:100
888755
022000
0
0
000
dro
dm
000020
100000
0300000
0.811
0:0205
1.
00
00
1.0
0n
1:0
0.0
ong
csg
102101
0000
0
0 SEHHES
opo
cpo-
000000
U.76
2E00000
0.09222
cik
00125
00009)
033800
00120
1.8
00
001200
css
tap fuo
ene
owe
cus
CWS
094722
000020
0
030200
2000
LH0000
1
110
0.22911
000052
0,0055
1.0
0:26211
002222
0,200
0
0 028769
0 190015
010015
0
0:09156
100008
0,0007
1000
0
000000
000004
063776
0,0035
fav
027212
000017
0
002000
0
0000
gan
gr
gla
glc
1
00000
00011
000000
0
00000
0
0 016164
00002
000025
als
1000111
0
0,00010
110000
00000
646
8m
0,316
000022
0.0000
2M)
LD
10
18
1.0
0.90016
0
09313
0.95312
000124
0.65109
001216
001216
han
0.3448
00024
0,5752
0,01427
000000
0.5679
0.01
beb
beb
beb
4.99145
0
5:00
his.
U62
000867
0,00811
462312
0,6329
hun
hry
han
brv
hin
0,3392
0,002
000000
00009)
0,52252
0956259
0.0558
000488
76
00016
0000
0
000038
00005
hyg
hye
1.0
(0)
Ln
0.0
00
00
10
0.0
00
tho
0:98715
000021
06115
0.00485
0
0.0001
60 0
0.0048
0 074008
ind
ilo
ind
Hn
Ho
0.
0:3348
0:90399
000033
000411
005531
071166
00018
0.00322
0 05755
0.928
0
0.0065
0.795
000031
0.1014
6,5277
0,0118
0.0233
000188000000
a00145
000054
60916
000011
09916
6.9916
0000
80016
0000
6.9916
09916
Sodti
000022
067416
0003066
0.33116
0.00366
0.96557
000755
073771
0000000
0,0006
0.7441
jay
000S53
1
000045
(30055
1
0
000054
jps
001301
036444
0,00442
0.
071921
030844
0:045
kan
kan
kin
10
100
10
QU
10
60
Ou
1.0
kat
kat
Kat
1.0
100
0
58 U5
10
80
1.
56 05-05
1.0
up
kaz
kast
0.96721
000051
0,0SSS
097521
0000E1
0 0
000021
0
000888
khh
kes
kics
khp
kic
0.8714
0.72727
0000
000725
14
007627
000896
727227
I
000001
0
0
00000
0
007754
001757
ktic
k2k
kak
100
0.5
000881
86315S
0333533
G0
0L5699
000083
1
khh
1htm
kmm
1.0
D0
1a
0.0
10
10
10
00
10
0.0
1.0
00
kin
kin
bin
0.76156
00014
0,S1OKT
000238
031867
000120
BAS47
000072
088822
0,0038
1800
kir
bir
bir
0.94488
0.0008
0.3500
00117
0.00644
0.9774
10000
035714
0017
0 B1011
000997
kmds
krah
km/
0.991194
000011
0.82034
0,0268
899194
0.00011
0.50
00001
018678
000288
0100366
kns
Knc
kns
0.56667
0000673
0.61454
0,0997
B66667
0.00633
0.6667
B00005
0.63441
0,00977
86488
kne
knc 0.57709
0000354
U.5611
0,05442
097778
0.00011
0.97778
000021
0.63766
0,05422
0.7610
000214
kon
Kag
kon
0,40739
010
0,55172
0,0816
0
00
0,39725
0,0
0
050816
000816
036743
koe
kox
kor 0.931888
0,0008
083333
0,00129
0.96
0.00544
0.99174
0.000
0
0,00129
036774
(L00021
loo
110
lto
1.0
0.0
1.0
0.0
1.0
0.0
1.0
0.0
1.0
0.0
1.0
1ij
lij 0,49735
001324
0.63446
0,00805
U.33953
0.01052
0.6499
0000399
0.40052
0,00805
U.53111
EL00541
Table 46: Comparison of GlotLII vs OpenI.ID
on UDHR benchmark (part I)

---

with coufdheee threhudd e
GDnLLIIM
Openi 3D
Glei.I--I f:.1
Openl.m.66.
Opent de.5
6o039.3 3
UDIR Codeis) OpeaLIID Codees)
F1"
FPR
F11
FPR
F11
7PR.
FH1
FPR.
F1"
FPR.
F11
FPRI
lis
0.9825
020000 00
0.677C
0.9631
0.002)
0.92699
600172
0.8112
0.00011
038187
09005
19105
010053
111
lgg
1as
1ax
30
1ea
0311100
083541
01588
00007
0.0004
0000181
0.0001
00000110000000000000000000000000
04438
04
001000
001
001227
605306
0
035
0
020754
0,00043
037477
178866
0300)
0.0021
033155
0.3023
IL4775
08:50
06
090064
000001
0,8852
0.51
001323
08 087166
0.0055
ni
1931
075.00
0.0
03150
(0001)
100000
13399
0.
113055
000007
0100071
0.0
675596
00000
000011
00400
631544
00000
00000
nu
nd
mo1
839122
00001
000000
04
0
11
110
20
00000
02001
cin
ml
037740
000099
055
000455
mi
mm
00000000000000000000000000000000
000015
06227
00000
000000
00000
000003
0
0.5530
20000
1
1002
0
0000a
0000
000064
6312
500N
034115
ssh
000001
(US111
I 100001
spi
sa
180
00
50:00
0
02015
300
000000
000000
00
pan
18
01
110
8,0
60
10
0000
pap
100351
000255
S
000004
pes
12-
0000
$:48
gn
pol
10
0000
00000
1
0
033333
000052
000235
000
000051
000
6
C000
00050
C
qry
0
0000
00819
000014
00000
COL
0 1
og
on
00
00
00
211
BLAME
1.
000000
gy
10000
00006
0
0
000011
000011
sno
500
500
18
00
0
00002
110
110
800
000001
000012
10000
03533
60006
000065000000
098533
000053
071554
000N1
2000
00000
ad
ap
0
000663
0
020633
10
00
000033
000000
1
0
16
00002
1
036183
0200
10000
exh
30
ten
Sm
14
0000000
18
000000
1/8
1
000000
12
00
GATES
10
00
10
on
on
16
00
on
000011
1005
0000
so
uA
000183
R200
00000
0
00011
000700
0000
tar
MONS
sic
skr
de
936361
00000000000000000000000000000000
000003
0
70007
60
000221
0000
80
0.6660
0.96
000007
000007
00
anb
and
mnh
ud
0000
1000
01
600001
0.6550
sic
122
0
000:
061851
DO0DS
W37
00
09200
00000
09916
30
0/006
0
sol
wol
041755
0000000000000
vial
sal
000091
0.0
00
1165255
0
0.00543
0
060031
0.59177
6,0000
08
099187
00
20
yee
sin br
yer
$38305
096345
0.0001
D0004
0.02400
0.38553
0.84611
0.00224
0.96306
(U6618
800061
0.0
0J5628
000081
0,0
0.11667
0.71366
0/61033
0.9466
0:179
0.00224
0,65 15)
0.8887
000321
0
Table 47: Comparison of GlotLIID vs OpenI.ID on UDHR benchmark (part 2)

---

with confidence threshold
GlotLID-MM
NLLB
GloILID-M& 9-3
GledLID-M
d
NLLB 6-33
NLLB
UDHR Codeets)
NLLLB Code(s)
FRT
FPR
FI"
FFR
FH
FPR
F11
FPR
F11
FPR]
abk
abk
abk
6e:05
0.9916
05
19916
Q.U
skn
amch
als
all
arb
afr
ady
ady
acc
.fr
1111
<dy
0.0000000
0.504400
0.56230
0.00909
1
1.1
18
0.000%
0,0000
0.00068
000011
0.00126
0.0
0.0
0.0
0
0.5314
0-45096
060894
00000
200000
000055
000067
000031
002
0
0.96366
0
0,96774
0,91683
0,53425
07877
1.0
1.0
0
000002
0
0.00647
00
00
0
0
091663
085405
09756
099666
0.838
1.0
1.0
0.001
000001
0.00108
1
0.0
0.0
0.0
0
0.6520
0.57005
0.715
0.4165
0.91477
0.63153
0.9625
0:96333
000003
0.0375
000264
000067
0.0057
000375
0
0,62494
0.7201
028225
162434
0.6
0,00510
0.001
6.045
0,00088
0,00513
0.0033
0.0038
00
ast
ast
am
221
km
255
0000000000000000000000000000000
(300011
100011
000001
(04400
037261
075666
0000000000
00007
0000755
000004
1397521
100000000000000000000000000000
000011
QU0011
10
0
0393335
039615
00
0.0
0.0
0:50129
000000000000
000077
UX0055
087211
00502
0:0005
0:0055
0.00177
ben
bel
azb
hem
bon
azj
hem
421
ban
hem
0.
0.96333
00000
0
0300622
0.0
0,29911
044714
030012
0,7044
0.2254
0000042
0100229
000967
0000511
000155
0
098833
0.93361
0U0211
0:005
110
000215000000
110
0
0.55319
0
0003006
098313
0
0000335
0.0111
0.0
0.0
0
0:2265
0
0000442
000051
000555
0
1
0:00355
6:0094
0011155
ben
bos
His
bho
bug
bod
bho
bod
hen
hic
bic
0.78510
0.
0,59091
L0
L0
0.000
10000000000
0.00000
0.000%
000000
0.0
01
00000
10000
0
0,78519
L.
1.0
(0005)
100
0
0
0.9%
L.
0000
000000
0.0
0.0
671855
066116
1992
00000
000001
65
0,512
0.0055
00000
0:0157
0.000
0.0
bul
bull
000000000000
0000
095218
0.96
10000
0000
ceb
zes
010
00000
00055
000
10
000
chy
000034
0
000000
000652
0.0
0.5740
cáb
0.975
0.000
0
0.00000
0.56
0000000
000000
LG
00
eym
000000
1.0
100
1.0
0.0 0.0
0
dai
000001
000000
dyn
dyu
0220000
000000
017260
000000
doo
deo
eng
<ng
eli
0.0
0
0
092385
0.9984
opo
000000
est
dkk
000000
ess-
eus
0
owe
290
dao
tao
00
0,9220
00
fri
dui
for
foe
for
fra
Bio
G)
Tru
0.941
10
00
0
0
0
00012
100000
0,020
00000000000000000000000000000000
000001
00000
000000
0000
000011
00
0
06387
002200
(001)
0.221
LETOD
gur
for
gax
fus
fux
1
04180
000077
028220
00000000000000000000000000000000
01015
0
00000)
gle
gla
glo
0.5
08225
0980
000776
000011
080312
0
000220000000
00001
0200ER
00
0
08713
0008
00002
alg
gm
6/6
0.999955
0.500
0.0023
020011
06
001286
0.82977
11001D
10
0
6 1300
0000
007780
hut
auj
g0)
0.90706
100
0
00
068765
100
000998
09913
L.
10
02012
1D
100
1:0
hus
0100448
00024
032556
0.96
197207
000000
hcb
heb
beb
0.99145
0
0.0011
1.8
LI
1.0
100
hin
hin
0.62
0
0,009
0,22372
0,0232
0
02007A
hun
hrs
hun
hav
bun
0.32192
0,00297
0.0000
0
0.000
0.00
0.54507
000084
000046
0,0027
0000
00000
hye
10
010
097728
00
00
0D
tha
0
1
0 50677
00002
0,99355
000011
0. 53666
155154
0
ind
slo
1lo
0
0.WT
00000
0
0,
0,00086
00000
00001
100099
020011
0
0
(10867
0:001
1155115
ist
1300011
0
0.7612
0
000011
039166
0
U20222
10016
jir
kil
ita
ign
100
00110055
1300016
(10
0.1111
0.36695
011119
0:13
000KS
8000555
000022
033916
0.9881
000000000000
000013
000256
110
036087
895561
1000000000000000000000000000000
100001
au
0
0
0.3244
0
00000
QU082:
(00083
U3557
030022
000591
kin
kan
T0
1.0
60
100
100
kit
10
QU
1.0
110
10
10
E
kus
kaz
000004
0.00034
0421
00000
D43491
00085
khi
khp
kea
khi
2bp
1hp
kmm
0.35704
0300255
0001114
06
0
0 0,27888
071551
00000
000868
10
33533
00
0
0,71356
00000
32
00142
khon
1hm
Ehm
10
0.6
0.6625
000032
100
110
1.0
0.0
1.0
000022
1.0
00
kin
km
kin
0.76316
0.0044
082258
0822580000
000055
0,8547
000022
000073
D 0.00055
kir
kir
kir
0.34488
0.0008
0.57416
00041660000
000498
000468000
000064
496744
0.0041
0.58557
0.655
0.00%
kmb
kmb
kmb
0.99191
0.00011
0882622
000229
0000290000000
1
000011
099194
0U001
0,83226
0.928
kmr
kmr
0.66667
0.00673
0.47773
01177330000
000721
00021100000
0.6667
0000633
066667
000808
0.50652
21
0
knc
Kio
0.97059
0.40789
0.00034
0.0
0.45199
0.00444
0.00333
0000330000000
00004400000000000000000000000000
0,40749
0,00021
0.0
097778
0.00211
1000211000
0,57021
0.4156
100033
0.3127
000007
kar
0,34188
0.0008
0.51064
0106643
0.00054
0.0001
0,00310
lao
0.0
0.992
65:05
1.0
0.0
1,0
0.0
10
Table 48: Comparison of GlotLII vs NL.BB on UDHR benchmark (part 1)

---

vid confidence dhrehhldd
GanLID-M
SLLB
GiaLIIM fn
GiaLID- 6n.5
NLLB
m-33
NLLBK 0
LIDIERCCOON Cedrici NELE Codees)
VRT
FT.
1PR VFR.
PTC
FPR.
F11
FPR.
FE?
FPR.
FT"
Ea
19
111
111
14.
11:1 534
100000
000000
0395
00000
0.89951
18
0.0011
0000A
0.0
0.00011
100001
0.0
0.5000
0.52862
B101467
100006
035305
05
12a.
lag
155
Ies.
105
100
1000
10000
110200
100001
10070
080
C133
000000000000 16
0
000000
50010
00000T
00005
00045
1000 10550
00000
0
0.000
0.00223
092141
019011
063158
0
0.0356
000300
10000
04
0
001330
0000
2000
nic
mkd
mdd
110
1993
68
90
00
000000
00000
00
0.0
B0
mi
mli
nos
100000
000000
0000
0
0000
1000000000000000000000000000000
10000
16
0.0
00
aid
100
10000
8.2000
Po 05
100
sp
sya
eci
00
1 1
css
pan
10
100
00
rop
pen a
00
0:
pol
0
0
Notter
0:116
10224
2
00
60
10
67700
0.8905
B00022
09
1000000000000000000000000000000
0000
33000
ad
00
00
19916
00
10000
00
000
1000
1
10000
100
00
00
smm
1M
I0
00000
119
10
10
00
10
170
11
T1
00
10
10
00
1.
69
10
00
ten
sir
BO
sub
1
rnl
00077
023355
100
sin
000000
wol
sbo
vol
09852
6
0
0.0003
0
0M
1.0
100
10
230
1ho
10r
085106
-
0.0011
00024
00
00004000000
0
0.4MAT
0
800811
1
888829
-
000161
0.0
0.0
000033
09
00
0300L3
064193
1100
Table 49: Comparison of GlotLID vs NLLB on UDHR benchmark (part2) 2)


Reviews:
[{"review_id": "UsJj2AAOTM", "reviewer": "Reviewer_TXgM", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their insightful and helpful comments!\n\n**Reasons To Reject**:\n\n> 1_The_paper_uses_only_off-the-shelf_models_for_evaluation,_so_the_contribution_is_limited.\n\n- We agree that our contribution in terms of novel deep learning / machine learning methods is limited. However, this is not the goal and focus of the paper.\n\n- Please note that, in section 2, we explain why these models were chosen. Our main use case is  corpus creation and because of the requirements that we explain (broad coverage, open source, ease of use, uncertatiy assessment, efficiency and granular flexibility), we were not able to find viable alternative options. In addition, these models are the only ones that have gained sufficient trust of the community and are used in most corpus development.\n\n- Our work is the first to rigorously evaluate these models in different setups: eg. when the base set is known and when it is not (B?/B!), when confidence thresholds are used or not, etc. None of these setups were systematically considered in previous works.\n\n- We hope the reviewer can acknowledge this work in language identification, not only for its coverage of a broad set of low-resource languages but also for its substantial contribution to the design of evaluation and baseline selection.\n\n> 2_The_proposed_FastText_system_still_suffers_from_issues_like_poor_separation_of_related_languages.\n\n- Yes, the fasttext model, along with its many attributes, shows some limitations in effectively separation of very close languages, such as dialects. To address this challenge, our model is designed with the capability to adjust its level of granularity. For such languages, the model can provide predictions at a macro level by adding the probabilities of its varieties to macrolanguage probability.\n\n**Typos Grammar Style And Presentation Improvements:**:\n> 1_FastText_is_cited,_but_it_would_be_worthwhile_to_add_a_sentence_describing_what_kind_of_architecture_it_uses.\n\n- We will add a more detailed description of the FastText architecture.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The dataset associated with the paper, GlotLID-M, will likely be useful to Language ID researchers in the future. Their evaluation procedures are also designed not to make LID too easy, explicitly attempting to mark test data that doesn't belong to any of the languages of interest (for which the classifier will output labels). This is possible because their dataset includes 1832 languages, but the set of interest for classification only includes 1665.", "reasons_to_reject": "The paper uses only off-the-shelf models for evaluation, so the contribution is limited. The proposed FastText system still suffers from issues like poor separation of related languages.", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper's main contribution is the collection of a diverse 1832-language dataset (many low-resource) for Language ID, GlotLID-M. Unlike previous efforts, the data is not scraped from the web - the authors note that scraped corpora such as mc4 have many data quality issues (Kreutzer et al 2022). \n\nThe authors train an off-the-shelf FastText classifier on the data, and compare the results on other LangID datasets (like UDHR) with a few baseline LangID systems (like CLD3). Results are comparable or slightly better in terms of F1 measures.\n\n", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "FastText is cited, but it would be worthwhile to add a sentence describing what kind of architecture it uses - I believe the classification itself is multinomial logistic regression. This would facilitate comparisons with the other baselines."}}, {"review_id": "tgx2VMjfIF", "reviewer": "Reviewer_6ooP", "summary": "", "questions": "A) Exactly which sources are used to make the training dataset?\n\nB) Can you show that the UDHR test set does not appear in train?\n\nC) How do you handle \"undetermined\" labels when calculating the metrics?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their insightful and helpful comments!\n\n**Summary**\n\nTo summarize our response:\n\n(i) UDHR is the best benchmark available for low-resource\nlanguages. As we show below it has very little contamination\nfor low-resource languages. So as an evaluation for\nlow-resource languages (as opposed to high-resource\nlanguages) it serves our goals well.  All prior work has the\nproblem of contamination of UDHR for high-resource\nlanguages. We believe we should not be penalized for\nfollowing established methodology and for focusing our\nevaluation on low-resource languages (which are the point of\nour paper).\n\n(ii) We clearly state that GlotLID's core support\nis for the 1665 languages with low FPR: see L432-437. We do\nnot overclaim that we support all 1832 languages (but it's\nimportant to still train on them to minimize close-couzin\nerrors).\n\n(iii) We agree that we should have used formal notation to define our decision rule.\nWe will add the following to the paper:\nLet $p' = \\max_{l \\in M_i} P_{m_i}(l|s)$ and $l' =\n\\argmax_{l \\in M_i} P_{m_i}(l|s)$.\nHere $s$ is the string to be classified and $M_i$ is the set of classes that the LID $m_i$ predicts.\nWe assign $l'$ to $s$ if\n$p' > \\theta$ and undetermined otherwise. $\\theta$ is the\nclassification threshold.  For setting $B$!, we additionally\nset all probabilities of languages not in $B$ to zero before\napplying max/argmax.\n\n\n**Reasons To Reject**:\n\n> '1_I_do_not_find_their_UDHR-based_test_set_convincing. A_brief_search_found_several_examples_of_Wikipedia_pages_in_different_languages_containing_the_UDHR_in_whole_or_in_part.'\n\n- The purpose of adding UDHR baseline besides the flores-200 is because of the many low-resource languages that flores-200 does not support. \n\n- Here, we calculated the contamination as follows. We count a UDHR test sentence as occurring in our training data if all of its word fourgrams occur in the training data.\n\n- For 292 languages: none of the UDHR test sentences appear in our training data.\n\n- For 57 languages: less than 10 percent of UDHR test sentences appear in our training data.\n\n- The 25 languages with a contamination rate over 10 percent are all high resource languages.\n\n(292+57+25=374 -- these are all our UDHR languages)\n\nSo UDHR is an excellent benchmark for testing the performance of LID for *low-resource* languages (although not for high-resource languages).\n\nWe will discuss and resolve these contamination issues in the revised version. Thanks for pointing them out.\n\n> '2_key_details_missing_or_unclear_about_the_test_set'\n\n> '2_1_such as the number of languages covered (section 5.1 mentions 419 translations but table 1 says 374)'\n\nIn all our evaluations, GlotLID as well as the baselines can only make predictions for languages that they are trained on. GlotLID's training set contains 374 of the 419 UDHR languages, so GlotLID can only make predictions for those 374 languages. The baselines are treated the same way.\n\nHowever, it is very important to point out that we do *not* remove any language from UDHR test. So the 419-374 languages not covered by GlotLID are still in UDHR test and GlotLID generates many false positives based on them. We view this setting as the only realistic model of how LID is used in the real world: there will always be data from languages that the LID was not trained on.\n\n> '2_2_the number of lines after the filtering described in section 5.1 and the length of these lines given the shortest ones were discarded'\n\n- UDHR consist of 30 short articles, and we do the filtering only to remove short sentences such as: \"Article 1-30\". We will add more details on length distribution in the appendix of the revised version.\n\n> '3_The_authors_claim_on_line_454_that_the_significantly_lower_F1_scores_on_UDHR_are_due_to_domain_shift_and_an_increase_in_languages,_but_I_feel_that_more_analysis_is_needed_to_support_this_claim_(e.g._error_analysis)._It_may_be_that_there_is_some_undetected_flaw_in_the_test_set.'\n\n- Yes, as said in L308-312, there are some flaws in the UDHR data. Some of these issues have now been resolved, we will make them available in the revised version.\n\n> '4_I_think_the_disparity_of_data_should_be_made_clearer_(e.g._the_authors_should_say_x_number_of_languages_have_more_than_1000_lines_of_data,_with_y_additional_languages_with_smaller_amounts_of_data_'\n\n- We explained in L204-205 for 1677 languages we have more than 1000 sentences and additional details for each language are available in Appendix §B. This of course means that for the rest of languages (1832 - 1677) we have less than 1000 sentences. We will make it clearer in the revised version.\n\n> '5_It_is_also_not_clear_at_points_how_many_classes_are_in_the_model_(line_72_claims_the_model_covers_1665,_but_table_1_tests_on_1832_at_most)'\n'\n\n- We explained in L219-224 our model always runs on all of the 1832 languages, however our claim for the number of languages that GlotLID supports is only for the 1665 languages with low FPR (L432-437). The reason we always use all 1832 languages is to mitigate the out-of-model cousin errors.\n\n> '6_It is not clear how the authors chose which languages to support: line 1204 says that 1667 languages have more than 1000 training sentences, but the tables in the appendix report scores on the GlotLID-C test set for languages with very small amounts of data'\n\n- We explained in L219-224 our model always runs on all of the 1832 languages, so we report all of the scores. Choosing 1665 languages as said in L432-437 is based on the low FPR. \n\n> '7_In_line_621,_the_paper_does_mention_that_there_should_have_been_a_much_higher_threshold_for_the_amount_of_training_data:_I_would_agree_with_this,_since_it_would_likely_result_in_a_stronger_system_with_less_danger_of_'representation_washing'_by_over-claiming_the_number_of_languages_supported.'\n\n- Yes, as mentioned after L621 the number of 1665 languages that we use throughout the paper already reflects this insight. Even though we train on 1832 languages, we claim reasonable performance for only 1665 (Table 1). We agree that ideally we would have several tiers of reliability and also provide a system with (say) 1000 languages that has even higher reliability than we have for the system with 1665.\n\n> '8_Finally,_whilst_I_think_that_comparing_with_several_strong_baselines_is_good,_it_is_currently_difficult_to_make_a_direct_comparison_between_the_different_models_due_to_a_lack_of_clarity_in_the_evaluation._I_understand_point_about_testing_on_the_full_set_of_languages_to_make_the_set-up_more_realistic_(section_6.1),_but_I_think_making_a_direct_comparison_is_misleading_since_a_higher-coverage_model_will_always_'win'_in_the_naive_case.'\n\nWe agree that we should have used formal notation to define our decision rule.\nWe will add the following to the paper:\nLet $p' = \\max_{l \\in M_i} P_{m_i}(l|s)$ and $l' =\n\\argmax_{l \\in M_i} P_{m_i}(l|s)$.\nHere $s$ is the string to be classified and $M_i$ is the set of classes that the LID $m_i$ predicts.\nWe assign $l'$ to $s$ if\n$p' > \\theta$ and undetermined otherwise. $\\theta$ is the\nclassification threshold.  For setting $B$!, we additionally\nset all probabilities of languages not in $B$ to zero before\napplying max/argmax. In direct comparisons of a model $m_i$ and a model $m_j$ on F1, we\nrestrict the set of languages to $M_i \\cap M_j$. So we do not take an average of F1 over languages\nthat a model does not cover. The average is over the languages that both models support.\n\n\n**Questions For The Authors:**\n\n> 'A)_Exactly_which_sources_are_used_to_make_the_training_dataset?'\n\n- We reviewed sources referenced in L183-185 and ended up with more than 150 sources. \n\n- We cited the ones with the most contributions in the paper in L190-195. We will cite/mention all of the sources even with small contributions in the appendix and in the code repository.\n\n> 'B)_Can_you_show_that_the_UDHR_test_set_does_not_appear_in_train?'\n\n- Answer is given above.\n\n> 'C)_How_do_you_handle_undetermined_labels_when_calculating_the_metrics?'\n\n- Answer is given above.\n\n**Missing References:**\n\n> '1_Brown_generated_a_very_similar_high-coverage_training_set:_https://www.cs.cmu.edu/~ralf/langid.html'\n\n- This reference has been cited in L127, both the paper and the whatlang tool, which the mentioned link is refering to.\n\n**Typos Grammar Style And Presentation Improvements:**\n\n> '1_Granularity flexibility' (line 155) - interesting point, but unsure how this has influenced the choice of low-resource LID. Perhaps move to the discussion?\n\n- GlotLID-M, along with its many attributes, shows some limitations in effectively separating very close languages, such as dialects. To match this challenge, our model is designed with the capability to adjust granularity. For such languages, the model can provide predictions at a macro level by adding the probabilities of its varieties to macrolanguage probability. \n\n> '2_Tables_in_general:_explain_meaning_of_bold_and_underline_in_caption' \n- We will add this.\n\n> '3_1_The_font_is_way_too_small_for_the_tables_in_appendix_section_B,_even_on_a_screen._Perhaps_just_have_single_language_per_line_and_use_longtable?\n- We will enlarge the font.\n\n>'3_2_It_would_also_be_good_to_clarify_if_the_number_of_sentences_refers_to_train_or_train_and_test.'\n- It is referenced before train/test split in L205. It's for both. We will clarify it.\n\n> '4_Line_250,_it_would_be_good_to_expand_a_bit_more_on_how_fastText_is_explainable'\n- We will expand on this using the extra space granted.\n\n> '5_Line_424,_not_necessarily_the_'cousin'_problem,_could_just_be_out_of_model'\n- Out-of model cousin occurs when any language that is not supported by the model interferes with some related languages that are supported. We acknowledge that unrelated out-of-model languages may also interfere. However for the sake of simplicity we called both as out-of model cousin. We will clarify this point.\n\n> '6_Paragraphs_on_lines_531_and_line_548_seem_to_be_implying_that_the_training_data_has_mixed_languages._Is_there_evidence_of_this?'\n\n- We did a number of audits and did find cases like this. For example, the Farsi training data contains a fair bit of Arabic (because Arabic phrases and citations are frequently used).\n\n> '7_Line_578,_what_script_is_the_Cherokee_data_in?_Latin?\n- Our Cherokee training data is in Latin.\n\n> '8_Line_714,_expand_in_limitation_section_on_why_corpora_from_religious_sources_are_a_limitation_in_particular'\n\n- Because there is often a domain mismatch between religious sources and the text we want to apply LID to, e.g., most text found on the web is not religious.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1) Openly-available dataset including a large number of languages, plus a fastText LID model trained on this data\n2) Clear list of desiderata for low-resource LID models and how their choice for comparison LID models fits into this (section 2)\n3) Large number of strong baseline models\n4) Careful justification for some of their evaluation measures: for example, their argument for their evaluation scenario in section 6.1 and 'cl' in section 7 with relation to their chosen task of corpus creation.", "reasons_to_reject": "I think that the main reason to reject this paper is that I do not find their UDHR-based test set convincing. Firstly, I suspect that it appears in the training data for the model, specifically in Wikipedia (line 190). A brief search found several examples of Wikipedia pages in different languages containing the UDHR in whole or in part ( https://en.wikipedia.org/wiki/Dignity, https://sco.wikipedia.org/wiki/Universal_Declaration_o_Human_Richts, https://eo.wikipedia.org/wiki/Universala_Deklaracio_de_Homaj_Rajtoj, https://arz.wikipedia.org/wiki/%D8%A7%D9%84%D8%A7%D8%B9%D9%84%D8%A7%D9%86_%D8%A7%D9%84%D8%B9%D8%A7%D9%84%D9%85%D9%89_%D9%84%D8%AD%D9%82%D9%88%D9%82_%D8%A7%D9%84%D8%A7%D9%86%D8%B3%D8%A7%D9%86, https://es.wikipedia.org/wiki/Declaraci%C3%B3n_Universal_de_los_Derechos_Humanos). This contradicts the claim on line 276 that the test data is not part of train. I appreciate that the FLORES-200 test set is also sourced from wiki articles, but most of it consists of new translations. In addition, there are key details missing or unclear about the test set, such as the number of languages covered (section 5.1 mentions 419 translations but table 1 says 374), the number of lines after the filtering described in section 5.1 and the length of these lines given the shortest ones were discarded. Finally, the authors claim on line 454 that the significantly lower F1 scores on UDHR are due to domain shift and an increase in languages, but I feel that more analysis is needed to support this claim (e.g. error analysis). It may be that there is some undetected flaw in the test set.\n\nA secondary reason to reject this paper is that I believe it currently over-claims or is unclear on the number of languages the model supports. The dataset contains at most 1832 languages, but many of these only contain a very small number of sentences. I think the disparity of data should be made clearer (e.g. the authors should say x number of languages have more than 1000 lines of data, with y additional languages with smaller amounts of data - see https://www.cs.cmu.edu/~ralf/langid.html for a similar corpus). It is also not clear at points how many classes are in the model (line 72 claims the model covers 1665, but table 1 tests on 1832 at most). Even for the smaller number of languages (1665), it is not clear how the authors chose which languages to support: line 1204 says that 1667 languages have more than 1000 training sentences, but the tables in the appendix report scores on the GlotLID-C test set for languages with very small amounts of data (e.g.  Dutton World Speedwords, Evenki, Fe' Fe', Kukna, Lezghian). It is doubtful that a LID classifier could form a useful representation on such a small amount of data. In line 621, the paper does mention that there should have been a much higher threshold for the amount of training data: I would agree with this, since it would likely result in a stronger system with less danger of 'representation washing' by over-claiming the number of languages supported. \n\nFinally, whilst I think that comparing with several strong baselines is good, it is currently difficult to make a direct comparison between the different models due to a lack of clarity in the evaluation. I understand point about testing on the full set of languages to make the set-up more realistic (section 6.1), but I think making a direct comparison is misleading since a higher-coverage model will always 'win' in the naive case. I thought the confidence filtering/thresholding aspect was interesting, but it was unclear how 'undetermined' labels affected the results, especially F1. This is particularly important when comparing with models missing languages, as for them a 'successful' result on a language they don't cover is to return no label at all. The paper should make it clearer how the 'underdetermined' label feeds in to the metrics.", "questions_to_authors": "A) Exactly which sources are used to make the training dataset?\n\nB) Can you show that the UDHR test set does not appear in train?\n\nC) How do you handle \"undetermined\" labels when calculating the metrics?", "paper_topic_and_main_contributions": "The paper is concerned with creating a corpus for training language identification models which covers as many languages as possible and presents a model trained on this corpus. This is for the purpose of creating corpora for low-resource languages. \n\nThe main contributions of this paper are the following:\n\n- A dataset intended for training language identification models, which includes 1665+ languages\n- A list of desiderata for low-resource language identification models\n- A fastText language identification model trained on the above dataset\n- Comparison of the trained model with four strong baselines\n- Analysis of the model's performance with a focus on their intended application of corpus creation", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "Brown generated a very similar high-coverage training set: https://www.cs.cmu.edu/~ralf/langid.html"}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "- 'Granularity flexibility' (line 155) - interesting point, but unsure how this has influenced the choice of low-resource LID. Perhaps move to the discussion? \n- Tables in general: explain meaning of bold and underline in caption \n - The font is way too small for the tables in appendix section B, even on a screen. Perhaps just have single language per line and use longtable? It would also be good to clarify if the number of sentences refers to train or train and test.\n- In line 250, it would be good to expand a bit more on how fastText is explainable (line 250) \n- Line 424 – not necessarily the 'cousin' problem, could just be out of model \n- Paragraphs on lines 531 and line 548 seem to be implying that the training data has mixed languages. Is there evidence of this?  \n- Line 578 – what script is the Cherokee data in? Latin? \n- Line 714 – expand in limitation section on why corpora from religious sources are a limitation in particular"}}, {"review_id": "Ci531RTny0", "reviewer": "Reviewer_9oC8", "summary": "", "questions": "1. In Section 3 Preprocessing, how exactly do you ensure that? Do you train a model on the dataset from vaan Ensh et al?\n2. In the L233, its mentioned that fastext is good as it provides prob estimates. but how do we know that they are well-calibrated? It would be good to report the calibration metrics such as  Measuring Calibration in Deep Learning, Nixon et al.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their insightful and helpful comments!\n\n**Reasons To Reject:**\n\n> '1_It_would_be_good_to_add_statistical_deviation_of_the_model_across_say_3_runs?'\n\n- In contrast to typical deep learning models, fasttext variations across runs are extremely small, so we deemphasized this investigation for the submitted paper.\n\n- We will add mean and std of 3 runs in the revised version.\n\n> '2_Apart from results reported in numbers, it would be good to show some examples where the proposed method wins over existing methods and where it fails.'\n\n- We will use the extra space to add such examples of wins and fails.\n\n> '3_1_Since_corpus_creation_is_one_major_contribution,_it_would_have_been_nice_to_get_some_of_the_data_manually_verified_as_well,_I_know_its_not_cheap_to_do_that_but_would_further_ensure_that_the_corpus_is_of_high_quality.'\n\n- Manual verification would require native speakers: identifying, wetting and organizing the work of such a large number of experts would need to allocate a substantial effort to this task; instead we performed some automated verifications, for example based on the practice introduced in https://arxiv.org/pdf/2305.12182.pdf (CF2 in section 3.4). We will elaborate on it.\n\n> '3_2_Additionally_it_would_be_good_to_know_more_details_on_the_corpus_collected,_like_distributon_by_domain_etc.'\n\n- Granted extra space we will give more information about this.\n\n**Questions For The Authors:**\n\n> '1_In_Section_3_Preprocessing,_how_exactly_do_you_ensure_that?_Do_you_train_a_model_on_the_dataset_from_vaan_Ensh_et_al?'\n\n- Using the Unicode character database (https://www.unicode.org/Public/15.0.0/ucd/Scripts.txt), we identify the script in which each sentence is written. After that, we verify whether the combination of the script and claimed language for the sentence actually exists.\n\n> '2_In_the_L233,_its_mentioned_that_fastext_is_good_as_it_provides_prob_estimates._but_how_do_we_know_that_they_are_well-calibrated?_It_would_be_good_to_report_the_calibration_metrics_such_as_Measuring_Calibration_in_Deep_Learning,_Nixon_et_al.'\n\n- We will add calibration scores and reliability diagrams - note that for n-way clustering, calibration can be defined in several ways. We intend to use both (aggregated) marginal calibration scores (as in Nixon et al, 2020), but also confidence calibration metrics.\n\n**Typos Grammar Style And Presentation Improvements**\n\n> '1_It_would_be_nice_to_expand_FPR_in_L080_(F1_is_pretty_common).'\n- Sure, we will add this in the revised version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Extensive experiments and analysis, I especially liked how for each result/experiment/evaluation measure the authors have justified appropriately.\n2. Overall, the paper is well written and easy to understand, I liked how section 2 has clearly outlined the motivation of the work by using the titles in bold before every paragraph. Also, I liked how the authors have clearly explained the choice of their baselines, its a very good practice.\n3. The authors plan to release the code/data.", "reasons_to_reject": "1. It would be good to add statistical deviation of the model across say 3 runs?\n2. Apart from results reported in numbers, it would be good to show some examples where the proposed method wins over existing methods and where it fails.\n3. Since corpus creation is one major contribution, it would have been nice to get some of the data manually verified as well, I know its not cheap to do that but would further ensure that the corpus is of high quality. Additionally it would be good to know more details on the corpus collected, like distributon by domain etc.", "questions_to_authors": "1. In Section 3 Preprocessing, how exactly do you ensure that? Do you train a model on the dataset from vaan Ensh et al?\n2. In the L233, its mentioned that fastext is good as it provides prob estimates. but how do we know that they are well-calibrated? It would be good to report the calibration metrics such as  Measuring Calibration in Deep Learning, Nixon et al.", "paper_topic_and_main_contributions": "The paper proposes a language id method, specifically focussed towards covering low-resource languages and covers 1665 languages, which is an improvement over previous such works on lang id which are focussed more on the high-to-med resource languages. As part of this effort, they also curate a dataset covering these set of languages for lang id training.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "1. It would be nice to expand FPR in L080 (F1 is pretty common)."}, "extra_text": {}}]

Author rebuttal:

Meta-review: Summary (adapted from Reviewer 9oC8): The paper proposes language identification datasets and models, specifically focused on covering low-resource languages and covering 1665 languages, which is an improvement over previous works.

As reviewers pointed out, there are a number of things missing from the paper to fully understand the experiments and resource construction. For example, the paper does not actually contain a precise list of the sources for dataset construction, and the decision rule was not clearly explained. These are well-addressed in the authors’ response.

There are substantial risks of dataset mislabeling and contamination involved in creating LID datasets. The authors address some of these issues in the paper, and others are addressed in the response period. Particularly, the “reasons to reject” raised by Reviewer 6ooP all received clear responses, and I do not believe that they represent soundness issues with the paper, although they do reflect some information that should have been provided in the original submission.

The authors must ensure that any clarifications and information they provided in the response period are added to the paper in revision, as well as the formatting requests (the appendix results are unreadable).

The paper includes a substantial analysis section which explores the performance of the models and discusses some quirks of the assembled data. This is a very useful contribution on top of the rest of the paper.
Decision: Accept-Findings