Package: lingua-language-detector
1. What does this library do?Its task is simple: It tells you which language some text is written in.
This is very useful as a preprocessing step for linguistic data
in natural language processing applications such as text classification and
spell checking. Other use cases, for instance, might include routing e-mails
to the right geographically located customer service department, based on the
e-mails' languages.2. Why does this library exist?Language detection is often done as part of large machine learning frameworks
or natural language processing applications. In cases where you don't need
the full-fledged functionality of those systems or don't want to learn the
ropes of those, a small flexible library comes in handy.Python is widely used in natural language processing, so there are a couple
of comprehensive open source libraries for this task, such as Google'sCLD 2andCLD 3,Langid,FastText,FastSpell,SimplemmaandLangdetect.
Unfortunately, except for the last one they have two major drawbacks:Detection only works with quite lengthy text fragments. For very short
text snippets such as Twitter messages, they do not provide adequate results.The more languages take part in the decision process, the less accurate are
the detection results.Linguaaims at eliminating these problems. She nearly does not need any
configuration and yields pretty accurate results on both long and short text,
even on single words and phrases. She draws on both rule-based and statistical
methods but does not use any dictionaries of words. She does not need a
connection to any external API or service either. Once the library has been
downloaded, it can be used completely offline.3. A short history of this libraryThis library started as a pure Python implementation. Python's quick prototyping
capabilities made an important contribution to its improvements. Unfortunately,
there was always a tradeoff between performance and memory consumption. At first,Lingua'slanguage models were stored in dictionaries during runtime. This led
to quick performance at the cost of large memory consumption (more than 3 GB).
Because of that, the language models were then stored in NumPy arrays instead of
dictionaries. Memory consumption reduced to approximately 800 MB but CPU
performance dropped significantly. Both approaches were not satisfying.Starting from version 2.0.0, the pure Python implementation was replaced with
compiled Python bindings to the nativeRust implementationofLingua.
This decision has led to both quick performance and a small memory
footprint of less than 1 GB. The pure Python implementation is still available
in aseparate branchin this repository and will be kept up-to-date in subsequent 1.* releases.
Both 1.* and 2.* versions will remain available on the Python package index (PyPI).4. Which languages are supported?Compared to other language detection libraries,Lingua'sfocus is onquality over quantity, that is, getting detection right for a small set of
languages first before adding new ones. Currently, the following 75 languages
are supported:AAfrikaansAlbanianArabicArmenianAzerbaijaniBBasqueBelarusianBengaliNorwegian BokmalBosnianBulgarianCCatalanChineseCroatianCzechDDanishDutchEEnglishEsperantoEstonianFFinnishFrenchGGandaGeorgianGermanGreekGujaratiHHebrewHindiHungarianIIcelandicIndonesianIrishItalianJJapaneseKKazakhKoreanLLatinLatvianLithuanianMMacedonianMalayMaoriMarathiMongolianNNorwegian NynorskPPersianPolishPortuguesePunjabiRRomanianRussianSSerbianShonaSlovakSloveneSomaliSothoSpanishSwahiliSwedishTTagalogTamilTeluguThaiTsongaTswanaTurkishUUkrainianUrduVVietnameseWWelshXXhosaYYorubaZZulu5. How accurate is it?Linguais able to report accuracy statistics for some bundled test data
available for each supported language. The test data for each language is split
into three parts:a list of single words with a minimum length of 5 charactersa list of word pairs with a minimum length of 10 charactersa list of complete grammatical sentences of various lengthsBoth the language models and the test data have been created from separate
documents of theWortschatz corporaoffered by Leipzig University, Germany. Data crawled from various news websites
have been used for training, each corpus comprising one million sentences.
For testing, corpora made of arbitrarily chosen websites have been used, each
comprising ten thousand sentences. From each test corpus, a random unsorted
subset of 1000 single words, 1000 word pairs and 1000 sentences has been
extracted, respectively.Given the generated test data, I have compared the detection results ofLingua,FastText,FastSpell,Langdetect,Langid,Simplemma,CLD 2andCLD 3running over the data ofLingua'ssupported 75 languages. Languages that are
not supported by the other detectors are simply ignored for them during the
detection process.Each of the following sections contains two plots. The bar plot shows the detailed accuracy
results for each supported language. The box plot illustrates the distributions of the
accuracy values for each classifier. The boxes themselves represent the areas which the
middle 50 % of data lie within. Within the colored boxes, the horizontal lines mark the
median of the distributions.5.1 Single word detectionBar plot5.2 Word pair detectionBar plot5.3 Sentence detectionBar plot5.4 Average detectionBar plot5.5 Mean, median and standard deviationThe table below shows detailed statistics for each language and classifier
including mean, median and standard deviation.Open tableLanguageAverageSingle WordsWord PairsSentencesLingua(high accuracy mode)Lingua(low accuracy mode)LangdetectFastTextFastSpell(conservative mode)FastSpell(aggressive mode)LangidCLD3CLD2SimplemmaLingua(high accuracy mode)Lingua(low accuracy mode)LangdetectFastTextFastSpell(conservative mode)FastSpell(aggressive mode)LangidCLD3CLD2SimplemmaLingua(high accuracy mode)Lingua(low accuracy mode)LangdetectFastTextFastSpell(conservative mode)FastSpell(aggressive mode)LangidCLD3CLD2SimplemmaLingua(high accuracy mode)Lingua(low accuracy mode)LangdetectFastTextFastSpell(conservative mode)FastSpell(aggressive mode)LangidCLD3CLD2SimplemmaAfrikaans796467367073305555-58383711495012213-816266236774104656-979398749495809896-Albanian888079666666655565206954533535353318182195868466666663487717100999998989898989923Arabic989497969696919067-968894898989847919-999698989898909282-100991001001001009810099-Armenian100100-100100100949910022100100-1001001008310010036100100-1001001009910010014100100-1001001001009710014Azerbaijani9082-786985688172-7771-574367366234-9278-806990698282-9996-9894100989999-Basque8475-717171526261-7156-444444183323-8776-707070526269-9392-100100100869291-Belarusian9792-859295858476-9280-698187696742-9995-889498878687-100100-98991009910099-Bengali100100100989898929963-100100100949494929819-100100100999999889969-100100100100100100979999-Bokmal5850--697513--503927--53553--155947--707712--457775--859123--90Bosnian3529-9546553319-2923-954542194-3529-10647642815-4136-8446485236-Bulgarian87787278899267706668705651568083464532449181688188956266726799969699989993989391Catalan705854576366384838595133253342445194327460515763672942306287828683858881847981Chinese10010064717171969233-100100394646469092--1001005668686897832-1001009710010010010010098-Croatian736073477281484251-533649286264162634-745772427987384247-908697727693905873-Czech80717176768066647450665452586164443950318472737978836965804491878892889286889176Danish81707062767860585950614550355658332627208470685775786154564798959395989986959483Dutch776458787178645847585536275546553429113281614981708161474250969498100979998979092English816360969696855456655529229090908422122789625898989871445569999799100100100999710098Esperanto8466-767676445750-6744-5151515227-8561-797979305146-9893-100100100969898-Estonian9283837373736770657180626250505037412444968887737373676973701009910096979798999997Finnish9691939293938380777690778482828262584447989595969696888489811001001001001001001009998100French89777583838371554665745248626262422212349483788686867449486899989999999998948094Ganda9184------61-7965------23-9587------62-100100------99-Georgian100100-99999999981004100100-979797979910011100100-9999991001001002100100-100100100100961000German89807389898981666472745749767676614027389484709393938162667810099100100100100100989899Greek10010010099999910010010075100100100989898100100100741001001001001001001001001006010010010010010010010010010092Gujarati100100100100100100100100100-10010010099999910099100-100100100100100100100100100-100100100100100100100100100-Hebrew100100100100100100100---100100100999999100---100100100100100100100---100100100100100100100---Hindi73336887728860587756111447453774134562642060886589474576494679999969992959911Hungarian9590889292928376757287777380808064534158989491969696867685621001001001001001001009910095Icelandic9388-657071667166648372-394950334226439792-5764656670735910099-98999999999990Indonesian614780696877514662263925564352561626362061468468738254456326837110095789382668832Irish9185-606669636766778270-354147284229669490-576668646678769695-89939397949290Italian877177898989666244586942507474742831724927481929292705732571009899100100100100989394Japanese100100100878787869833-100100997272726197--1001001008989899696--100100100100100100100100100-Kazakh9694-887691808277-8988-725279676243-9894-908094788388-100100-10096100969999-Korean10010010099999910099100-100100100989898100100100-100100100100100100100100100-10010010010010010010098100-Latin8773-505050216246637249-242424-449339376-41414125842639794-85868661838893Latvian93878982828483757245857576656669645133369790928384868677843399979997979898989865Lithuanian9587878181818072706686767161616158423050988991838383857582621009810099999999999988Macedonian8472867486935160601366527151778315302712867088728396445470119995100100979994978415Malay3131-153952112218132622-143638211933836-19526492222102835-12295422342326Maori9283-----5261-8464-----2212-9288-----4372-9998-----9198-Marathi85398880875808483-74167761961706965-853089811569798486-96729899195919899-Mongolian9795-818589868378-9288-596672686343-9998-869194908792-9999-98991009999100-Nynorsk6652-29637032-54244125-842435-1866649-18587016-50229181-61879675-9345Persian90808190799292766112786264795784835713129480809281949470725100981001009899100999918Polish95908992929289777586857774808080735138729893939797979380878710099100100100100100999999Portuguese81696073818454535461594229476667192120268570547181854440486099959899969998979497Punjabi100100100100100100100100100-10010010099999910099100-100100100100100100100100100-100100100100100100100100100-Romanian87727764646461535457694956383838312411349274796060606048535199949795959592889686Russian907884949497757160667659708688926048265495848798979975726862989296100989991938783Serbian8878-765376647869-7462-544754396329-9080-765876637578-9992-985298899599-Shona9181-----7665-7856-----5124-9686-----7971-100100-----9999-Slovak84757465808368637168644950416364403238459078756281866661766699979891979897969993Slovene8267735975776363487261394832565733298488768725474786160427299939890969795999296Somali928590245152-6970-82647641820-3827-969095154648-7083-100100100528989-10099-Sotho8672-----4954-6743-----1513-9075-----3354-10097-----9895-Spanish705656746473654843504426255148523716121669494672607459323441979498100859498968592Swahili8170734141414257574660434777732516268468742424242449594198979992929298989772Swedish84726876798165615359644640515759353014298876677882856356526299949698989996969387Tagalog78667645464642-50125236511111112-9983677828282826-441198969998989898-9515Tamil100100100100100100100100100-100100100100100100100100100-100100100100100100100100100-10010010010010010010099100-Telugu10010010010010010010099100-10010010010010010010099100-100100100100100100100100100-10010010010010010010099100-Thai10010010010010010010099100-100100100100100100100100100-100100100100100100100100100-10010010010010010010098100-Tsonga8472------61-6646------19-8973------68-9897------97-Tswana8471------56-6544------17-8873------57-9996------94-Turkish94878286868667696676847163707070504130559891848888886770717810010010010010010084979796Ukrainian92868391959876817778847566789094546246629792849495987783887595939810010010096989997Urdu907983637580586161-80656740596830398-947883506874465375-969497999999999299-Vietnamese918793898989866663-7976817171716526--948798979797937490-999810010010010010099100-Welsh91828564697249697269786169354146114334589687886171743966856099999996969795989890Xhosa8269----536671-6445----134045-8567----496571-9894----969297-Yoruba7462-888-1537-5033-111-51-7761-111-1122-9692-212222-2888-Zulu8170----66354-6245----03518-8372----66351-9794----119293-Mean86788274778168696552746165586266484834348978827477826567685096939892919690939473Median89.079.082.578.079.083.067.068.063.059.074.057.063.557.561.067.041.541.026.533.094.081.084.081.081.086.067.066.071.560.099.097.099.099.098.099.098.098.098.090.0Standard Deviation13.1217.3413.4323.0719.917.024.6119.0418.5723.4618.4825.0123.7228.5225.3124.2232.3327.8628.7418.9413.1418.9515.6426.4521.6719.6728.521.8322.724.4811.0511.912.7819.4619.111.7820.2113.9512.2531.916. How fast is it?The accuracy reporter script measures the time each language detector needs
to classify 3000 input texts for each of the supported 75 languages. The results
below have been produced on an iMac 3.6 Ghz 8-Core Intel Core i9 with 40 GB RAM.Lingua inmulti-threaded modeis one of the fastest algorithms in this comparison. CLD 2, CLD 3 and fasttext
are similarly fast as they have been implemented in C or C++. Pure Python libraries
such as Simplemma, Langid or Langdetect a significantly slower.DetectorTimeLingua (low accuracy mode, multi-threaded)3.00 secLingua (high accuracy mode, multi-threaded)7.97 secCLD 28.65 secfastText10.50 secCLD 316.77 secLingua (low accuracy mode, single-threaded)20.46 secLingua (high accuracy mode, single-threaded)51.88 secFastSpell (aggressive mode)51.92 secFastSpell (conservative mode)52.32 secSimplemma2 min 36.44 secLangid3 min 50.40 secLangdetect10 min 43.96 sec7. Why is it better than other libraries?Every language detector uses a probabilisticn-grammodel trained on the character
distribution in some training corpus. Most libraries only use n-grams of size 3
(trigrams) which is satisfactory for detecting the language of longer text
fragments consisting of multiple sentences. For short phrases or single words,
however, trigrams are not enough. The shorter the input text is, the less
n-grams are available. The probabilities estimated from such few n-grams are not
reliable. This is whyLinguamakes use of n-grams of sizes 1 up to 5 which
results in much more accurate prediction of the correct language.A second important difference is thatLinguadoes not only use such a
statistical model, but also a rule-based engine. This engine first determines
the alphabet of the input text and searches for characters which are unique
in one or more languages. If exactly one language can be reliably chosen this
way, the statistical model is not necessary anymore. In any case, the
rule-based engine filters out languages that do not satisfy the conditions of
the input text. Only then, in a second step, the probabilistic n-gram model is
taken into consideration. This makes sense because loading less language models
means less memory consumption and better runtime performance.In general, it is always a good idea to restrict the set of languages to be
considered in the classification process using the respective api methods.
If you know beforehand that certain languages are never to occur in an input
text, do not let those take part in the classifcation process. The filtering
mechanism of the rule-based engine is quite good, however, filtering based on
your own knowledge of the input text is always preferable.8. Test report generationIf you want to reproduce the accuracy results above, you can generate the test
reports yourself for all classifiers and languages by installingPoetryand executing:poetry install --no-root --only script
poetry run python3 scripts/accuracy_reporter.pyFor each detector and language, a test report file is then written into/accuracy-reports.
As an example, here is the current output of theLinguaGerman report:##### German #####

>>> Accuracy on average: 89.27%

>> Detection of 1000 single words (average length: 9 chars)
Accuracy: 74.20%
Erroneously classified as Dutch: 2.30%, Danish: 2.20%, English: 2.20%, Latin: 1.80%, Bokmal: 1.60%, Italian: 1.30%, Basque: 1.20%, Esperanto: 1.20%, French: 1.20%, Swedish: 0.90%, Afrikaans: 0.70%, Finnish: 0.60%, Nynorsk: 0.60%, Portuguese: 0.60%, Yoruba: 0.60%, Sotho: 0.50%, Tsonga: 0.50%, Welsh: 0.50%, Estonian: 0.40%, Irish: 0.40%, Polish: 0.40%, Spanish: 0.40%, Tswana: 0.40%, Albanian: 0.30%, Icelandic: 0.30%, Tagalog: 0.30%, Bosnian: 0.20%, Catalan: 0.20%, Croatian: 0.20%, Indonesian: 0.20%, Lithuanian: 0.20%, Romanian: 0.20%, Swahili: 0.20%, Zulu: 0.20%, Latvian: 0.10%, Malay: 0.10%, Maori: 0.10%, Slovak: 0.10%, Slovene: 0.10%, Somali: 0.10%, Turkish: 0.10%, Xhosa: 0.10%

>> Detection of 1000 word pairs (average length: 18 chars)
Accuracy: 93.90%
Erroneously classified as Dutch: 0.90%, Latin: 0.90%, English: 0.70%, Swedish: 0.60%, Danish: 0.50%, French: 0.40%, Bokmal: 0.30%, Irish: 0.20%, Tagalog: 0.20%, Tsonga: 0.20%, Afrikaans: 0.10%, Esperanto: 0.10%, Estonian: 0.10%, Finnish: 0.10%, Italian: 0.10%, Maori: 0.10%, Nynorsk: 0.10%, Somali: 0.10%, Swahili: 0.10%, Turkish: 0.10%, Welsh: 0.10%, Zulu: 0.10%

>> Detection of 1000 sentences (average length: 111 chars)
Accuracy: 99.70%
Erroneously classified as Dutch: 0.20%, Latin: 0.10%9. How to add it to your project?Linguais available in thePython Package Indexand can be installed with:pip install lingua-language-detector10. How to build?Linguarequires Python >= 3.8.
First create a virtualenv and install the Python wheel for your platform withpip.git clone https://github.com/pemistahl/lingua-py.git
cd lingua-py
python3 -m venv .venv
source .venv/bin/activate
pip install --find-links=lingua lingua-language-detectorIn the scripts directory, there are Python scripts for writing accuracy reports,
drawing plots and writing accuracy values in an HTML table. The dependencies
for these scripts are managed byPoetrywhich
you need to install if you have not done so yet. In order to install the script
dependencies in your virtualenv, runpoetry install --no-root --only scriptThe project makes uses of type annotations which allow for static type checking withMypy. Run the following commands for checking the types:poetry install --no-root --only dev
poetry run mypyThe Python source code is formatted withBlack:poetry run black .11. How to use?11.1 Basic usage>>>fromlinguaimportLanguage,LanguageDetectorBuilder>>>languages=[Language.ENGLISH,Language.FRENCH,Language.GERMAN,Language.SPANISH]>>>detector=LanguageDetectorBuilder.from_languages(*languages).build()>>>language=detector.detect_language_of("languages are awesome")>>>languageLanguage.ENGLISH>>>language.iso_code_639_1IsoCode639_1.EN>>>language.iso_code_639_1.name'EN'>>>language.iso_code_639_3IsoCode639_3.ENG>>>language.iso_code_639_3.name'ENG'11.2 Minimum relative distanceBy default,Linguareturns the most likely language for a given input text.
However, there are certain words that are spelled the same in more than one
language. The wordprologue, for instance, is both a valid English and French
word.Linguawould output either English or French which might be wrong in
the given context. For cases like that, it is possible to specify a minimum
relative distance that the logarithmized and summed up probabilities for
each possible language have to satisfy. It can be stated in the following way:>>>fromlinguaimportLanguage,LanguageDetectorBuilder>>>languages=[Language.ENGLISH,Language.FRENCH,Language.GERMAN,Language.SPANISH]>>>detector=LanguageDetectorBuilder.from_languages(*languages)\.with_minimum_relative_distance(0.9)\.build()>>>print(detector.detect_language_of("languages are awesome"))NoneBe aware that the distance between the language probabilities is dependent on
the length of the input text. The longer the input text, the larger the
distance between the languages. So if you want to classify very short text
phrases, do not set the minimum relative distance too high. Otherwise,Nonewill be returned most of the time as in the example above. This is the return
value for cases where language detection is not reliably possible.11.3 Confidence valuesKnowing about the most likely language is nice but how reliable is the computed
likelihood? And how less likely are the other examined languages in comparison
to the most likely one? These questions can be answered as well:>>>fromlinguaimportLanguage,LanguageDetectorBuilder>>>languages=[Language.ENGLISH,Language.FRENCH,Language.GERMAN,Language.SPANISH]>>>detector=LanguageDetectorBuilder.from_languages(*languages).build()>>>confidence_values=detector.compute_language_confidence_values("languages are awesome")>>>forconfidenceinconfidence_values:...print(f"{confidence.language.name}:{confidence.value:.2f}")ENGLISH:0.93FRENCH:0.04GERMAN:0.02SPANISH:0.01In the example above, a list is returned containing those languages which the
calling instance of LanguageDetector has been built from, sorted by
their confidence value in descending order. Each value is a probability between
0.0 and 1.0. The probabilities of all languages will sum to 1.0.
If the language is unambiguously identified by the rule engine, the value 1.0
will always be returned for this language. The other languages will receive a
value of 0.0.There is also a method for returning the confidence value for one specific
language only:>>>fromlinguaimportLanguage,LanguageDetectorBuilder>>>languages=[Language.ENGLISH,Language.FRENCH,Language.GERMAN,Language.SPANISH]>>>detector=LanguageDetectorBuilder.from_languages(*languages).build()>>>confidence_value=detector.compute_language_confidence("languages are awesome",Language.FRENCH)>>>print(f"{confidence_value:.2f}")0.04The value that this method computes is a number between 0.0 and 1.0. If the
language is unambiguously identified by the rule engine, the value 1.0 will
always be returned. If the given language is not supported by this detector
instance, the value 0.0 will always be returned.11.4 Eager loading versus lazy loadingBy default,Linguauses lazy-loading to load only those language models on
demand which are considered relevant by the rule-based filter engine. For web
services, for instance, it is rather beneficial to preload all language models
into memory to avoid unexpected latency while waiting for the service response.
If you want to enable the eager-loading mode, you can do it like this:LanguageDetectorBuilder.from_all_languages().with_preloaded_language_models().build()Multiple instances ofLanguageDetectorshare the same language models in
memory which are accessed asynchronously by the instances.11.5 Low accuracy mode versus high accuracy modeLingua'shigh detection accuracy comes at the cost of being noticeably slower
than other language detectors. The large language models also consume significant
amounts of memory. These requirements might not be feasible for systems running low
on resources. If you want to classify mostly long texts or need to save resources,
you can enable alow accuracy modethat loads only a small subset of the language
models into memory:LanguageDetectorBuilder.from_all_languages().with_low_accuracy_mode().build()The downside of this approach is that detection accuracy for short texts consisting
of less than 120 characters will drop significantly. However, detection accuracy for
texts which are longer than 120 characters will remain mostly unaffected.In high accuracy mode (the default), the language detector consumes approximately
1 GB of memory if all language models are loaded. In low accuracy mode, memory
consumption is reduced to approximately 103 MB.An alternative for a smaller memory footprint and faster performance is to reduce the set
of languages when building the language detector. In most cases, it is not advisable to
build the detector from all supported languages. When you have knowledge about
the texts you want to classify you can almost always rule out certain languages as impossible
or unlikely to occur.11.6 Detection of multiple languages in mixed-language textsIn contrast to most other language detectors,Linguais able to detect multiple languages
in mixed-language texts. This feature can yield quite reasonable results but it is still
in an experimental state and therefore the detection result is highly dependent on the input
text. It works best in high-accuracy mode with multiple long words for each language.
The shorter the phrases and their words are, the less accurate are the results. Reducing the
set of languages when building the language detector can also improve accuracy for this task
if the languages occurring in the text are equal to the languages supported by the respective
language detector instance.>>>fromlinguaimportLanguage,LanguageDetectorBuilder>>>languages=[Language.ENGLISH,Language.FRENCH,Language.GERMAN]>>>detector=LanguageDetectorBuilder.from_languages(*languages).build()>>>sentence="Parlez-vous français? "+\..."Ich spreche Französisch nur ein bisschen. "+\..."A little bit is better than nothing.">>>forresultindetector.detect_multiple_languages_of(sentence):...print(f"{result.language.name}: '{sentence[result.start_index:result.end_index]}'")FRENCH:'Parlez-vous français? 'GERMAN:'Ich spreche Französisch nur ein bisschen. 'ENGLISH:'A little bit is better than nothing.'In the example above, a list ofDetectionResultis returned. Each entry in the list describes a contiguous single-language text section,
providing start and end indices of the respective substring.11.7 Single-threaded versus multi-threaded language detectionTheLanguageDetectormethods explained above all operate in a single thread.
If you want to classify a very large set of texts, you will probably want to
use all available CPU cores efficiently in multiple threads for maximum performance.Every single-threaded method has a multi-threaded equivalent that accepts a list of texts
and returns a list of results.Single-threadedMulti-threadeddetect_language_ofdetect_languages_in_parallel_ofdetect_multiple_languages_ofdetect_multiple_languages_in_parallel_ofcompute_language_confidence_valuescompute_language_confidence_values_in_parallelcompute_language_confidencecompute_language_confidence_in_parallel11.8 Methods to build the LanguageDetectorThere might be classification tasks where you know beforehand that your
language data is definitely not written in Latin, for instance. The detection
accuracy can become better in such cases if you exclude certain languages from
the decision process or just explicitly include relevant languages:fromlinguaimportLanguageDetectorBuilder,Language,IsoCode639_1,IsoCode639_3# Include all languages available in the library.LanguageDetectorBuilder.from_all_languages()# Include only languages that are not yet extinct (= currently excludes Latin).LanguageDetectorBuilder.from_all_spoken_languages()# Include only languages written with Cyrillic script.LanguageDetectorBuilder.from_all_languages_with_cyrillic_script()# Exclude only the Spanish language from the decision algorithm.LanguageDetectorBuilder.from_all_languages_without(Language.SPANISH)# Only decide between English and German.LanguageDetectorBuilder.from_languages(Language.ENGLISH,Language.GERMAN)# Select languages by ISO 639-1 code.LanguageDetectorBuilder.from_iso_codes_639_1(IsoCode639_1.EN,IsoCode639_1.DE)# Select languages by ISO 639-3 code.LanguageDetectorBuilder.from_iso_codes_639_3(IsoCode639_3.ENG,IsoCode639_3.DEU)12. What's next for version 2.1.0?Take a look at theplanned issues.13. ContributionsAny contributions toLinguaare very much appreciated. Please read the instructions
inCONTRIBUTING.mdin the repository of the Rust implementation for how to add new languages to the library.