Package: ai4bharat-transliteration
AI4Bharat Indic-TransliterationAn AI-based transliteration engine for 21 major languages of the Indian subcontinent.This package provides support for:Python Library for transliteration from Roman to Native scriptHTTP API server that can be hosted for interaction with web applicationsAboutThis library is based on ourresearch workcalledIndic-Xlitto build tools that can translit text between Indic languages and colloquially-typed content (in English alphabet). We support both Roman-to-Native back-transliteration (English script to Indic language conversion), as well as Native-to-Roman transliteration (Indic to English alphabet conversion).An online demo is available here:https://xlit.ai4bharat.orgLanguages SupportedISO 639 codeLanguageasAssamese - অসমীয়াbnBangla - বাংলাbrxBoro - बड़ोguGujarati - ગુજરાતીhiHindi - हिंदीknKannada - ಕನ್ನಡksKashmiri - كٲشُرgomKonkani Goan - कोंकणीmaiMaithili - मैथिलीmlMalayalam - മലയാളംmniManipuri - ꯃꯤꯇꯩꯂꯣꯟmrMarathi - मराठीneNepali - नेपालीorOriya - ଓଡ଼ିଆpaPanjabi - ਪੰਜਾਬੀsaSanskrit - संस्कृतम्sdSindhi - سنڌيsiSinhala - සිංහලtaTamil - தமிழ்teTelugu - తెలుగుurUrdu - اُردُوUsagePython LibraryImport the wrapper for transliteration engine by:fromai4bharat.transliterationimportXlitEngineExample 1: Using word Transliteratione=XlitEngine("hi",beam_width=10,rescore=True)out=e.translit_word("namasthe",topk=5)print(out)# output: {'hi': ['नमस्ते', 'नमस्थे', 'नामस्थे', 'नमास्थे', 'नमस्थें']}Arguments:beam_widthincreases search size, resulting in improved accuracy but increases time/compute. (Default:4)topkreturns only specified number of top results. (Default:4)rescorereturns the reranked suggestions after using a dictionary. (Default:True)Romanization:By default,XlitEnginewill load English-to-Indic model (default:src_script_type="roman")To load Indic-to-English model, usesrc_script_type="indic"For example: (also applicable for all other examples below)e=XlitEngine(src_script_type="indic",beam_width=10,rescore=False)out=e.translit_word("नमस्ते",lang_code="hi",topk=5)print(out)# output: ['namaste', 'namastey', 'namasthe', 'namastay', 'namste']Example 2: word Transliteration without rescoringe=XlitEngine("hi",beam_width=10,rescore=False)out=e.translit_word("namasthe",topk=5)print(out)# output: {'hi': ['नमस्थे', 'नामस्थे', 'नमास्थे', 'नमस्थें', 'नमस्ते']}Example 3: Using Sentence Transliteratione=XlitEngine("ta",beam_width=10)out=e.translit_sentence("vanakkam ulagam")print(out)# output: {'ta': 'வணக்கம் உலகம்'}Note:Only single top most prediction is returned for each word in sentence.Example 4: Using Multiple language Transliteratione=XlitEngine(["ta","ml"],beam_width=6)# leave empty or use "all" to load all available languages# e = XlitEngine("all)out=e.translit_word("amma",topk=3)print(out)# output: {'ml': ['അമ്മ', 'എമ്മ', 'അമ'], 'ta': ['அம்மா', 'அம்ம', 'அம்மை']}out=e.translit_sentence("vandhe maatharam")print(out)# output: {'ml': 'വന്ധേ മാതരം', 'ta': 'வந்தே மாதரம்'}## Specify language name to get only specific language resultout=e.translit_word("amma",lang_code="ml",topk=5)print(out)# output: ['അമ്മ', 'എമ്മ', 'അമ', 'എഎമ്മ', 'അഎമ്മ']Example 5: Transliteration for all available languagese=XlitEngine(beam_width=10)out=e.translit_sentence("namaskaar bharat")print(out)# sample output: {'bn': 'নমস্কার ভারত', 'gu': 'નમસ્કાર ભારત', 'hi': 'नमस्कार भारत', 'kn': 'ನಮಸ್ಕಾರ್ ಭಾರತ್', 'ml': 'നമസ്കാർ ഭാരത്', 'pa': 'ਨਮਸਕਾਰ ਭਾਰਤ', 'si': 'නමස්කාර් භාරත්', 'ta': 'நமஸ்கார் பாரத்', 'te': 'నమస్కార్ భారత్', 'ur': 'نمسکار بھارت'}Web API ServerRunning a flask server using a 3-line script:fromai4bharat.transliterationimportxlit_serverapp,engine=xlit_server.get_app()app.run(host='0.0.0.0',port=8000)Then on browser (or) curl, use link ashttp://{IP-address}:{port}/tl/{lang-id}/{word_in_eng_script}Example:http://localhost:8000/tl/ta/ammahttp://localhost:8000/languagesDebugging errorsIf you face any of the following errors:ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 88 from C header, got 80 from PyObject
ValueError: Please build (or rebuild) Cython components withpython setup.py build_ext --inplace.Run:pip install --upgrade numpyRelease NotesThis package contains applications built around the Transliteration engine. The contents of this package can also be downloaded fromour GitHub repo.All the NN models of Indic-Xlit are released under MIT License.