Package: anuvaad
AnuvaadState of the art open-source translation models for Indic languages.Installation# CPU pytorch will be installed if torch is not installedpipinstall--upgradeanuvaadUsageAs a Python modulefromanuvaadimportAnuvaadanu=Anuvaad("english-telugu")# Single sentence translation# beam_size is optional and defaults to 4anu.anuvaad("YS Jagan is the chief minister of Andhra Pradesh.")# "వైఎస్ జగన్ ఆంధ్రప్రదేశ్ ముఖ్యమంత్రి."# Batch translationanu.anuvaad(["YS Jagan is the chief minister of Andhra Pradesh.","Nara Lokesh suffered a humiliating defeat in Mangalagiri."])# ['వైఎస్ జగన్ ఆంధ్రప్రదేశ్ ముఖ్యమంత్రి.', 'మంగళగిరిలో నారా లోకేష్కు అవమానకరమైన ఓటమి ఎదురైంది.']As a service# Starting the api servicedockerrun-it-eBATCH_SIZE=1-p8080:8080notaitech/anuvaad:english-telugu# Running a predictioncurl-d'{"data": ["YS Jagan is the chief minister of Andhra Pradesh."]}'-H"Content-Type: application/json"-XPOSThttp://localhost:8080/syncAvailable ModelsAnuvaad BLEUGoogle BLEUenglish-telugu12.7211737437640096.841437460383768english-tamil12.7370361492146945.558450942590664english-malayalam17.78574664672199619.569069412553812english-kannada7.8888860419338153.2803251953567893english-marathi23.0275595539251812.888112016722792english-hindi29.17589221321695418.130893478614375english-bengalienglish-punjabienglish-gujaratiGoogle BLEU is calculated from translations generated by theGOOGLETRANSLATE() functionon google sheets.The testing scripts and data from Tatoeba is present athttps://github.com/notAI-tech/Anuvaad-testing-scriptshttps://docs.google.com/spreadsheets/d/1tYYZObELj-k6mJCnM6uf7xg3JChbSkjs8YvZsOgHacQ/edit?usp=sharingis the sheet containing the predictions from Anuvaad and GOOGLETRANSLATE function on the Tatoeba data from which the above scores are calculated.My thoughts on the evaluation/accuracy of the model(s):Unlike classification/ sequence labelling tasks, for open-domain translation or summarization systems it is very hard to quantify the accuracy through numbers.This is because, most accuracy measurements actually measure the overlap of character/word n-grams between the expected output and predicted output.These scores definitely help when evaluating/comparing multiple models on a particular dataset, but the number don't translate well for open-domain models.For example, Anuvaad translates the sentenceAn advance is placed with the Medical Superintendents of such hospitals who then provide assistance on a case to case basis.(taken fromhttp://data.statmt.org/pmindia/v1/parallelcorpus) toऐसे अस्पतालों के चिकित्सा अधीक्षकों के साथ एडवांस रखा जाता है, जिसके बाद मामले के आधार पर सहायता प्रदान की जाती है।where as the expected translation of the sentence from the dataset isअग्रिम धन राशि इन अस्पतालों को चिकित्सा निरीक्षकों को दी जाएगी, जो हर मामले को देखते हुए सहायता प्रदान करेंगे।.In the above example, Although Anuvaad's translation is correct (in the sense that translation conveys the same thing as the original sentence), the BLEU score with n=3 will be 0.Similarly, a model trained on the pmindia dataset will have bad score on a different dataset which uses a different style of writing, even if the translation is semantically correct.Our aim in building Anuvaad is to build a general purpose, open-domain translation module that can flexibly translate text from various domains.https://docs.google.com/spreadsheets/d/1_TTtBEvVgemQfGbRBSZYkECMMt5r7L9-dt0FGVUbmOY/edit?usp=sharingis a sheet comparing translations from Anuvaad, ilmulti (https://github.com/jerinphilip/ilmulti) and Google Translate (=GOOGLETRANSLATE(text, "en", "language") function on google sheets) on 100 randomly selected English sentences from Tatoeba.