Itihasa: A large-scale corpus for S anskrit to E nglish translation (2021)

Abstract: This work introduces Itihasa, a large-scale translation dataset containing 93,000 pairs of Sanskrit shlokas and their English translations. The shlokas are extracted from two Indian epics viz., The Ramayana and The Mahabharata. We first describe the motivation behind the curation of such a dataset and follow up with empirical analysis to bring out its nuances. We then benchmark the performance of standard translation models on this corpus and show that even state-of-the-art transformer architectures perform poorly, emphasizing the complexity of the dataset.

## introduction sanskrit is one of the oldest languages in the world and most indo-european languages are influenced by it @xcite . there are about 30 million pieces of sanskrit literature available to us today @xcite , most of which have not been digitized. among those that have been, few have been translated. the main reason for this is the lack of expertise and funding. an automatic translation system would not only aid and accelerate this process, but it would also help in democratizing the knowledge, history, and culture present in this literature. in this work, we present itihāsa, a large-scale sanskrit-english translation corpus consisting of more than 93,000 shlokas and their translations. itihāsa, literally meaning 'it happened this way' is a collection of historical records of important events in indian history. these bodies of work are mostly composed in the form of verses or shlokas, a poetic form which usually consists of four parts containing eight syllables each (fig. 1 ). the most important among these works are the rāmāyana and the mahābhārata. the rāmāyana, which describes the events in the life of lord rāma, consists of 24,000 verses. the mahābhārata details the war between cousins of the kuru dynasty, in 100,000 verses. the mahābhārata is the longest poem ever written with about 1.8 million words in total and is roughly ten times the length of the iliad and the odyssey combined. only two authors have attempted to translate the unabridged versions of both the rāmāyana and the mahābhārata to english: manmatha nāth dutt in the 1890s and bibek debroy in the 2010s. m. n. dutt was a prolific translator whose works are now in the public domain. these works are published in a shloka-wise format as shown in fig. 1 which makes it easy to automatically align shlokas with their translations. though many of m. n. dutt's works are freely available, we choose to extract data from the rāmāyana (vālmiki and dutt, 1891), and the mahābhārata @xcite , mainly due to its size and popularity. as per our knowledge, this is the biggest sanskrit-english translation dataset to be released in the public domain. we also train and evaluate standard translation systems on this dataset. in both translation directions, we use moses as an smt baseline, and transformer-based seq2seq models as nmt baselines (see §4). we find that models which are generally on-par with human performance on other translation tasks, perform poorly on itihāsa, with the best models scoring between 7-8 bleu points. this indicates the complex nature of the dataset (see §3 for a detailed analysis of the dataset and its vocabulary). ## data preparation the translated works of the rāmāyana and the mahābhārata were published in four and nine volumes respectively. 3 all volumes have a standard two-column format as shown in fig. 2 . each page has a header with the chapter name and page number separated from the main text by a horizontal line. the two columns of text are separated by a vertical line. the process of data preparation can be divided into (i) automatic ocr extraction, and (ii) manual inspection for alignment errors. ## analysis in total, we extract 19,371 translation pairs from 642 chapters of the rāmāyana and 73,659 translation pairs from 2,110 chapters of the mahābhārata. it should be noted that these numbers do not correspond to the number of shlokas because, in the original volumes, shlokas are sometimes split and often combined to make the english translations flow better. we reserve 80% of the data from each text for training mt systems and use the rest for evaluation. from the evaluation set, 33% is used for development and 67% for testing. the absolute sizes of the split data are shown in tab. 1. due to sanskrit's agglutinative nature, the dataset is asymmetric in the sense that, the number of words required to convey the same information, is less in sanskrit when compared with english. the rāmāyana's english translations, on average, have 2.54 words for every word in its shloka. this value is even larger in the mahābhārata with 2.82 translated words per shloka word. this effect is clearly seen when we consider the vocabulary sizes and the percentage of common tokens between the texts. for this, we tokenize the data with two different tokenization schemes: wordlevel and byte-pair encoding @xcite . for word-level tokenization, the translations of the rāmāyana (the mahābhārata) have 16,820 (31,055) unique word tokens, and the shlokas have 66,072 (184,407) tokens. the english vocabularies have 11,579 common tokens which is 68.8% of the rāmāyana's and 37.3% of the mahābhārata's. but the overlap percentages drop significantly for the sanskrit vocabularies. in this case, we find 21,635 common tokens which amount to an overlap of 32.7% and 11.7% respectively. as shown in fig. 4 , this trend holds for bpe tokenization as well. ## experiments we train one smt and five nmt systems in both directions and report the (i) character n-gram f-score, (ii) token accuracy, (iii) bleu @xcite , and (iv) translation edit ratio @xcite scores in tab. 2. for smt, we use moses @xcite and for nmt, we use sequence-to-sequence (seq2seq) transformers @xcite . we train the seq2seq models from scratch by initializing the encoders and decoders with standard bert (b2b) architectures. these tiny, mini, small, medium, and base models have 2/128, 4/256, 4/512, 8/512, and 12/768 layers/dimensions respectively. see @xcite for more details. in our early experiments, we also tried initializing the encoders and decoders with weights from pre-trained indic language models like muril @xcite , but they showed poor performance and thus are not reported here. implementation details all models are trained using huggingface transformers @xcite . both source and target sequences are truncated at 128 tokens. we train wordpiece tokenizers on our dataset and use them for all models. adam optimizer (kingma and ba, 2014) with weightdecay of 0.01, and learning rate of 5 × 10 -5 is used. all models are trained for 100 epochs. the learning rate is warmed up over 8,000 steps and decayed later with a linear scheduler. we use a batch size of 128, and use standard cross-entropy loss with no label smoothing. we run into memory errors on bigger models (medium and base), but maintain the effective batch-size and optimization steps by introducing gradient accumulation and increasing the number of epochs, respectively. also, to reduce the total training time of bigger models, we stop training if the bleu score does not improve over 10 epochs. during generation, we use a beam size of 5 and compute all metrics against truncated references. ## related work early translation efforts from sanskrit to english were limited to the construction of dictionaries by western indologists @xcite . over the years, though notable translation works like ganguli (1883) have been published, the lack of digitization has been a bottleneck hindering any meaningful progress towards automatic translation systems. this has changed recently, at least for monolingual data, with the curation of digital libraries like gretil foot_6 and dcs foot_7 . currently, the largest freely available repository of translations are for the bhagavadgita @xcite and the rāmāyana @xcite . however, labeled datasets for other tasks, like the ones proposed in @xcite @xcite have resulted in parsers @xcite and sandhi splitters @xcite which are pre-cursors to modular translation systems. though there have been attempts at building sanskrit translation tools @xcite , they are mostly rule-based and rely on manual intervention. we hope that the availability of the itihāsa corpus pushes the domain towards endto-end systems. ## conclusion in this work, we introduce itihāsa, a large-scale dataset containing more than 93,000 pairs of sanskrit shlokas and their english translations from the rāmāyana and the mahābhārata. first, we detail the extraction process which includes an automated ocr phase and a manual alignment phase. next, we analyze the dataset to give an intuition of its asymmetric nature and to showcase its complexities. lastly, we train state-of-the-art translation models which perform poorly, proving the necessity for more work in this area.