C o P ara: The First D ravidian Paragraph-level n-way Aligned Corpus (2023)

Abstract: We present CoPara, the first publicly available paragraph-level (n-way aligned) multilingual parallel corpora for Dravidian languages. The collection contains 2856 paragraph/passage pairs between English and four Dravidian languages. We source the parallel paragraphs from the New India Samachar magazine and align them with English as a pivot language. We do human and artificial evaluations to validate the high-quality alignment and richness of the parallel paragraphs of a range of lengths. To show one of the many ways this dataset can be wielded, we finetuned IndicBART, a seq2seq NMT model on all XX-En pairs of languages in CoPara which perform better than existing sentence-level models on standard benchmarks (like BLEU) on sentence level translations and longer text too. We show how this dataset can enrich a model trained for a task like this, with more contextual cues and beyond sentence understanding even in low-resource settings like that of Dravidian languages. Finally, the dataset and models are made available publicly at CoPara to help advance research in Dravidian NLP, parallel multilingual, and beyond sentence-level tasks like NMT, etc.

## introduction public and quality multilingual data for indic languages, explorations in nlp for the same, and in general community interest has grown in the last few years which gave us large parallel multilingual sentence level datasets and models @xcite . more specifically there is also much needed and quality research coming out in dravidian nlp through community efforts and workshops like dravidianlangtech @xcite . however, while we improve and amass resources & techniques on dravidian sentence-level nlp, we should also utilize publicly available data to explore if there are structures beyond sentences that can be mined to inform these growing models of longer form texts @xcite . this information could lead the models one step further into solving straightforward problems like translating a paragraph (and not translating parts of it and stitching it back together) or more subtle ones like coreference resolution or author style feature distinction @xcite . this first paragraph/passage level multilingual n-way aligned (pla from here on) dataset in dravidian languages thus is a step towards opening up research for these languages on document level parallel corpus creation/nmt @xcite and be an early part of the rare but growing work in pla corpus creation & nlp @xcite @xcite . the pla work cited above and classic sentencelevel aligned works (sla, like europarl @xcite ) show how pla has directly bettered nmt, has been helpful in obtaining better aligned sentence level texts, linking entities across wikipedia like databases, better literary & medical text translation, etc. in the context of dravidian languages, early work like @xcite & more recently dravidianlangtech '21 @xcite concluded (across 2 works) that sentence length & complexity were crucial barriers to a good translation in this language family as well. with these motivations we present copara the first public dravidian languages' pla corpus (example at figure 1 ) which has a fair share of small & long passages, some quality checks, and show how it bettered dravidian nmt (both sentence and paragraph level) as an example of use. 2 literature review 2.1 the dravidian languages following is a brief introduction to the 4 dravidian languages and their morpho-syntactic features of interest to motivate beyond sentence need of context in a corpus, summarised from gutman and avanzati (2013)'s compilation (find a more detailed sociolinguistic history at madasamy et al. (2022)). some languages in the dravidian language family (mainly south india and asia), lack a written form, but the following 4 prominent ones have developed a large body of written literature. these agglutinative languages are mostly head-final with a flexible subject-object-verb (sov) word order, requiring the finite verb at the sentence's end. each sentence permits one finite verb accompanied by one or more non-finite verbs. subordinate clauses in these languages typically precede the main clause. kannada is a pro-drop language that syntactically depends on case suffixes, postpositions, participles, gerunds, and infinitives. since this makes the verb rich to express person and number, it allows subject omission. tamil has its subject is usually in the nominative case. subject-verb agreement exists, and verbs agree in person, number, and gender. it employs postpositions and case inflection to indicate syntactical relations. telugu's syntactic functions are conveyed through case suffixes and postpositions following the oblique stem. it is a pro-drop language too but lacks coordinating conjunctions, and coordinated phrases lengthen their final vowels. relative clauses are formed using a relative participle instead of a finite verb. finally, malayalam closely resembles tamil, but has diverged since the 8th century. it is agglutinative like dravidian languages but has lost subject-verb and person-number agreement. ## the copara dataset the dataset is based on the "new india samachar" magazine (pib, 2020), a publication launched by the information and broadcasting (i&b) ministry of india in 2020 and published on a fortnightly basis in english and twelve different indic languages. the magazine disseminates information on cabinet decisions, features content like 'mann ki baat', and discussions on prevailing issues. we processed (pipeline at figure 2 ) 15 of these issues published in 2022 (subset picked randomly) to create copara. thus we had a total of 75 issues as we aimed to align each of the 15 issues in an n-way manner across 5 languages: english and 4 dravidian languages @xcite . this resulted in 2856 n-way aligned paragraphs with statistics as highlighted by the following sections detail the steps of the data processing and show a detailed analysis of alignment quality too. ## neural machine translation by fine-tuning on copara we now show how copara can be used to improve nmt in dravidian languages. for the same, we take a multilingual indic language model finetuned on the nmt task. we further finetune it on our dataset and check if the performance increases as compared to base model's results. ## conclusion we establish copara as the first parallel paragraphlevel dravidian n-way corpus by showing how it was formed and its quality by doing various evaluations. we then show that it significantly increases a sentence-level nmt model's performance on not just sentence level but also on paragraph-level data. we show that it does not hamper the base model's performance on sentence-level nmt, while enhancing it for processing paragraphs. these analyses are done on a different benchmark and on two different metrics as well. we hope that this opens up avenues for long text and document-level nlp in dravidian languages and that copara is grown both in quantity and quality by succeeding works.