B angla P araphrase: A High-Quality B angla Paraphrase Dataset (2022)

Abstract: In this work, we present BanglaParaphrase, a high-quality synthetic Bangla Paraphrase dataset curated by a novel filtering pipeline. We aim to take a step towards alleviating the low resource status of the Bangla language in the NLP domain through the introduction of BanglaParaphrase, which ensures quality by preserving both semantics and diversity, making it particularly useful to enhance other Bangla datasets. We show a detailed comparative analysis between our dataset and models trained on it with other existing works to establish the viability of our synthetic paraphrase data generation pipeline. We are making the dataset and models publicly available at https://github.com/csebuetnlp/banglaparaphrase to further the state of Bangla NLP.

## introduction bangla, despite being the seventh most spoken language by the total number of speakers 1 and fifth most spoken language by native speakers 2 is still considered a low resource language in terms of language processing. @xcite have classified bangla in the language group that has substantial lackings of efforts for labeled data collection and preparation. this lacking is rampant in terms of high-quality datasets for various natural language tasks, including paraphrase generation. paraphrases can be roughly defined as pairs of texts that have similar meanings but may differ structurally. so the task of generating paraphrases given a sentence is to generate sentences with different wordings or/and structures to the original sentences while preserving the meaning. paraphrasing can be a vital tool to assist language understanding tasks such as question answering @xcite , style transfer @xcite , semantic parsing @xcite , and data augmentation tasks @xcite . paraphrase generation has been a challenging problem in the natural language processing domain as it has several contrasting elements, such as semantics and structures, that must be ensured to obtain a good paraphrase of a sentence. syntactically bangla has a different structure than high-resource languages like english and french. the principal word order of the bangla language is subject-objectverb (sov). still, it also allows free word ordering during sentence formation. the pronoun usage in the bangla language has various forms, such as "very familiar", "familiar", and "polite forms" foot_0 . it is imperative to maintain the coherence of these forms throughout a sentence as well as across the paraphrases in a bangla paraphrase dataset. following that thread, we create a bangla paraphrase dataset ensuring good quality in terms of semantics and diversity. since generating datasets by manual intervention is time-consuming, we curate our banglaparaphrase dataset through a pivoting @xcite approach, with additional filtering stages to ensure diversity and semantics. we further study the effects of dataset augmentation on a synthetic dataset using masked language modeling. finally, we demonstrate the quality of our dataset by training baseline models and through comparative analysis with other bangla paraphrase datasets and models. in summary: ## related work paraphrase generation datasets and models are heavily dominated by high-resource languages such as english. but for low-resource languages such as bangla, this domain is less explored. to our knowledge, only @xcite described the use of indicbart @xcite to generate paraphrases using the sequence-to-sequence approach for the bangla language. one of the most challenging barriers to paraphrasing research for low-resource languages is the shortage of goodquality datasets. among recent work on lowresource paraphrase datasets, @xcite introduced a comprehensive dataset for the finnish language. the opusparcus dataset @xcite consists of paraphrases for six european languages. for indic languages such as tamil, hindi, punjabi, and malayalam, anand @xcite introduced a paraphrase detection dataset in a shared task. scherrer (2020) introduced a paraphrase dataset for 73 languages, where there are only about 1400 sentences in total for the bangla language, mainly consisting of simple sentences. ## conclusion & future works in this work, starting from a pure synthetic paraphrase dataset, we introduced an automated filtering pipeline to curate a high-quality bangla paraphrase dataset, ensuring both diversity and semantics. we trained the mt5-small and banglat5 models with our dataset to generate quality paraphrases of bangla sentences. our choice of the initial monolingual corpus has been made to include highly representative sentences for the bangla language, which is large enough for an isolated paraphrase generation task. the corpus can easily be extended for desired pretraining tasks using a larger monolingual corpus. furthermore, we plan on improving the mlm scheme by automating parts of speech selection and using labse with banglabert embeddings to compare semantics at the sentence level, which would ensure better filters and better evaluation of generated paraphrases. though our work is language-agnostic, the extent to which our approach applies to other low-resource languages given language-specific components (datasets and models) is subject to further experimentation. in future work, we want to investigate the viability of our synthetic data generation pipeline in the context of paraphrase datasets in different languages included in popular benchmarks such as @xcite . additionally, we want to investigate how our paraphrase dataset and models can be used to improve the performance of other lowresource tasks in bangla, such as readability detection @xcite and cross-lingual summarization @xcite 1 where n is defined as the maximum n-gram we considered, and ngram s and ngram c are the lists of n-grams present in the source and candidate sentences. in all experiments, we use @xmath0. this score can be treated as the inverse of the bleu score since it minimizes the number of n-gram overlaps between the two sentences. we also present a pinc score vs. data amount plot in figure 1 , which we used to select the thresholds. bertscore plot a plot of bertscore with banglabert embeddings after the banglaparaphrase dataset has been filtered with a pinc score of 0.76 threshold is shown in figure 2.