Varta: A Large-Scale Headline-Generation Dataset for I ndic Languages (2023)

Abstract: We present Varta, a large-scale multilingual dataset for headline generation in Indic languages. This dataset includes more than 41 million pairs of headlines and articles in 14 different Indic languages (and English), which come from a variety of high-quality news sources. To the best of our knowledge, this is the largest collection of curated news articles for Indic languages currently available. We use the collected data in a series of experiments to answer important questions related to Indic NLP and multilinguality research in general. We show that the dataset is challenging even for state-of-the-art abstractive models and that they perform only slightly better than extractive baselines. Owing to its size, we also show that the dataset can be used to pre-train strong language models that outperform competitive baselines in both NLU and NLG benchmarks.

## introduction headline generation is a special case of abstractive summarization where the goal is to create a brief, often single-sentence 'summary' of a news article. unlike traditional summaries, which are a few sentences long and concisely convey the most important information from an article, a headline typically highlights one key fact from the article that is considered to be the most significant. in recent years, headlines have also been written to be catchy and increase click-through rates. this trend has made the task harder, as the lexical overlap between headlines and articles is low, while their compression ratio is high. there are several datasets available to train and evaluate headline generation and abstractive summarization models, but they are largely limited to english @xcite 2016; @xcite . although there have been efforts to create multilingual datasets such as mlsum @xcite and xlsum @xcite , the representation of indic languages in these datasets is still minimal. 1 as a result, it is challenging to create good headline generation or summarization systems for these languages, which are spoken by approximately one in five people worldwide. motivation indic headline generation is particularly interesting because this family of languages presents unique challenges. for example, though most indic languages are closely related, they use different scripts which makes transfer learning harder. many of the languages are morphologically rich, making headlines compact and thus harder to predict. with few exceptions, most indic languages are low-resource and are underrepresented in multilingual models, which makes few-shot learning ineffective. recent efforts have shown that language family-specific pretraining enables better transfer among languages @xcite and that quality data is required in large quantities for good downstream performance on text generation tasks like summarization @xcite . thus, there is a strong need for a multilin-gual, large-scale, high-quality dataset for indic languages. to fill this gap, we introduce vārta, a dataset consisting of more than 41 million articleheadline pairs in 15 languages (14 indic languages + english). the data is crawled from dailyhunt, a popular news aggregator in india that pulls highquality articles from multiple trusted and reputed news publishers. it also covers a wide range of topics such as politics, science, entertainment, sports, business, etc., which makes the dataset diverse both in terms of domains and writing styles. ## data vārta contains 41.8 million high-quality news articles in 14 indic languages and english. with 34.5 million non-english article-headline pairs, it is the largest headline-generation dataset of its kind. in this section, we detail how the dataset is collected along with the various processing steps involved, before moving on to some interesting statistics and analysis of the data. ## experiments rather than trying to squeeze optimal performance from models, the aim of our experiments is to perform a detailed study of how different training settings affect model behavior. in this section, we first present a list of research questions that our experiments try to answer. we then provide a brief overview of the models used in the experiments. lastly, we describe the different training and evaluation configurations of our experiments before presenting our findings in §5. ## conclusion in this work, we create vārta, a large-scale headline-generation dataset for indic languages. we show that this dataset is challenging even for state-of-the-art text generation models. utilizing the size and quality of the dataset, we answer pertinent research questions about multilingual models, from an indic text generation perspective. we also show that vārta can be used as a pretraining corpus to train strong nlu and nlg models.