LANS : Large-scale A rabic News Summarization Corpus (2023)

Abstract: Text summarization has been intensively studied in many languages, and some languages have reached advanced stages. Yet, Arabic Text Summarization (ATS) is still in its developing stages. Existing ATS datasets are either small or lack diversity. We build, LANS, a large-scale and diverse dataset for Arabic Text Summarization task. LANS offers 8.4 million articles and their summaries extracted from newspapers websites’ metadata between 1999 and 2019. The high-quality and diverse summaries are written by journalists from 22 major Arab newspapers and include an eclectic mix of at least more than 7 topics from each source. We conduct an intrinsic evaluation on LANS by both automatic and human evaluations. Human evaluation of 1,000 random samples reports 95.4% accuracy for our collected summaries, and automatic evaluation quantifies the diversity and abstractness of the summaries.

## introduction every day there is an abundant amount of text published on the internet, such as news articles, scientific papers, product reviews, and blogs. therefore, the need for text summarization is compelling to make use of this information overload. for a summarized text, a good one should be concise and include the main information of the original text @xcite . for some languages like english, the field has developed rapidly and achieved competitive results @xcite @xcite . unlike english, the field in arabic has been slowly and fairly developing in the past few years; thus, it has not reached its advanced shape. in the field of arabic text summarization (ats) @xcite @xcite @xcite , the dearth of a diverse and large summarization dataset is one of the main existing difficulties that ats researchers encounter @xcite . concerted efforts have been made to overcome those challenges by building various arabic datasets for the task such that easc @xcite , kalimat(el-haj and koulali, 2013), tac2011 (el-ghannam and el-shishtawy, 2014), ant @xcite , and xl-sum @xcite , but those datasets have limitations in terms of diversity or size. therefore, the demand for a diverse and large-scale dataset is crucial to advance the ats field. the diversity in the ats dataset is in twofold. the first kind of diversity exists in the modern standard arabic (msa). even though 22 countries use msa as an official standard language, each country has its own dialects (dialectal arabic) for communication. each country's dialects have some effects on the msa style of writing and the choice of words. for example, in a sentence describing the rounds of a soccer match, moroccan msa would use the word " " for "rounds" and " " for the word "the match" while saudi msa would use " " and " " respectively. second, there is diversity in news categories. each newspaper has different news topics, such as finance, politics, sports, health, local, international news, and more. not all ats datasets include both diversity aspects in one dataset (see in terms of size, the available ats datasets contain a range of 100 to 41,000 training samples, which make them too small to fully train a summarization model. the performance in summarization models evidently relies on a substantial amount of applicable training samples @xcite @xcite @xcite . thus, we expect a largescale dataset which is provided in this work. to overcome the current limitations in diversity and size, we introduce a new ats dataset (lans) that includes both types of diversity and large-scale to present new opportunities to ats models and improve their summary accuracies. to achieve msa diversity, that is the variety of each arab country dialects' impact on its msa, lans encompasses 19 arab countries and collected articles along with their summaries of 22 popular newspapers (see lans exploited the metadata of newspapers' archives to extract and build the dataset. in figure. 1, a high-level example is shown to demonstrate where the collected information originated from two parallel views: the webpage view and its html source code view. the webpage view shows what a reader sees when reading a news article: the url, title, bold part or the abstract sentence/s, and article bodies. lans pursues the metadata attributes from the html source code -specifically (og:description) to extract the summaries from. in the webpage view, the summaries lie either in bold text or before the article's paragraphs. in the html source code view, the summaries lie in the metadata attributes, in our case between (og:description) tags, which we extracted as the news articles' summaries. after the extraction, we cleaned and filtered 11m news articles to present 8.4m articles along with their summaries. to quantify the quality of the collected summaries and examine their summarization properties, we conducted an automatic evaluation based on 3 common metrics. moreover, we corroborated the evaluation with human evaluation of 1,000 samples to verify the accuracy of using the abstract from the html source code's metadata as a summary. the human evaluation verifies that using the summary available in the metadata has a 95.4% accuracy. considering the large size of lans, 8.4 million, lans can benefit the ats field, because large datasets improve nlp tasks, such as numerous training samples for pre-trained models @xcite . besides, both types of diversities create opportunities for researchers to construct more accurate ats models. our main contributions are as follows: (1) we curate lans, a large-scale ats dataset of 8.4 million arabic news articles paired with their summaries written by journalists between 1999 to 2019. to our knowledge, it is the largest to date. (2) lans is collected from 22 reputable arab newspapers to achieve high quality of diversity in msa, and for each source, there are at least 7 topics to achieve diversity in categories. (3) to quantify the intrinsic quality of lans, a human evaluation is conducted on 1,000 random samples and verifies 95.4% accuracy of the summaries. plus, the automatic evaluation on the whole dataset quantifies the abstractness and properties of the summaries. the essex arabic summaries corpus (easc) dataset. easc @xcite is an ats dataset, where each summary is extracted from the texts by mechanical turk. its text source is two arabic newspapers (alrai and alwatan) and the arabic language version of wikipedia. as a result, it contains 153 arabic articles and 765 summaries (5 summaries per article). in short, easc has highquality human-generated summaries but it is too small and lacks diversity. kalimat dataset. el-haj et al. worked on a dataset called kalimat @xcite . it has 20,291 extractive single-document and multidocument system summaries, and includes only 6 categories. it has been collected from only one source, which is alwatan newspaper from oman. the single-document summaries are generated based on their model gen-summ which inputs the article and its first sentence, then outputs the extractive summary. the multi-document summaries were generated for each 10, 100, and 500 articles 1 an english text summarization dataset of news paired with human summaries. https://duc.nist.gov/ in different categories. the generated summaries also lack human evaluation of the summaries. arabic news texts corpus (ant) and xl-sum. ant @xcite , and xl-sum @xcite are the most recent works. ant collected 31,798 documents paired with summaries using rss feeds from 5 arab news sources: alarabiya, bbc, cnn, france24, and skynews, while xl-sum collected 40,327 only from bbc. ant includes 6 categories, while xl-sum reported none. unlike ant, lans utilized the html source code og:description tag to collect the summaries which is similar to @xcite . ant is evaluated on several extractive summarization methods such as lexrank, textrank, luhn and lsa. xl-sum fine-tuned mt5 on their dataset and randomly sampled 500/500 development and test set respectively. besides, they conducted human evaluation on 250 random samples. when compared to our lans, our work collected nearly 8 million articles with summaries from 19 arab countries local newspapers. moreover, experts evaluated 1,000 random summaries from lans to substantiate the validity of the summaries. ## lans dataset this section details how lans is collected starting from the scraping process to building the dataset and how it is shaped for public use. ## lans description lans builds 8,443,484 articles and their summaries from 22 newspapers of 19 arab countries dated from 1999 to 2019. the high-level overall statistics in lans encompasses 19 arab countries for msa diversity. one of the overlooked aspects of diversity in arabic is the diversity of msa in the arab countries. it is true that all the newspapers in the arab countries use the same msa, but events, culture, and use of vocabulary are different from one country to another. therefore, it is necessary to collect such diverse data from each country. to achieve msa diversity in lans, our dataset encompasses 19 arab countries -except for the comoros islands, djibouti, and somalia because of the scarcity of data in their newspapers. further, lans provides a wide-ranging topic variety. the collected data from each country covers different categories, and some newspapers have more categories than others, which enhances the diversity of categories in lans. some newspapers have only a few categories (not less than 7), while some others have more than 9 categories including local news, international, political, financial, society, sports, technology, art, health, and religious news articles. this category diversity is one of the features of lans. it allows researchers to not only create subdatasets, but also create sub-subdataset of any of the subdatasets. for example, a subset can be all articles/summaries from saudi arabia. then, a sub-subdataset can be the local news categories from the subset of saudi arabia articles/summaries. this type of diversity can be created from lans. the dataset is chunked into separate xml files, each file is under 2 gb to make it easier to load and process. the total size of the whole dataset is 32gb. each country's dataset is a subset of the whole dataset, and researchers have the freedom to choose a subset or several subsets (by specific countries) to train and evaluate ats models. ## experiment since the ats field is still under-researched for abstractive summarization, it is difficult to achieve multiple comparisons among the available works. therefore, we created a translatesummarize-translate pipeline from the available pretrained state-of-the-art multi-language models such mt5 @xcite , mbart @xcite , and criss @xcite . for our experiment, we chose mt5 becasue of its wide coverage of 101 languages and support for 41 languages. the model is utilized to generate summaries of the 1,000 randomly sampled articles, and then compare them with lans ground-truth summaries using rouge-n. in a high-level description, the pipeline inputs the preprocessed samples as mentioned earlier in section 3.2.2, translates the articles (@xmath0, generates summaries from the translated articles, then translates the generated summaries (@xmath1. the model for each step of the pipeline will be given later. some of the pipeline steps to generate automatic text summaries are tuned to adapt arabic language. firstly, we preprocess the text, as detailed in section 3.2.2. secondly, we translate the articles from arabic to english. we apply opus-mt (tiedemann and thottingal, 2020) project. opus-mt is based on marian-nmt @xcite , a state-of-the-art transformer-based neural machine translation (nmt), and trained on opus data using opus-mt-train. the translation achieves accurate results in machine translation. next, since articles are translated into english, we process the articles to generate automatic text summaries using mt5 which inherits all the benefits of t5 @xcite . the automatic text summaries currently are english. finally, we translate automatic text summaries into arabic by again applying the opus-mt project as described in the second step. an example of the ground-truth summary and a generated arabic summary are displayed in both summaries are evaluated by rouge @xcite evaluation metric and will be used for human evaluation (see sec 6.2). we apply rouge-1, rouge-2, and rouge-l to consider different summary lengths. moreover, we also show how lemmatization impacts the accuracy. the results are reported in ## intrinsic evaluation of lans we apply two methods of evaluation to validate the reliability of the summaries from lans. the first is an automatic evaluation which examines the summarization techniques in lans. it uses the following metrics: compression ratio, fragment density, and coverage. the automatic evaluation has been performed on the whole dataset. the second evaluation is performed by experts which verifies the quality of lans by randomly extracting 1,000 articles and their respective summaries, which are evaluated by experts. ## conclusion this work presents lans, a large-scale and diverse text summarization dataset of more than 8 million new articles paired with their summaries written by journalists. the summaries are collected from the metadata of 22 scraped popular arab newspapers' websites from the period between 1999 to 2019. for each of those resources, lans considered a wide range of topics. the work applied two evaluation methods (automatic and human) to verify the superiority of the extracted summaries in lans. the dataset can be accessed upon request. 3 . lans offers this dataset for researchers to advance the field of ats, and takes advantage of the data to train and evaluate the results of new models on this dataset.