A Novel W ikipedia based Dataset for Monolingual and Cross-Lingual Summarization (2021)

Abstract: Cross-lingual summarization is a challenging task for which there are no cross-lingual scientific resources currently available. To overcome the lack of a high-quality resource, we present a new dataset for monolingual and cross-lingual summarization considering the English-German pair. We collect high-quality, real-world cross-lingual data from Spektrum der Wissenschaft, which publishes human-written German scientific summaries of English science articles on various subjects. The generated Spektrum dataset is small; therefore, we harvest a similar dataset from the Wikipedia Science Portal to complement it. The Wikipedia dataset consists of English and German articles, which can be used for monolingual and cross-lingual summarization. Furthermore, we present a quantitative analysis of the datasets and results of empirical experiments with several existing extractive and abstractive summarization models. The results suggest the viability and usefulness of the proposed dataset for monolingual and cross-lingual summarization.

## introduction the summarization research has recently shifted from monolingual summarization (ms) to crosslingual summarization (cls) @xcite @xcite . however, due to the absence of real cross-lingual datasets, recent cls studies @xcite @xcite are conducted on existing monolingual news datasets and off-theshelf machine translation (mt) systems which may introduce noise into pseudo-cross-lingual summarization (pcls) data. as these cls studies rely on only news data, the trained summarization models may not work well for other domains such as scientific texts. although some efforts have been made for investigating the ms task on scientific papers @xcite @xcite ; however, there is no study on scientific text for cls to date. another aspect of consideration is that most cls studies intend to generate the summaries in english from a local language but not vice versa to facilitate the local readers. this paper aims to address this issue by developing a summarization dataset containing scientific texts of the english-german language pair from two resources, spektrum der wissenschaft (spektrum) and the wikipedia science portal (wsp). the paper explores the cls task by using scientific english documents to generate german summaries for the local readers. to the best of the authors' knowledge, the collected wsp or wikipedia dataset represents the largest cls dataset of the english-german pair so far. we believe that the novel wikipedia dataset encourages new avenues of research in the less explored areas of cls. contributions: this paper has several contributions, including data collection, dataset generation, statistical analysis of the datasets, and an empirical evaluation of ms and cls. we collect our primary dataset from spektrum, consisting of 1,510 english science articles with human-written german summaries. in addition, we propose a novel scoring method, which validates the data present in the spektrum dataset before data extraction. to complement the spektrum dataset, we harvest our second dataset from wsp, containing 51,312 english and german science articles. the collection of data from two different resources ensures diversity in the written text and topics. it is worth noting that the wikipedia dataset can also be used for ms, which distinguishes it from existing datasets. we perform a detailed statistical analysis of the dataset that highlights the interesting patterns. furthermore, we conduct an empirical evaluation with several extractive baselines and existing abstractive summarization models to validate the usability of our dataset for ms and cls. moreover, linguistic quality is evaluated on a subset of the output summaries of the ms and cls experiments by human judges. ## conclusions lack of cross-lingual experimental datasets impedes the progress of cls research. in this paper, we present a new ms and cls dataset extracted from spektrum and wikipedia. our empirical investigation demonstrates the viability and amenability of the proposed dataset and also highlights the challenging nature of the dataset for recent summarization models. our results demonstrate the significance of constructing real crosslingual datasets for cls. furthermore, the english and german summaries scored reasonably well in terms of correctness and fluency based on human evaluations. we anticipate that the proposed dataset will encourage the study and research of ms and cls. in the future, we will scale the cls experiments using science domain pre-trained models.