Models and Datasets for Cross-Lingual Summarisation (2021)

Abstract: We present a cross-lingual summarisation corpus with long documents in a source language associated with multi-sentence summaries in a target language. The corpus covers twelve language pairs and directions for four European languages, namely Czech, English, French and German, and the methodology for its creation can be applied to several other languages. We derive cross-lingual document-summary instances from Wikipedia by combining lead paragraphs and articles’ bodies from language aligned Wikipedia titles. We analyse the proposed cross-lingual summarisation task with automatic metrics and validate it with a human study. To illustrate the utility of our dataset we report experiments with multi-lingual pre-trained models in supervised, zero- and few-shot, and out-of-domain scenarios.

## introduction given a document in a source language (e.g., french), cross-lingual summarisation aims to produce a summary in a different target language (e.g., english). the practical benefits of this task are twofold: it not only provides rapid access to salient content, but also enables the dissemination of relevant content across speakers of other languages. for instance, providing summaries of articles from french or german newspapers to non-french or non-german speakers; or enabling access to summary descriptions of goods, services, or knowledge available online in foreign languages. figure 1 shows an example of an input document in french (left) and its summary in english and other languages (right). recent years have witnessed increased interest in abstractive summarisation @xcite thanks to the popularity of neural network models and the availability of datasets @xcite @xcite containing hundreds of thousands of document-summary pairs. although initial efforts have overwhelmingly focused on english, more recently, with the advent of cross-lingual representations @xcite and large pre-trained models @xcite , research on multi-lingual summarisation (i.e., building monolingual summarisation systems for different languages) has been gaining momentum @xcite . while creating large-scale multi-lingual summarisation datasets has proven feasible @xcite , at least for the news domain, cross-lingual datasets are more difficult to obtain. in contrast to monolingual summarisation, naturally occurring documents in a source language paired with summaries in different target languages are rare. for this reason, existing approaches either create large-scale synthetic data using back-translation @xcite , translate the input documents @xcite , or build document-summary pairs from social media annotations and crowd-sourcing @xcite . recent efforts @xcite have been directed at the creation of a large-scale cross-lingual dataset in the domain of how-to guides. despite being a valuable resource, how-to guides are by nature relatively short documents (391 tokens on average) and their summaries limited to brief instructional sentences (mostly commands). to further push research on cross-lingual summarisation, we propose a large dataset with document-summary pairs in four languages: czech, english, french, and german. 1 inspired by past research on monolingual descriptive summarisation @xcite @xcite @xcite , we derive crosslingual datasets from wikipedia foot_1 , which we collectively refer to as xwikis. we exploit wikipedia's interlanguage links and assume that given any two related wikipedia titles, e.g., huile d'olive (french) and olive oil (english), we can pair the the lead paragraph from one title with the body of the other. we assume that the lead paragraph can stand as the summary of the article (see figure 1 ). our dataset covers different language pairs and enables different summarisation scenarios with respect to: degree of supervision (supervised, zeroand few-shot), combination of languages (crosslingual and multi-lingual), and language resources (high-and low-resource). to illustrate the utility of our dataset we report experiments on supervised, zero-shot, few-shot, and out-of-domain cross-lingual summarisation. for the out-of-domain setting, we introduce voxeurop, a cross-lingual news dataset. 3 in experiments, following recent work @xcite , we focus on all-to-english summarisation. in addition to assessing supervised and zero-shot performance of multilingual pre-trained models @xcite , we also provide a training mechanism for few-shot cross-lingual summarisation. foot_3 ## the xwikis corpus wikipedia articles are organised into two main parts, a lead section and a body. for a given wikipedia title, the lead section provides an overview conveying salient information, while the body provides detailed information. indeed, the body is a long multi-paragraph text generally structured into sections discussing different aspects of the wikipedia title. we can thus consider the body and lead paragraph as a document-summary pair. furthermore, a wikipedia title can be associated with wikipedia articles in various languages also composed by a lead section and body. based on this insight, we propose the cross-lingual abstractive document summarisation task of generating an overview summary in a target language y from a long structured input document in a source language x. figure 1 illustrates this with an example. for the wikipedia title huile d'olive (olive oil), it shows the french document on the left and overview summaries in german, french, czech, and english on the right. below, we describe how our dataset was created, analyse its main features (section 2.1), and present a human validation study (section 2.2). ## experimental setup datasets and splits we work with the d @xmath0, d f @xmath1, and d @xmath2.e., first column in to train an english monolingual summariser, we created a monolingual dataset d @xmath3. we selected a set of wikipedia titles disjoint from those paragraph extraction to deal with very long documents, we carry out an initial extractive step @xcite . specifically, we rank document paragraphs (represented as vectors of their tf-idf values) using lexrank @xcite and then select the top ranked paragraphs up to a budget of 600 tokens. ## conclusion we presented a new summarisation dataset in four languages (german, french, czech, and english) which we hope will be a valuable resource for crosslingual and monolingual summarisation. we evaluated a wide range of models on the cross-lingual summarisation task, including zero-and few-shot variants some of which show promising results. future work directions are many and varied. we would like to further investigate maml variants for few-shot summarisation, and expand on document views for cvt (e.g., by looking at semantic roles and discourse relations). a the xwikis corpus dataset creation our corpus was created with english, german, french and czech wikipedia dumps from june 2020. 10 we adapted wikiextractor @xcite to obtain the lead section and body of wikipedia articles. we preserved the structure of the input document, and section mark-ups were kept (e.g., <@xmath4. we used a dump of the same date for the table containing the wikipedia interlanguage links. 11 we performed text normalisation (a variant of nfkc normalization) with sentence-piece @xcite .