\mu PLAN : Summarizing using a Content Plan as Cross-Lingual Bridge (2024)

Abstract: Cross-lingual summarization aims to generate a summary in one languagegiven input in a different language, allowing for the dissemination ofrelevant content among different language speaking populations. Thetask is challenging mainly due to the paucity of cross-lingualdatasets and the compounded difficulty of summarizing andtranslating.This work presents \mu PLAN, an approach to cross-lingual summarization that uses an intermediate planning step as a cross-lingual bridge. We formulate the plan as a sequence of entities capturing thesummary’s content and the order in which it should becommunicated. Importantly, our plans abstract from surface form: usinga multilingual knowledge base, we align entities to their canonicaldesignation across languages and generate the summary conditioned onthis cross-lingual bridge and the input. Automatic and human evaluation on the XWikis dataset (across four language pairs) demonstrates that our planning objective achieves state-of-the-art performance interms of informativeness and faithfulness. Moreover, \mu PLAN modelsimprove the zero-shot transfer to new cross-lingual language pairscompared to baselines without a planning component.

## introduction given a document or multiple documents in a source language (e.g., english), cross-lingual summarization @xcite aims to generate a summary in a different target language (e.g., czech or german). it enables the rapid dissemination of relevant content across speakers of other languages. for instance, providing summaries of english news articles to czech or german speakers; or making available to english speakers the content of product and service descriptions in foreign languages. recent years have seen tremendous progress in abstractive summarization @xcite thanks to advances in neural network models and the availability of large-scale datasets @xcite @xcite . while initial efforts have focused on english, more recently, with the advent of cross-lingual representations @xcite and large pre-trained models @xcite , research on multilingual summarization (i.e., building monolingual summarization systems for different languages) has also gained momentum @xcite @xcite . cross-lingual summarization faces the compounded challenge of having to tackle difficulties relating to both monolingual summarization (e.g., long inputs and outputs, hallucinations; @xcite ) and machine translation (e.g., data imbalance, alignment across languages; @xcite . recent work has shown that introducing an intermediate content planning step is helpful for summarization in english, resulting in higher quality summaries, especially in terms of faithfulness @xcite @xcite . in this work, we argue that content planning also has the potential for producing higher quality outputs for cross-lingual summarization. in particular, it provides a way of sharing task-specific knowledge across languages, while formalizing important aspects of the summarization task: identifying salient content in the source documents, organizing this information in a meaningful order, and standardizing it across different source and target language pairs. we present µplan, a cross-lingual summarization method that uses content planning as a crosslingual bridge (figure 1 ). building upon previous work @xcite , we express our content plans as entity chains, i.e., ordered sequences of salient entities. although more elaborate plan representations have been proposed in the literature @xcite @xcite , entities are a natural choice for our task for two reasons. they can mitigate hallucinations in generated summaries which are commonly related to entities @xcite @xcite and are well-suited as a bridge across languages, thanks to the availability of multilingual knowledge bases (e.g., dbpedia) which represent entities in different languages. an interesting question for our summarization task is which language to use for the content plan, given that the source document and target summary are in different languages. we employ a multilingual knowledge base to align the entities across languages, which allows us to canonically transpose the plan to different languages without the use of machine translation. we use a transformer-based encoder-decoder model @xcite that first encodes the document in the source language and then decodes to generate an intermediate plan representation and the summary in the target language conditioned on the plan and the input. we evaluate our method on the xwikis dataset (perez-beltrachini and lapata, 2021), a cross-lingual abstractive summarization dataset derived from wikipedia 2 articles aligned across four different languages (english, czech, french, and german). we augment the training data for fine-tuning by annotating each target summary with its corresponding content plan. we investigate two distinct cross-lingual tasks, namely from english to other languages (@xmath0. we demonstrate that models finetuned with our planning objective outperform regular generated summaries both in terms of rouge and faithfulness on the xwikis dataset across all language pairs, in both settings. given the scarcity of cross-lingual datasets, we also investigate zero-2 https://www.wikipedia.org/ shot cross-lingual transfer to new language pairs and demonstrate that µplan models outperform comparison systems without planning components. our contributions can be summarized as follows: (a) we introduce a training objective for cross-lingual abstractive summarization that uses entity planning as a bridge between languages. using automatic and human evaluation, we show that it yields better quality summaries and more effective zero-shot transfer to new language pairs than non-planning baselines; and (b) we leverage a multilingual knowledge base to annotate the training data with plans, thus transposing entity names to their canonical designation in all languages, avoiding errors induced by mistranslation altogether. this strategy enables the mapping of entities that do not have an equivalent name in the target language to fully-localized paraphrases. cross-lingual benchmarks include wikilingua @xcite , a dataset derived from multilingual how-to guides, which are relatively short and their summaries limited to brief instructional sentences. crosssum @xcite contains over a million article and summary samples, aligned from the multilingual xl-sum @xcite dataset, but the summaries are limited to one or two sentences. @xcite propose a wikipedia-based cross-lingual dataset, rudolph hass southern california united states postal service hass avocado horticulture en knowledge base q5679460 q844837 q668687 q48803 q5679460 the hass avocado was first grown and sold by southern california mail carrier and amateur horticulturist rudolph hass, who also gave it his name. rudolph hass südkalifornien united states postal service hass avocado gartenbau de summary content plans figure 2 : plan annotation on an example summary (salient entities highlighted in yellow). after pivoting on the knowledge base, corresponding canonical entities in english are shown in the bottom left. most times they match the surface form in the summary (in red), other times they have the same root (in green) but they could differ greatly when entities need disambiguation (in blue). the aligned german content plan is shown in the bottom right. but it only includes the english to german language direction. we work with xwikis (perez-beltrachini and lapata, 2021), a cross-lingual dataset derived from wikipedia with long input documents and long target summaries across four languages: english, czech, french, and german. we compare these datasets in appendix a. ## conclusion in this work we present µplan, an approach to cross-lingual summarization that uses an intermediate planning step as a cross-lingual bridge. since hallucinations and mistranslations in cross-lingual summarization are often tied to incorrect entities, we formulate the content plan as a sequence of entities expressing salient content and how it should be presented. evaluation on the xwikis dataset demonstrates that this planning objective achieves state-of-the-art performance in @xmath1. in this work, we use the embedded hyperlinks in wikipedia articles to extract salient entities and align them on the wikidata knowledge base. with recent entity annotation systems such as refined @xcite , the same operation can be applied on out-of-domain data, including the multilingual alignment of the entity names. unlike latent variable-based intermediate representations, our content plans are interpretable (they are expressed in natural language) and can be easily edited, e.g., by filtering the entities at inference time or with a human in the loop @xcite @xcite . using forced prompting methods as described in the oracle experiments, would also allow us to localize entity names at inference time from a knowledge base. in the future, we plan to explore the task transfer capabilities of µplan in low-resource settings as we cannot realistically expect to have large-scale cross-lingual data on all possible language pairs. 2154 limitations an ethical consideration with generative language models is the problem of misinformation. while the work we present here makes a step towards improving the faithfulness and factual consistency of text generation systems, it is important to note that current systems are still far from perfect in this respect. they can make mistakes and thus their output should be checked and used with caution.