A Survey on Cross-Lingual Summarization (2022)

Abstract: Cross-lingual summarization is the task of generating a summary in one language (e.g., English) for the given document(s) in a different language (e.g., Chinese). Under the globalization background, this task has attracted increasing attention of the computational linguistics community. Nevertheless, there still remains a lack of comprehensive review for this task. Therefore, we present the first systematic critical review on the datasets, approaches, and challenges in this field. Specifically, we carefully organize existing datasets and approaches according to different construction methods and solution paradigms, respectively. For each type of dataset or approach, we thoroughly introduce and summarize previous efforts and further compare them with each other to provide deeper analyses. In the end, we also discuss promising directions and offer our thoughts to facilitate future research. This survey is for both beginners and experts in cross-lingual summarization, and we hope it will serve as a starting point as well as a source of new ideas for researchers and engineers interested in this area.

## introduction to help people efficiently grasp the gist of documents in a foreign language, cross-lingual summarization (xls) aims to generate a summary in the target language from the given document(s) in a different source language. this task could be regarded as a combination of monolingual sum-marization (ms) and machine translation (mt), both of which are unsolved natural language processing (nlp) tasks and have been continuously studied for decades @xcite . xls is an extremely challenging task: (1) from the perspective of data, unlike ms, naturally occurring documents in a source language paired with the corresponding summaries in different target languages are rare, making it difficult to collect large-scale and human-annotated datasets @xcite ; (2) from the perspective of models, xls requires both the abilities to translate and summarize, which makes it hard to generate accurate summaries by directly conducting xls @xcite . despite its importance, xls has attracted a little attention @xcite in the statistical learning era due to its difficulties and the scarcity of parallel corpus. recent years have witnessed the rapid development of neural networks, especially the emergence of pre-trained encoder-decoder models @xcite @xcite @xcite , making neural summarizers and translators achieve impressive performance. meanwhile, creating large-scale xls datasets has proven feasible by utilizing existing ms datasets @xcite or internet resources @xcite . the aforementioned successes have laid the foundation for the xls research field and gradually attracted interest in xls. in particular, recent researchers put their efforts into solving the xls task and published more than 20 papers over the past five years. nevertheless, there still lacks a systematic review of progresses, challenges, and opportunities of xls. to fill the above gap and help new researchers, in this paper we provide the first comprehensive review of existing efforts relevant to xls and give multiple promising directions for future research. specifically, we first briefly introduce the formal definition and evaluation metrics of xls ( § 2), which serves as a strong background before delving further into xls. then, we provide an exhaustive overview of existing xls research datasets ( § 3). in detail, to alleviate the scarcity of xls data, previous work resorts to different ways to construct large-scale benchmark datasets, which are divided into synthetic datasets and multi-lingual website datasets. the synthetic datasets @xcite @xcite are constructed through (manually or automatically) translating the summaries of existing ms datasets from a source language to target languages while the multi-lingual website datasets @xcite @xcite are collected from websites that provide multi-lingual versions for their content. next, we thoroughly introduce and summarize existing models, which are organized with respect to different paradigms, namely, pipeline ( § 4) and end-to-end ( § 5). in detail, the pipeline models adopt either translate-then-summarize approaches @xcite @xcite @xcite @xcite or summarize-then-translate approaches @xcite . in this manner, the pipeline models avoid conducting xls directly, thus bypassing the model challenge we discussed previously. however, the pipeline method suffers from error propagation and recurring latency, making it not suitable for the real-world scenario @xcite . consequently, the end-to-end method has attracted more attention. to alleviate the model challenge, it generally utilizes the related tasks (e.g., ms and mt) as auxiliaries or resorts to external resources. the end-to-end models mainly fall into four categories: multi-task methods @xcite @xcite @xcite , knowledgedistillation methods @xcite @xcite , resource-enhanced methods @xcite , and pre-training methods @xcite @xcite @xcite . for each category, we will thoroughly go through the previous work and discuss the corresponding pros and cons. finally, we also point out multiple promising directions on xls to push forward the future research ( § 6), followed by conclusions ( § 7). our contributions are concluded as follows: ## datasets in this section, we review available large-scale xls datasets 6 and further divide them into two categories: synthetic datasets ( § 3.1) and multilingual website datasets ( § 3.2). for each category, we will introduce the construction details and the key characteristics of the corresponding datasets. in addition, we compare these two categories to provide a deeper understanding ( § 3.3). 1 if the source language is a programming language while the target language is a human language, the task becomes code summarization, which is beyond the scope of this survey. ## pipeline methods early xls work generally focuses on the pipeline methods whose main idea is decomposing xls into ms and mt sub-tasks, and then accomplishing them step by step. these methods can be further divided into summarize-then-translate (sum-trans) and translate-then-summarize (trans-sum) types according to the finished order of sub-tasks. for each type, we will systematically present previous methods. additionally, we compare these two types to provide deeper analyses. ## end-to-end methods though the pipeline method is intuitive, it 1) suffers from error propagation; 2) needs either a large corpus to train mt models or the monetary cost of paid mt services; 3) has a latency during inference. thanks to the rapid development of neural networks, many end-to-end xls models are proposed to alleviate the above issues. in this section, we take stock of previous endto-end xls models and further divide them into four frameworks (cf., figure 1 ): multi-task framework ( § 5.1), knowledge-distillation framework, ( § 5.2), resource-enhanced framework ( § 5.3), and pre-training framework ( § 5.4). for each framework, we will entirely introduce its core idea and rimless colored blocks denote the input or output sequences of the corresponding tasks. note that the knowledge-distillation framework might contain more than one teacher model, and the auxiliary/pre-training tasks used in the multi-task/pre-training framework are not limited to mt and ms, here we omit these for simplicity. ## prospects in this section, we discuss and suggest the following promising future directions, which meet actual application needs: the essence of xls. unifying two abilities (i.e., translation and summarization abilities) in a single model is non-trivial @xcite . even though the effectiveness of the state-of-theart models has been proved, the essence of xls remains unclear, especially (1) the hierarchical relationship between mt&ms and xls @xcite , and (2) the theoretical analysis for what makes mt&ms help xls? ## conclusion in this paper, we present the first comprehensive survey of current research efforts on xls. we systematically summarize existing xls datasets and methods, highlight their characteristics, and compare them with each other to provide deeper analyses. in addition, we give multiple perspective directions to facilitate further research on xls. we hope that this xls survey can provide a clear picture of this topic and boost the development of the current xls technologies.