A Deep Reinforced Model for Zero-Shot Cross-Lingual Summarization with Bilingual Semantic Similarity Rewards (2020)

Abstract: Cross-lingual text summarization aims at generating a document summary in one language given input in another language. It is a practically important but under-explored task, primarily due to the dearth of available data. Existing methods resort to machine translation to synthesize training data, but such pipeline approaches suffer from error propagation. In this work, we propose an end-to-end cross-lingual text summarization model. The model uses reinforcement learning to directly optimize a bilingual semantic similarity metric between the summaries generated in a target language and gold summaries in a source language. We also introduce techniques to pre-train the model leveraging monolingual summarization and machine translation objectives. Experimental results in both English–Chinese and English–German cross-lingual summarization settings demonstrate the effectiveness of our methods. In addition, we find that reinforcement learning models with bilingual semantic similarity as rewards generate more fluent sentences than strong baselines.

## introduction cross-lingual text summarization (xls) is the task of compressing a long article in one language into a summary in a different language. due to the dearth of training corpora, standard sequence-to-sequence approaches to summarization cannot be applied to this task. traditional approaches to xls thus follow a pipeline, for example, summarizing the article in the source language followed by translating the summary into the target language or viceversa @xcite . both of these approaches require separately trained summarization and translation models, and suffer from error propagation @xcite . prior studies have attempted to train xls models in an end-to-end fashion, through knowledge distillation from pre-trained machine translation (mt) or monolingual summarization models @xcite , but these approaches have been only shown to work for short outputs. alternatively, @xcite proposed to automatically translate source-language summaries in the training set thereby generating pseudo-reference summaries in the target language. with this parallel dataset of source documents and target summaries , an end-to-end model is trained to simultaneously summarize and translate using a multi-task objective. although the xls model is trained end-to-end, it is trained on mt-generated reference translations and is still prone to compounding of translation and summarization errors. in this work, we propose to train an end-to-end xls model to directly generate target language summaries given the source articles by matching the semantics of the predictions with the semantics of the source language summaries. to achieve this, we use reinforcement learning (rl) with a bilingual semantic similarity metric as a reward @xcite . this metric is computed between the machine-generated summary in the target language and the gold summary in the source language. additionally, to better initialize our xls model for rl, we propose a new multi-task pretraining objective based on machine translation and monolingual summarization to encode common information available from the two tasks. to enable the model to still differentiate between the two tasks, we add task specific tags to the input @xcite . we evaluate our proposed method on english-chinese and english-german xls test sets. these test corpora are constructed by first using an mtsystem to translate source summaries to the target language, and then being post-edited by human annotators. experimental results demonstrate that just using our proposed pre-training method without fine-tuning with rl improves the bestperforming baseline by up to 0.8 rouge-l points. applying reinforcement learning yields further improvements in performance by up to 0.5 rouge-l points. through extensive analyses and human evaluation, we show that when the bilingual semantic similarity reward is used, our model generates summaries that are more accurate, longer, more fluent, and more relevant than summaries generated by baselines. ## model in this section, we describe the details of the task and our proposed approach. ## results the main results of our experiments are summarized in applying reinforcement learning to fine-tune the model with both rouge (rl-rouge), xsim (rl-xsim) or their mean (rl-rouge+xsim) as rewards results in further improvements. our proposed method, rl-xsim performs the best overall, indicating the importance of using cross-lingual similarity as a reward function. rl-rouge uses a machine-generated reference to compute the rewards since target language summaries are unavailable, which might be a reason for its worse performance. ## related work most previous work on cross-lingual text summarization utilize either the summarize-then-translate or translate-then-summarize pipeline @xcite @xcite , prior work in monolingual summarization have explored hybrid extractive and abstractive summarization objectives which inspires our distillation objective @xcite @xcite . this line of research mainly focus on either compressing sentences extracted by a pre-trained model or biasing the prediction towards certain words. language generation models trained with crossentropy using teacher-forcing suffer from exposure bias and a mismatch between training and evaluation objective. to solve these issues, using reinforcement learning to fine-tune such models have been explored for monolingual summarization where rouge rewards is typically used @xcite @xcite . other rewards such as bert score @xcite have also been explored @xcite . computing such rewards requires access to the gold summaries which are typically unavailable for cross-lingual summarization. this work is the first to explore using cross-lingual similarity as a reward to work around this issue. ## conclusion in this work, we propose to use reinforcement learning with a bilingual semantic similarity metric as rewards for cross-lingual document summarization. we demonstrate the effectiveness of the proposed approach in a resource-deficient setting, where target language gold summaries are not available. we also propose simple strategies to better initialize the model towards reinforcement learning by leveraging machine translation and monolingual summarization. in future work, we plan to explore methods for stabilizing reinforcement learning as well to extend our methods to other datasets and tasks, such as using the bilingual similarity metric as a reward to improve the quality of machine translation.