ACROSS : An Alignment-based Framework for Low-Resource Many-to-One Cross-Lingual Summarization (2023)

Abstract: This research addresses the challenges of Cross-Lingual Summarization (CLS) in low-resource scenarios and over imbalanced multilingual data. Existing CLS studies mostly resort to pipeline frameworks or multi-task methods in bilingual settings. However, they ignore the data imbalance in multilingual scenarios and do not utilize the high-resource monolingual summarization data. In this paper, we propose the Aligned CROSs-lingual Summarization (ACROSS) model to tackle these issues. Our framework aligns low-resource cross-lingual data with high-resource monolingual data via contrastive and consistency loss, which help enrich low-resource information for high-quality summaries. In addition, we introduce a data augmentation method that can select informative monolingual sentences, which facilitates a deep exploration of high-resource information and introduce new information for low-resource languages. Experiments on the CrossSum dataset show that ACROSS outperforms baseline models and obtains consistently dominant performance on 45 language pairs.

## introduction given a source document, cross-lingual summarization (cls) aims to generate a summary in a different language. therefore, cls helps users quickly understand news outlines written in foreign, unknown to them, languages. early cls approaches typically use pipeline frameworks @xcite , which are intuitive but suffer from the problem of error cascading. researchers have recently turned to endto-end models @xcite @xcite that are immune to this problem. however, these studies are limited to bilingual learning and do not conform to the reality of multilingual scenarios. toyota will recall 6.5 million vehicles globally. ## related work early cls research typically used pipeline methods, such as the translate-then-summarize @xcite or summarizethen-translate methods @xcite @xcite , which are sensitive to error cascading that causes their subpar performance. thanks to the development of the transformerbased methods @xcite , researchers introduced teacher-student frameworks @xcite wherein the cls task can be approached via an encoder-decoder model. thereafter, the multi-task framework started to be popular in this field @xcite @xcite . recently, researchers have begun to investigate how to fuse translation and summarization tasks into a unified model to improve the performance on the cls tasks @xcite @xcite @xcite . for example, @xcite considered compression so that their model can handle both the cls and translation tasks at different compression rates. focusing on multi-task learning, these multi-task studies attempt to improve cls performance using machine translation (mt) and mls tasks in bilingual settings. however, such approaches still establish implicit connections among languages and leave aside the information of high-resource data. @xcite recognized the limitations of the above-mentioned scenarios. they proposed a new dataset, crosssum, in multilingual scenarios and introduced a method balancing the number of different language pairs in a batch, which could alleviate the uneven distribution of training samples and balance performance in different languages. however, deep semantical correlations across languages as well as abundant information from highresource data have not been investigated. in contrast to the aforementioned methods, across introduces cross-lingual alignment and a novel data augmentation method, which can improve low-resource performance from both model and data perspectives. the encoder and decoder sides. figure 2 illustrates the overall framework. ## conclusion in this work, we propose across, a many-toone cross-lingual summarization model. inspired by the alignment idea, we design contrastive and consistency loss for across. experimental results show that with the across framework, cls model improves the low-resource performance by effectively utilizing high-resource monolingual data. our findings point to the importance of alignment in cross-lingual fields for future research. in the future, we plan to apply this idea to combine cls in multimodal scenarios, which might enable the model to better serve realistic demands.